Retail Brand Optimizes Creative Testing with AI

Challenge

A European fashion e-commerce brand was spending €120K/month on Meta Ads.

They knew creative was critical.

Everyone says: "Creative is 80% of performance."

So they tested. A lot.

The testing process:

Every week:

• Brief designers: 10 new ad concepts
• Designers create: 3-4 days
• Launch test: Friday
• Run for: 7 days
• Analyze results: Monday
• Decide winners: Tuesday
• Scale winners: Wednesday

Repeat forever.

The problems:

1. Testing was expensive

• Running 40-50 ad variations simultaneously
• Split budget across all tests
• Many tests inconclusive (not enough data)
• Expensive to produce creative that might fail

2. Analysis was subjective

• Designer A's ads seem to work better (or do they?)
• Videos outperform static (except when they don't)
• Lifestyle beats product shots (sometimes)
• No clear patterns

3. Decisions took forever

• Week to test
• 2-3 days to analyze
• Meetings to decide
• Finally scale (10+ days later)

4. Winners stopped working

• Scale a winning ad
• Performs great for 2 weeks
• Then dies
• Start over

The team was exhausted.

Creative treadmill. Always testing. Never confident. Unclear what actually works.

The creative team was frustrated:

"We make 100 ads per month. Maybe 5 are winners. Huge waste."

The CMO was concerned:

"We're spending €120K/month but don't know what good creative looks like."

Solution

They connected Cogny to understand creative performance.

Setup: 10 minutes

• Meta Ads connected
• Historical creative data imported
• AI began analysis

72 Hours Later: First Creative Insights

AI analyzed 840 ad variations from past 6 months.

Found patterns humans couldn't see.

Key AI Findings

Finding #1: Creative Lifespan is 16 Days

AI Analysis: "Winning creative performs optimally for 16 days average.

After day 16:

• CTR drops 35%
• CPA increases 48%
• ROAS declines 31%

Currently running winning ads for 30+ days.

Performance degradation hiding in aggregate metrics."

Recommendation: Rotate creative every 16 days. Have replacement ready before fatigue sets in.

Impact: Systematic refresh schedule prevents performance cliffs.

Finding #2: Testimonial + Product Format Wins

AI Analysis: "Analyzed creative performance across 840 variations.

Format breakdown:

• Testimonial + Product: 6.2 ROAS average (Best)
• Lifestyle only: 4.1 ROAS
• Product only: 3.8 ROAS
• Behind-the-scenes: 3.2 ROAS

Testimonial + Product variations:

• 18% of all creative
• 42% of total conversions
• 2.3x better than average"

Recommendation: Increase production of testimonial + product format from 18% to 40% of creative.

Impact: Focus creative resources on proven format.

Finding #3: Video Length Matters (But Not How They Thought)

AI Analysis: "Video performance by length:

6-15 seconds:

• Great CTR (4.2%)
• Poor conversion rate (1.8%)
• Low ROAS (3.1)

15-30 seconds:

• Good CTR (3.1%)
• Great conversion rate (3.9%)
• High ROAS (6.4)

30-60 seconds:

• Low CTR (1.8%)
• OK conversion rate (2.4%)
• Medium ROAS (4.1)

Sweet spot: 15-30 seconds Captures attention + provides enough info to convert."

Recommendation: Focus video production on 15-30 second format.

Impact: Stop producing 6-second and 60-second variations.

Finding #4: Designer Performance Patterns

AI Analysis: "Designer A's ads:

• Average ROAS: 5.8
• Style: Minimal, clean, product-focused with testimonial
• Success rate: 38% become winners

Designer B's ads:

• Average ROAS: 3.9
• Style: Busy, multiple products, lifestyle
• Success rate: 12% become winners

Designer C's ads:

• Average ROAS: 4.7
• Style: Bold colors, single product, clear CTA
• Success rate: 28% become winners"

Recommendation:

• Brief designers with winning patterns
• Designer A's minimalist style drives best results
• Shift 60% of creative to Designer A's approach

Impact: Higher hit rate on new creative.

Finding #5: Color Psychology Actually Matters

AI Analysis: "Ads with specific color patterns perform differently:

Earth tones (beige, brown, cream):

• ROAS: 6.1
• Audience: 25-45 age range
• Best for: Premium products

Vibrant colors (red, yellow, bright blue):

• ROAS: 4.2
• Audience: 18-30 age range
• Best for: Sale items

Monochrome (black, white, grey):

• ROAS: 5.3
• Audience: 30-50 age range
• Best for: Basics/essentials"

Recommendation: Match color palette to product category and target audience.

Impact: Strategic color choices instead of designer preference.

Finding #6: CTA Text Matters More Than Placement

AI Analysis: "CTA variations tested:

'Shop Now': 4.1 ROAS 'Discover More': 5.2 ROAS 'See Details': 3.8 ROAS 'Limited Time': 6.8 ROAS (WINNER) 'New Arrival': 5.9 ROAS

CTA placement (button vs text) had minimal impact. CTA wording drives performance."

Recommendation: Use "Limited Time" or "New Arrival" CTAs. Stop testing placement, focus on copy.

Impact: Better CTAs without designer time on placement testing.

Implementation

Week 1: New Creative Strategy

Based on AI insights, they developed new strategy:

Creative Production:

• 60% testimonial + product format
• 30% lifestyle with product
• 10% experimental

Video specs:

• All videos: 15-30 seconds
• Stop producing 6-sec and 60-sec

Design approach:

• Follow Designer A's minimal style as baseline
• Brief designers with color strategy
• Use "Limited Time" / "New Arrival" CTAs

Testing process:

• Launch 12 new variations per week (down from 40)
• Focus on proven formats
• Higher production quality (more time per ad)

Rotation schedule:

• Refresh every 16 days
• Have 3 variations per ad set ready
• Systematic rotation calendar

Weeks 2-8: Execute New Strategy

Creative production changed:

• From: 40 variations/week (10 winners)
• To: 12 variations/week (7 winners)
• Hit rate: 25% → 58%

Why higher hit rate:

• Following proven patterns
• Focus on what works
• Better briefs
• More time per creative

Performance improved across the board.

Results

After 8 Weeks

Creative Efficiency:

• Before: 40 new ads/week, 10 winners (25% hit rate)
• After: 12 new ads/week, 7 winners (58% hit rate)
• Result: 2.3x better hit rate, 70% less production

Campaign Performance:

• Before: 4.1 ROAS average
• After: 5.8 ROAS average
• Improvement: 41% ROAS increase

CAC:

• Before: €48
• After: €33
• Improvement: 31% reduction

Creative Lifespan Management:

• Before: Ads ran until performance tanked
• After: Systematic 16-day rotation
• Result: No more performance cliffs

Designer Productivity:

• Before: Constant pressure to produce volume
• After: Focus on quality, proven formats
• Result: Happier team, better creative

Testing Costs:

• Before: €30K/month on testing unproven formats
• After: €10K/month on focused tests
• Savings: €20K/month redirected to winners

What This Meant for the Business

Time Savings:

• Creative team: 40 ads/week → 12 ads/week
• Analysis time: 8 hours/week → 2 hours (AI does it)
• Decision making: 2-3 days → same day
• Total: ~50 hours/week saved

Budget Efficiency:

• Same €120K/month spend
• 41% better ROAS
• €140K more revenue per month
• Annual impact: €1.68M additional revenue

Creative Team Satisfaction:

• Before: Burning out on volume
• After: Proud of quality work
• Retention improved

Strategic Clarity:

• Clear formula for winning creative
• Confident in production decisions
• No more guessing

Key Insights from AI

1. Patterns Exist, Humans Just Can't See Them

840 ad variations over 6 months.

Humans saw: "Some ads work, some don't, no clear pattern"

AI saw: Clear patterns across format, length, color, copy, design style.

The patterns were there all along. Just buried in too much data.

2. Creative Fatigue is Predictable

Team thought fatigue was random.

"This ad stopped working... not sure why..."

AI found:

• Happens consistently around day 16
• Predictable CTR decline curve
• Can plan rotation schedule

Creative fatigue isn't random. It's mechanical. Therefore manageable.

3. Testing Everything is Wasteful

They tested 40 variations/week. Most failed.

AI showed them:

• What formats work
• What lengths work
• What styles work

Better: Test within proven frameworks.

Not "does video work?" (yes) But "which type of video in proven format works best?"

Narrower tests = higher hit rate.

4. Creative Guidance Beats Creative Freedom

Designers wanted total freedom.

"We're artists, don't constrain us!"

But given constraints (testimonial + product, 15-30 sec, minimal style):

• They produced better work
• Hit rate increased 2.3x
• Less frustration (clearer goals)

Constraints based on data = creativity enabler.

5. Volume Doesn't Equal Results

40 ads/week felt productive.

But 75% failed.

12 ads/week felt like less work.

But 58% succeeded.

Better to do less, better, based on what works.

What The Team Said

"We were testing blind. Making 100 ads per month hoping something worked. AI showed us exactly what works. Now we produce 30 ads per month and get better results."

— Creative Director

"The 16-day rotation insight was game-changing. We were running winning ads until they died. Now we rotate before fatigue. Performance stays high."

— Campaign Manager

"I was skeptical about AI telling us creative strategy. But the patterns were undeniable. Testimonial + product in 15-30 seconds just works. Now we lean into it."

— Head of Creative

"Best investment we made this year. €15K/year for Cogny, €1.68M additional revenue. 112x ROI."

— CMO

Lessons Learned

1. Test Smarter, Not More

More tests ≠ better results.

Better tests = better results.

Instead of:

• Test everything
• Hope something works
• 25% hit rate

Do:

• Test within proven patterns
• Data-informed hypotheses
• 58% hit rate

2. Creative is Science AND Art

Art: The execution, design, emotion Science: The format, timing, structure

Marry both:

• AI provides the science (what formats work)
• Designers provide the art (beautiful execution)

Neither alone is enough.

3. Rotation Beats Optimization at Scale

They were trying to optimize every ad.

Better approach:

• Find winners
• Run for 16 days
• Rotate to fresh creative

Rotation maintains performance. Optimization has diminishing returns.

4. Designer Constraints Increase Hit Rate

Give designer free rein:

• They test everything
• Most fails
• Frustrating for everyone

Give designer data-based guardrails:

• They create within proven formats
• Higher success rate
• Everyone happier

Data constraints = creative enabler.

5. Production Volume is Vanity Metric

"We produce 100 ads/month!" sounds impressive.

But if only 25 work, you're wasting resources.

Better metric: Hit rate.

30 ads/month with 58% hit rate > 100 ads/month with 25% hit rate.

Replicability

This result is replicable if you have:

✅ Heavy creative testing (20+ new ads/month) ✅ 6+ months of performance data (AI needs data to find patterns) ✅ Meta/TikTok/Video focus (where creative matters most) ✅ Unclear what works (no clear creative formula) ✅ Creative team bandwidth (want to produce less but better)

Less applicable if:

• Very small creative volume (<10 ads/month)
• Brand-focused (not performance)
• Short track record (need history)

Typical timeline:

• Week 1: AI analyzes creative history
• Week 2-4: Implement new strategy
• Month 2-3: See improved hit rate and ROAS

Expected outcomes:

• 1.5-2.5x creative hit rate improvement
• 20-50% ROAS improvement
• 30-50% reduction in creative production volume
• Better creative team morale

Want to Know What Creative Actually Works?

Most brands test endlessly.

Never find clear patterns.

AI analyzes all your creative performance and tells you:

• What formats work best
• Optimal video length
• Design styles that convert
• When to rotate creative
• Color strategies that perform
• CTA copy that drives results

Then you produce less creative. But it works better.

See your creative patterns:

Schedule a demo

We'll analyze your creative history and show you:

• What's working (and why)
• Patterns you're missing
• Optimal creative specs
• Expected hit rate improvement

Usually reveals 3-5 clear patterns. Most brands never see them.

---

About This Case Study

Written by the Cogny team—built by the founders who created AI optimization systems for Netflix, Zalando, and Momondo at Campanja, and scaled growth for Kry, Epidemic Sound, and Yubico through GrowthHackers.se.

Company details anonymized to protect client confidentiality. Creative performance patterns verified and representative of typical outcomes.

Last Updated: January 3, 2025