--- title: Judging creatives — spend and cold-audience results over in-platform ROAS slug: judging-creatives-by-spend-not-roas summary: Why Barry Hott and Savannah Sanchez judge new ads first by whether the system spends on them, why high-spend / low-ROAS ads are often the ones reaching cold audiences (AppLovin — users see ~8 ads before buying), and how to use Meta's audience-segment breakdown (new / engaged / existing) and the incremental-attribution column to stop crowning ads that just harvest 1-day-view credit from existing customers. domain: marketing tags: [ad-creative, creative-analysis, attribution, meta-ads, applovin, audience-segments, roas] related: [creative-hit-rate-and-spend-tiers, creative-metrics-hook-hold-ctr, ad-relevance-chain, applovin-ad-creative, last-click-attribution-biases, incrementality-test-design, meta-ads-manager-metrics-setup, meta-weekly-cadence-and-attribution, meta-scaling-winners, meta-push-delivery-and-creative-testing-tool] sources: [yt:Hh4zkUdRVzg, yt:u4B0GGBBDXY, yt:I0SPikVQdYs, yt:qoOBIPWH_qw] updated: 2026-10-24 --- # Judging creatives by spend, not in-platform ROAS ## TL;DR - First signal of a winner: **the system wants to spend on it.** Set up the account so it spends on promising ads and not on duds. Then you don't need to force budget onto every new ad.[^1][^2] - **Don't punish high-spend ads with weak ROAS/CPA.** The algorithm is usually taking them to colder, bigger audiences, which lowers account costs and helps the warm-audience ads perform.[^3] - Break results down by **audience segment** (new / engaged / existing customers). Pick and brief creative for what wins with the **coldest** audience, not the highest-ROAS row.[^4] - On AppLovin, users see about **8 different ads** before buying. The first, high-spend, "lowest-ROAS" ad still did work, so be patient and don't kill early.[^5] ## Details **Barry's caution.** Sometimes Meta does spend on attention-grabbers that drive no action. These days he rarely sees it, and he'd still ask *why* the system is spending before cutting.[^3] He looks mainly at spend: so many hidden variables (a comment thread, something in the top corner of the frame) drive an ad that dissecting it before launch isn't worth it.[^6] **Audience segments breakdown.** Define *existing customers* and *engaged audience* at ad-account level. The breakdown then shows spend and revenue by new / engaged / existing in every sales campaign, not only Advantage+ shopping.[^7] In one account Barry audited (7-day click + 1-day view, no customer exclusion): - Reported: $82K spend → $178K revenue (2.17 ROAS), but **$75K of it was 1-day view**. Adding *1-day engaged view* (10 s or 97% watched) cut that to $7K.[^8][^9] - Engaged audience: reported 3.2 ROAS vs **1.2** on incremental attribution. More sales were credited to view than to click, which he calls a red flag.[^10] - Existing customers (~8% of budget): reported 4.21 ROAS vs 1.58 (1-day click), 1.99 (7-day click), **~1.94** incremental.[^11] - New audiences: incremental sat between 1-day click and 7-day click, only ~0.55 below the reported number.[^12] The highest-ROAS cell in the grid was 1-day view on existing customers. Optimising toward it produces "ads that are just really good at taking credit for sales" that would happen anyway.[^13] **Meta's incremental attribution column** (rolled out quietly in spring 2025): Barry reads it as landing between 1-day and 7-day click. Use it as a sanity check, not as gospel.[^14] **What to do.**[^15][^16][^17] - Optimise on 7-day or 1-day click, exclude existing customers (or target them in a separate controlled campaign), and stop optimising for 1-day view. - Apply a business **multiplier** to click-attributed revenue: shares, off-platform word of mouth and later email activation never show in Ads Manager. Matching Shopify one-to-one shouldn't be the goal. - Rank and brief creative by results in the *new-audience* segment. **Savannah's routine.** On AppLovin she lets new creative "marinate" for weeks, works in one-month test sprints and watches which ads the platform pushes spend to, before looking at CPA/ROAS.[^18] On Meta and AppLovin alike: if a new concept gets no spend, treat it as a flop and don't force budget onto it ("trust the robots").[^19] ## Disagreements / caveats - Spend-as-signal assumes Meta/AppLovin allocate well. Barry concedes Meta "can get this wrong" with attention-bait.[^3] Dara Denney's diagnostic notes still pair spend with thumbstop/CTR/CVR ([[creative-metrics-hook-hold-ctr]]). - Ovadia Lebaton (The Perfect Jean) judges agency work partly on taste and admits being wrong about ads that later spent $500K.[^20] - The incremental column is Meta's own model. Validate it with a real holdout ([[incrementality-test-design]]). ## See also - [[creative-hit-rate-and-spend-tiers]] · [[last-click-attribution-biases]] · [[meta-ads-manager-metrics-setup]] - [[people/barry-hott]] · [[people/savannah-sanchez]] [^1]: yt:Hh4zkUdRVzg @ 07:42 — Barry Hott [^2]: yt:Hh4zkUdRVzg @ 06:25 — Barry Hott [^3]: yt:Hh4zkUdRVzg @ 13:49 — Barry Hott [^4]: yt:u4B0GGBBDXY @ 11:55 — Barry Hott [^5]: yt:I0SPikVQdYs @ 33:52 — Savannah Sanchez [^6]: yt:qoOBIPWH_qw @ 24:33 — Barry Hott [^7]: yt:u4B0GGBBDXY @ 01:38 — Barry Hott [^8]: yt:u4B0GGBBDXY @ 02:31 — Barry Hott [^9]: yt:u4B0GGBBDXY @ 03:24 — Barry Hott [^10]: yt:u4B0GGBBDXY @ 07:32 — Barry Hott [^11]: yt:u4B0GGBBDXY @ 08:17 — Barry Hott [^12]: yt:u4B0GGBBDXY @ 05:46 — Barry Hott [^13]: yt:u4B0GGBBDXY @ 12:42 — Barry Hott [^14]: yt:u4B0GGBBDXY @ 00:00 — Barry Hott [^15]: yt:u4B0GGBBDXY @ 11:08 — Barry Hott [^16]: yt:u4B0GGBBDXY @ 16:37 — Barry Hott [^17]: yt:u4B0GGBBDXY @ 17:28 — Barry Hott [^18]: yt:I0SPikVQdYs @ 42:22 — Savannah Sanchez [^19]: yt:I0SPikVQdYs @ 54:19 — Savannah Sanchez [^20]: yt:qoOBIPWH_qw @ 23:42 — Ovadia Lebaton