---
title: How often to run paywall tests — and why a 5% ARPU lift can be your whole ad margin
slug: paywall-test-velocity-and-arpu-margin
summary: Cal AI ran ~4 paywall tests a week (~300 in 18 months), single tests worth $10–50K/month, starting from ~100 users/day; Gleam argues a 5–6% ARPU lift can be the entire margin on paid ads and unlocks channels (and acquirers); treat the paywall as a never-finished marketing asset.
domain: mobile-apps
tags: [monetization, paywall, ab-testing, arpu, ltv, paid-ads]
related: [revenuecat-charts-and-experiments, net-revenue-after-refunds-experiment-metric, paywall-design-learnings-at-scale, product-experiment-process-okrs-and-activation, app-break-even-cpi-and-revenue-per-install, three-screen-pre-paywall-sequence]
sources: [yt:OxE79rpGioI, yt:s8Rped5LuWw, yt:_KilixPWX-c]
updated: 2026-10-24
---

# Paywall test velocity and the ARPU margin

## TL;DR
- **Always have 1–2 paywall tests live.** Cal AI ran ~4 a week for ~75 weeks (~300 tests); not having one live counted as "dropping the ball".[^1]
- Single tests were worth **+$10K to +$50K revenue a month** and took **20–30 minutes** to launch.[^2]
- **Start testing at ~100 new users a day** (Jake Castillo's threshold).[^3]
- Why it matters: apps win by **outspending competitors per customer**, and LTV sets what you can spend.[^4]
- On paid ads margins are thin (~0.9–1.0× ROAS in 30 days), so a **5–6% ARPU lift can be the entire margin** — and decides whether paid can scale at all.[^5][^6]
- **Big levers, not cosmetics**: Gleam's ugly paywalls converted about the same as pretty ones; 1-step → 3-step paywall and pushing yearly were the big wins ([[three-screen-pre-paywall-sequence]], [[weekly-vs-yearly-plans-and-trial-placement]]).[^7]

## Details
**Monetization unlocks distribution ([[people/charles-liu]]).** Some apps (his example: commoditised screen-time apps) can't find a viral organic format but have a problem-aware audience reachable by ads — ads need both creative and ARPU. Many acquirers are studios that grow through paid, so paid-ready unit economics also matter for an exit.[^5][^8] He wishes Gleam had run proper A/B tests earlier to separate paywall effects from changes in traffic mix.[^6]

**Not just the paywall.** Fixing the paywall alone rarely lifts ARPU: ad promise, onboarding (agitate the problem, cost of inaction, aspiration) and paywall must be congruent; info-product funnels are a good model for multi-step onboarding.[^9]

**Adapty's view** ([[people/freddy-fang]]): in its analysis, apps that test more reach higher LTV (no data shown).[^3]

**Paywall as marketing (2022, [[people/dan-burcaw]]).** Teams polish store pages and demo videos, then show an unmarketed, intimidating paywall — mobile paywalls lag the web. Make it controllable by non-developers (CMS) and treat it as a canvas "that's never finished"; changes only affect new subscribers, so it's lower-risk than a product change. Many apps still carry prices from years ago.[^10][^11][^12]

## Disagreements / caveats
- High velocity needs enough traffic per variant; at 100 users/day a test takes weeks to read and small lifts are noise — judge on matured net revenue ([[net-revenue-after-refunds-experiment-metric]]).
- "$50K from one test" is Jake's recollection at Cal AI's scale (millions of downloads).[^2]
- Vendors (Adapty, Nami) sell testing tools.

## See also
- [[revenuecat-charts-and-experiments]]
- [[product-experiment-process-okrs-and-activation]]

[^1]: yt:OxE79rpGioI @ 39:39 — Jake Castillo
[^2]: yt:OxE79rpGioI @ 40:31 — Jake Castillo
[^3]: yt:OxE79rpGioI @ 40:31 — Jake Castillo, Freddy Fang
[^4]: yt:OxE79rpGioI @ 41:28 — Jake Castillo
[^5]: yt:s8Rped5LuWw @ 35:16 — Charles Liu
[^6]: yt:s8Rped5LuWw @ 38:21 — Charles Liu
[^7]: yt:s8Rped5LuWw @ 39:19 — Kelvin Zhang
[^8]: yt:s8Rped5LuWw @ 36:49 — Charles Liu
[^9]: yt:s8Rped5LuWw @ 40:54 — Charles Liu
[^10]: yt:_KilixPWX-c @ 29:09 — Dan Burcaw
[^11]: yt:_KilixPWX-c @ 36:13 — Dan Burcaw
[^12]: yt:_KilixPWX-c @ 37:04 — Dan Burcaw