SSubida Skills

Esc to close · results from all notes, skills and people

Mobile apps›Monetization

Judge paywall tests on net revenue after refunds (matured cohorts)

Trial-start lifts from forcing tactics (no X, undismissable discount) can be wiped out by refunds — BoldVoice's refunds rose more than trial starts — so score paywall experiments on net revenue after refunds per exposed user on a matured cohort (trial + ~1 week + ≥2 weeks).

Updated 2026-10-24·3 sources·monetizationexperimentsmetricsrefundspaywall

Judge paywall tests on net revenue after refunds

TL;DR

  • Gold-standard metric (BoldVoice): net revenue after refunds per exposed user, on a matured cohort. Not trial starts, not even initial conversions.
  • Maturity rule of thumb: trial length (7 days) + ~1 week for conversion to settle + at least 2 more weeks for refunds.
  • BoldVoice tested (a) no X on the paywall and (b) an X that led to an undismissable discount offer. Both lifted trial starts — and refunds spiked by more than the trial gain.
  • Anything that forces or tricks people into a trial “you’ll pay for in refunds” — plus angry users and negative sentiment.
  • Also check failed payments: some “conversions” never actually bill.
  • Top-of-funnel metrics reach significance first, which is exactly why they’re tempting and misleading.

Details

What happened. Anada Lakra’s team ran two hard-paywall variants: removing the X so users had to start a trial to proceed, and keeping the X but following it with an X%-off offer that couldn’t be closed. Users had already invested in onboarding, so trial starts looked great and the team was ready to roll out. Once data matured, refunds rose by far more than trial starts did; neither shipped.12 BoldVoice now judges every paywall test on net revenue after refunds and waits a few weeks before deciding.3

Why a matured cohort. Refunds don’t come on day one; people request them after seeing the charge on their bank statement. So for a trial that starts on date X: allow ~1 extra week for conversion and at least 2 extra weeks for refunds to settle.4 Anada found that across many tests the bottom-line delta of copy/button “hacks” was small and sometimes negative even when trial-start rate rose; the start-rate metric gets the most traffic and hits p-values fastest.56

Tooling. David Barnard notes RevenueCat Experiments expanded its look-back window (to over a year) so you can re-check results at 3–6+ months; the hard part is the patience to sit on a promising result.78

Corroboration. Vahe Bagdasarian warns that aggressive trial framing (e.g. hiding plan length, “try free” cards) in a low trial-to-paid product can raise refunds or lower install-to-paid — test it and watch downstream.9

Product beats paywall. Anada’s biggest “fail of the year” was over-optimising every paywall word and button; once the paywall is sensible, work on product value. David adds: if trial cancellation is high, more onboarding screens won’t fix it — users aren’t finding value after the paywall.1011

Disagreements / caveats

  • This contrasts with Steve P. Young’s hard-paywall wins (Hard vs soft paywalls), which were judged mainly on trial starts and revenue; refund rates weren’t reported there — check them before copying.
  • Refund sensitivity varies by price point and audience; BoldVoice is ~$150/year with an adult professional audience.

See also

Footnotes

  1. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 22:34 — Anada Lakra ↩

  2. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 23:22 — Anada Lakra ↩

  3. The case for (and against) hard paywalls @ 06:21 — David Barnard ↩

  4. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 20:12 — Anada Lakra ↩

  5. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 17:50 — Anada Lakra ↩

  6. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 19:25 — Anada Lakra ↩

  7. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 21:00 — David Barnard ↩

  8. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 21:48 — Anada Lakra ↩

  9. Our No-Code Apps Made $50M Using THIS Paywall @ 39:27 — Vahe Bagdasarian ↩

  10. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 1:00:10 — Anada Lakra ↩

  11. Why BoldVoice charged from day 1 and ignored the Duolingo freemium playbook — Anada Lakra @ 16:12 — David Barnard ↩