---
title: Judge paywall tests on net revenue after refunds (matured cohorts)
slug: net-revenue-after-refunds-experiment-metric
summary: Trial-start lifts from forcing tactics (no X, undismissable discount) can be wiped out by refunds — BoldVoice's refunds rose more than trial starts — so score paywall experiments on net revenue after refunds per exposed user on a matured cohort (trial + ~1 week + ≥2 weeks).
domain: mobile-apps
tags: [monetization, experiments, metrics, refunds, paywall]
related: [hard-vs-soft-paywalls, revenuecat-charts-and-experiments, subscription-conversion-benchmarks, paywall-exit-offers, removing-the-free-trial, web2app-dark-patterns-and-compliance]
sources: [yt:-5zgc14t2gc, yt:L-W6tm-Yk5E, yt:24QMRDXck70]
updated: 2026-10-24
---

# Judge paywall tests on net revenue after refunds

## TL;DR
- **Gold-standard metric (BoldVoice): net revenue after refunds per exposed user, on a matured cohort.** Not trial starts, not even initial conversions.
- Maturity rule of thumb: trial length (7 days) **+ ~1 week** for conversion to settle **+ at least 2 more weeks** for refunds.
- BoldVoice tested (a) no X on the paywall and (b) an X that led to an undismissable discount offer. Both lifted trial starts — and **refunds spiked by more than the trial gain**.
- Anything that forces or tricks people into a trial "you'll pay for in refunds" — plus angry users and negative sentiment.
- Also check failed payments: some "conversions" never actually bill.
- Top-of-funnel metrics reach significance first, which is exactly why they're tempting and misleading.

## Details
**What happened.** Anada Lakra's team ran two hard-paywall variants: removing the X so users had to start a trial to proceed, and keeping the X but following it with an X%-off offer that couldn't be closed. Users had already invested in onboarding, so trial starts looked great and the team was ready to roll out. Once data matured, refunds rose by far more than trial starts did; neither shipped.[^1][^2] BoldVoice now judges every paywall test on net revenue after refunds and waits a few weeks before deciding.[^3]

**Why a matured cohort.** Refunds don't come on day one; people request them after seeing the charge on their bank statement. So for a trial that starts on date X: allow ~1 extra week for conversion and at least 2 extra weeks for refunds to settle.[^4] Anada found that across many tests the bottom-line delta of copy/button "hacks" was small and sometimes negative even when trial-start rate rose; the start-rate metric gets the most traffic and hits p-values fastest.[^5][^6]

**Tooling.** David Barnard notes RevenueCat Experiments expanded its look-back window (to over a year) so you can re-check results at 3–6+ months; the hard part is the patience to sit on a promising result.[^7][^8]

**Corroboration.** Vahe Bagdasarian warns that aggressive trial framing (e.g. hiding plan length, "try free" cards) in a low trial-to-paid product can raise refunds or lower install-to-paid — test it and watch downstream.[^9]

**Product beats paywall.** Anada's biggest "fail of the year" was over-optimising every paywall word and button; once the paywall is sensible, work on product value. David adds: if trial cancellation is high, more onboarding screens won't fix it — users aren't finding value after the paywall.[^10][^11]

## Disagreements / caveats
- This contrasts with Steve P. Young's hard-paywall wins ([[hard-vs-soft-paywalls]]), which were judged mainly on trial starts and revenue; refund rates weren't reported there — check them before copying.
- Refund sensitivity varies by price point and audience; BoldVoice is ~$150/year with an adult professional audience.

## See also
- [[revenuecat-charts-and-experiments]]
- [[hard-vs-soft-paywalls]]
- [[people/anada-lakra]]

[^1]: yt:-5zgc14t2gc @ 22:34 — Anada Lakra
[^2]: yt:-5zgc14t2gc @ 23:22 — Anada Lakra
[^3]: yt:L-W6tm-Yk5E @ 06:21 — David Barnard
[^4]: yt:-5zgc14t2gc @ 20:12 — Anada Lakra
[^5]: yt:-5zgc14t2gc @ 17:50 — Anada Lakra
[^6]: yt:-5zgc14t2gc @ 19:25 — Anada Lakra
[^7]: yt:-5zgc14t2gc @ 21:00 — David Barnard
[^8]: yt:-5zgc14t2gc @ 21:48 — Anada Lakra
[^9]: yt:24QMRDXck70 @ 39:27 — Vahe Bagdasarian
[^10]: yt:-5zgc14t2gc @ 1:00:10 — Anada Lakra
[^11]: yt:-5zgc14t2gc @ 16:12 — David Barnard