Deciding what to build — OKRs, analytics, A/B tests

TL;DR

• Symmetry went through three phases: build what we want (terrible) → build what users request (mediocre) → methodical experiments (works).¹
• Prioritise by reach × impact: a dark/light theme touches ~2% of DAU; the paywall touches everyone. Find the few high-leverage screens (for Symmetry: exercise search drives activation and retention; the workouts home screen doesn’t).²
• Quarterly OKRs, ambitious when young — +10% conversion in 90 days is easy a week after launch, hard for Netflix. Review metrics weekly or daily.³
• Track downloads, activation, D7/D30 retention, ARPU (per download, varies by country — it caps what you can pay creators).⁴
• Track every onboarding screen; analytics tells you where users drop, interviews/Discord/Reddit tell you why.⁵
• Problem → hypothesis → experiment, with expected effect size and guardrail metrics; A/B test everything for ≥2 weeks.⁶

Details

The process (Mauro (Symmetry)).

1. Start from a problem, not a solution: what problem, how do we know it’s real?⁶
2. Write several hypotheses/solutions; estimate how much each moves the target metric; list guardrails.⁶
3. Run control vs variant(s) at least two weeks, then analyse.⁷
4. Log everything in a database (“Evelyn” — Experiment Velocity Engine) fed by user interviews; experiments run in PostHog, paywall tests in Superwall.⁷⁸

Results he showed. Two very different redesigns of the post-onboarding workouts screen: zero movement — “let’s not focus here again”. One added screen at the end of onboarding where users commit/sign to working out: +3% retention.⁹

Activation first. For a gym tracker, activation (getting to the gym and logging a workout) is the metric; “if you move activation, retention follows”.¹⁰

Tooling. Claude connected via MCP to PostHog, Superwall and RevenueCat to query data; Wispr Flow for dictation; Notion, Slack.⁸

Most tests do nothing — and that’s useful (Pat Walls): knowing which areas don’t matter sharpens your sense of what to test next.¹¹

When building is cheap, choosing is the bottleneck. Gleam ships a new paywall in ~30 minutes, so most time goes to planning which experiment has the highest leverage.¹²

Disagreements / caveats

• Gleam’s founders argue retention is mostly structural to the category and over-investing in it early was a mistake; Symmetry treats activation/retention as the core lever. Likely both right for their category.¹³
• A 3% retention lift from one screen is a single self-reported test; sample sizes weren’t given.
• Compare Built With Science’s funnel process in Running funnel experiments — problem first, big swings, document everything.

See also

• How often to run paywall tests — and why a 5% ARPU lift can be your whole ad margin
• What to work on at each stage — from launch to paid ads

Footnotes

1. How I Built This $160K/Month Gym App @ 04:32 — Mauro (Symmetry) ↩

2. How I Built This $160K/Month Gym App @ 06:04 — Mauro (Symmetry) ↩

3. How I Built This $160K/Month Gym App @ 07:17 — Mauro (Symmetry) ↩

4. How I Built This $160K/Month Gym App @ 08:15 — Mauro (Symmetry) ↩

5. How I Built This $160K/Month Gym App @ 09:48 — Mauro (Symmetry) ↩

6. How I Built This $160K/Month Gym App @ 09:48 — Mauro (Symmetry) ↩ ↩² ↩³

7. How I Built This $160K/Month Gym App @ 11:22 — Mauro (Symmetry) ↩ ↩²

8. How I Built This $160K/Month Gym App @ 13:04 — Mauro (Symmetry) ↩ ↩²

9. How I Built This $160K/Month Gym App @ 12:10 — Mauro (Symmetry) ↩

10. How I Built This $160K/Month Gym App @ 08:15 — Mauro (Symmetry) ↩

11. How I Built This $160K/Month Gym App @ 16:37 — Pat Walls ↩

12. How two ex-engineers built a $140K/month social skills app | Gleam AI @ 44:29 — Kelvin Zhang ↩

13. How two ex-engineers built a $140K/month social skills app | Gleam AI @ 31:04 — Kelvin Zhang ↩