---
title: Deciding what to build — OKRs, funnel analytics and A/B-testing every product change
slug: product-experiment-process-okrs-and-activation
summary: Symmetry's (~$160K/month) product process — quarterly OKRs, event tracking on every onboarding screen, user interviews for the "why", problem-first hypotheses with guardrail metrics, ≥2-week A/B tests in PostHog — where a one-screen commitment step lifted retention 3% and a redesigned home screen moved nothing.
domain: mobile-apps
tags: [product-strategy, experimentation, ab-testing, okrs, analytics, activation, retention]
related: [paywall-test-velocity-and-arpu-margin, removing-product-friction-pivot, funnel-experiment-process-problem-first, app-growth-stages-organic-to-paid, revenuecat-charts-and-experiments]
sources: [yt:OV09yPLf8BI, yt:s8Rped5LuWw]
updated: 2026-10-24
---

# Deciding what to build — OKRs, analytics, A/B tests

## TL;DR
- Symmetry went through three phases: build what *we* want (terrible) → build what users *request* (mediocre) → **methodical experiments** (works).[^1]
- **Prioritise by reach × impact**: a dark/light theme touches ~2% of DAU; the paywall touches everyone. Find the few high-leverage screens (for Symmetry: exercise search drives activation and retention; the workouts home screen doesn't).[^2]
- **Quarterly OKRs, ambitious when young** — +10% conversion in 90 days is easy a week after launch, hard for Netflix. Review metrics weekly or daily.[^3]
- Track **downloads, activation, D7/D30 retention, ARPU** (per download, varies by country — it caps what you can pay creators).[^4]
- **Track every onboarding screen**; analytics tells you *where* users drop, interviews/Discord/Reddit tell you *why*.[^5]
- **Problem → hypothesis → experiment**, with expected effect size and guardrail metrics; A/B test everything for **≥2 weeks**.[^6]

## Details
**The process ([[people/mauro-symmetry]]).**
1. Start from a problem, not a solution: what problem, how do we know it's real?[^6]
2. Write several hypotheses/solutions; estimate how much each moves the target metric; list guardrails.[^6]
3. Run control vs variant(s) at least two weeks, then analyse.[^7]
4. Log everything in a database ("Evelyn" — Experiment Velocity Engine) fed by user interviews; experiments run in PostHog, paywall tests in Superwall.[^7][^8]

**Results he showed.** Two very different redesigns of the post-onboarding workouts screen: zero movement — "let's not focus here again". One added screen at the end of onboarding where users **commit/sign** to working out: **+3% retention**.[^9]

**Activation first.** For a gym tracker, activation (getting to the gym and logging a workout) is the metric; "if you move activation, retention follows".[^10]

**Tooling.** Claude connected via MCP to PostHog, Superwall and RevenueCat to query data; Wispr Flow for dictation; Notion, Slack.[^8]

**Most tests do nothing — and that's useful** ([[people/pat-walls]]): knowing which areas don't matter sharpens your sense of what to test next.[^11]

**When building is cheap, choosing is the bottleneck.** Gleam ships a new paywall in ~30 minutes, so most time goes to planning which experiment has the highest leverage.[^12]

## Disagreements / caveats
- Gleam's founders argue retention is mostly structural to the category and over-investing in it early was a mistake; Symmetry treats activation/retention as the core lever. Likely both right for their category.[^13]
- A 3% retention lift from one screen is a single self-reported test; sample sizes weren't given.
- Compare Built With Science's funnel process in [[funnel-experiment-process-problem-first]].

## See also
- [[paywall-test-velocity-and-arpu-margin]]
- [[app-growth-stages-organic-to-paid]]

[^1]: yt:OV09yPLf8BI @ 04:32 — Mauro (Symmetry)
[^2]: yt:OV09yPLf8BI @ 06:04 — Mauro (Symmetry)
[^3]: yt:OV09yPLf8BI @ 07:17 — Mauro (Symmetry)
[^4]: yt:OV09yPLf8BI @ 08:15 — Mauro (Symmetry)
[^5]: yt:OV09yPLf8BI @ 09:48 — Mauro (Symmetry)
[^6]: yt:OV09yPLf8BI @ 09:48 — Mauro (Symmetry)
[^7]: yt:OV09yPLf8BI @ 11:22 — Mauro (Symmetry)
[^8]: yt:OV09yPLf8BI @ 13:04 — Mauro (Symmetry)
[^9]: yt:OV09yPLf8BI @ 12:10 — Mauro (Symmetry)
[^10]: yt:OV09yPLf8BI @ 08:15 — Mauro (Symmetry)
[^11]: yt:OV09yPLf8BI @ 16:37 — Pat Walls
[^12]: yt:s8Rped5LuWw @ 44:29 — Kelvin Zhang
[^13]: yt:s8Rped5LuWw @ 31:04 — Kelvin Zhang