Apple PPO limitations and how practitioners work around them
The practical constraints of Apple's Product Page Optimization (new releases kill tests, review delays, one test at a time, aggregated international data, icon tests need a binary) and the workarounds Popcore and Klarna used in 2022–2023.
Apple PPO limitations and how practitioners work around them
TL;DR
- Releasing a new app version stops a running PPO test (2022–23). Plan tests around the release roadmap; a win needs ~7 days running + ~2 days for analytics to catch up.12
- Screenshot, video and icon tests don’t need App Review; other treatments do. Klarna, with weekly releases, tests screenshot reorders right after each release to get a full 7 days.34
- Stuck “Waiting for review” >2 days? Cancel and resubmit — usually passes in hours. Avoid submitting on Fridays.56
- One test at a time, and multi-locale tests report aggregated country metrics — Popcore tests en-US first, then 1–2 top locales, then rolls the winner to small locales untested.78
- Icon tests need every icon in the binary (Xcode 13+ asset catalog), and applying a winner needs another binary release.910
Details
Sources date from PPO’s first two years: Marina Roglic (Popcore, ASO Conference 2022), Mateusz Wrzeszcz (Klarna, 2023), Jevgen Tarasenko (Tilting Point, Feb 2022), Simon Thillay (AppTweak, Feb 2022). PPO launched 7 Dec 2021.11
Release conflicts. Marina’s workaround is organisational: get on the product team’s release channel and roadmap, sell the business case (native, free, winner is one click away), and share wins publicly so production becomes willing to wait.212 Mateusz could only run real iOS tests during a three-week release freeze.13 Storemaven’s Jonathan Fishman (2022) saw no company delaying a release for a screenshot test, which is why PPO lagged Google Play experiments in adoption.1415
Review and upload quirks (Marina, 2022).
- Drafting a test, then starting it after the version ships fails — the tested version shows as expired.3
- Assets stuck “processing”: delete and re-upload.6
- In PPO the video poster frame doesn’t keep the timestamp you set; she places the intended poster frame at second 29 of the video, which works 9 times out of 10.7
Localisation strategy. Because results can’t be split per locale and analytics show countries, not localisations, Popcore borrows a head start from Android learnings, then sequences: default language → top “whale” locales → apply elsewhere and monitor KPIs.78
Icon tests. Treatments (up to three plus default) must ship in the binary and pass review; the test itself then needs no review. Applying the winner can’t be done remotely — stop, ship a new binary with it as default, wait for review again. Popcore lacked dev capacity and used an external tool instead.910
CPPs aren’t a substitute. Some developers reported success replacing PPO with custom product pages; Popcore did not.16
Disagreements / caveats
- These are 2022–2023 observations. The 2026 notes in this base (Producing screenshots and running Product Page Optimization tests, Steve P. Young) still describe one draft at a time and submit-for-review, but don’t confirm whether a new release still stops a test — check current App Store Connect behaviour.
- Jevgen (2022) called native testing “revolutionary” despite these issues; Simon said PPO had so many implementation issues that it lacked even the appeal of Google Play experiments.1714
See also
- Store A/B tests — sample size, MDE and why bold variants win
- Reading store A/B test results — when to stop, applying ties, validating winners — re-analysing PPO data with a Bayesian calculator
Footnotes
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 04:03 — Marina Roglic ↩
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 04:59 — Marina Roglic ↩ ↩2
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 08:09 — Marina Roglic ↩ ↩2
-
Mastering A/B Testing for ASO | Mateusz Wrzeszcz | ASO Conference 2023 @ 38:06 — Mateusz Wrzeszcz ↩
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 08:58 — Marina Roglic ↩
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 09:48 — Marina Roglic ↩ ↩2
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 10:36 — Marina Roglic ↩ ↩2 ↩3
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 11:24 — Marina Roglic ↩ ↩2
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 14:33 — Marina Roglic ↩ ↩2
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 15:29 — Marina Roglic ↩ ↩2
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 01:34 — Marina Roglic ↩
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 05:45 — Marina Roglic ↩
-
Mastering A/B Testing for ASO | Mateusz Wrzeszcz | ASO Conference 2023 @ 36:17 — Mateusz Wrzeszcz ↩
-
Mastering ASO in 2022 with Simon Thillay of AppTweak @ 29:21 — Simon Thillay ↩ ↩2
-
Mastering ASO in 2022 with Simon Thillay of AppTweak @ 33:53 — Jonathan Fishman ↩
-
Making the most out of PPOs | Marina Roglic | ASO Conference 2022 @ 16:22 — Marina Roglic ↩
-
How Do In-App Events Influence ASO and What is the Correlation Between Organic and Other Metrics... @ 13:17 — Jevgen Tarasenko ↩