How to A/B test an App Store product page

Product page optimization is Apple's built-in A/B test for the default App Store product page. A test compares the original page with up to three treatments, and each treatment can use an alternate app icon, alternate screenshots, alternate app previews or a mix; whatever a treatment does not replace stays as it is on the original. The app has to be live on the App Store in the Ready for Distribution state, and the test is set up by someone with the Account Holder, Admin, App Manager or Marketing role. You choose a traffic proportion, the share of visitors picked at random to see a treatment instead of the original: with three treatments and 30%, each treatment reaches 10% of visitors. A visitor keeps seeing the same treatment for the whole test, and only people on iOS 15 or iPadOS 15 and later are included. By default a test covers every localization the app supports; a visitor shown a localization the test excludes is left out of it. App Store Connect can estimate how long a test will take from the app's own daily impressions and downloads, and Apple suggests fewer treatments or more traffic when that estimate runs past 90 days. Any new metadata in a treatment has to be approved by App Review before the test can start. Alternate icons are a special case: every icon variant has to be included in the binary of the published app, while a test without alternate icons can be submitted for review independently of a new app version. Once started, a test can't be changed, and a stopped test can't be restarted; it runs for up to 90 days or until you stop it, and an app runs one test at a time. Results appear in App Analytics once five first-time downloads are attributed to the test, showing impressions, conversion rate, improvement over the baseline and a confidence level for each treatment. A treatment that reaches 90% confidence is labeled Performing Better or Performing Worse; until then it is Collecting Data, and one unlikely to get there within 90 days is marked Likely to be Inconclusive. You can apply one treatment per test to the original product page and to versions in Ready for Distribution or Prepare for Submission, the action can't be undone, and applying a treatment while the test runs stops the test. Only the treatment's screenshots and app previews are applied; a winning icon has to be set as the default icon in the next app version. Product page optimization tests the default page only: it is not available for custom product pages, which are separate pages reached through their own links and keywords. Google Play has its own version in Play Console, store listing experiments, which test graphics such as the icon, feature graphic and screenshots.