How to Use Shopify Rollouts for Theme Changes and A/B Testing
How to Use Shopify Rollouts for Theme Changes and A/B Testing
Making changes to your Shopify store can improve the customer experience, increase engagement and support more sales. However, publishing a major design change to every visitor at once can create unnecessary risk.
Shopify Rollouts gives you a more controlled way to introduce and test storefront changes. You can use Rollouts to schedule theme updates, divide traffic between different theme experiences and automatically return visitors to your original theme when a test ends.
Instead of publishing an untested theme update to your entire audience, you can use Rollouts to control when the change begins, how much traffic it receives and what happens when the rollout ends.
What Is Shopify Rollouts?
Shopify Rollouts is a new admin feature that allows you to schedule storefront changes and divide traffic between different experiences (known as A/B testing.) Each experience is a different version of the theme or site.
One version acts as the control. This is normally the live theme that customers already see. The other version acts as the treatment. This can be an edited copy of your current theme or an entirely different theme available in your Shopify store.
For example, let’s say you want to test a new homepage design. You could create a rollout that sends:
- 50% of your traffic to the original homepage
- 50% of your traffic to the edited homepage
Shopify manages the traffic split during the scheduled testing period. At the end of the rollout, you can return all traffic to the original theme or automatically apply the new version.
Shopify Rollouts can also help you schedule changes around a promotion. You could prepare a sale-focused version of your theme, launch it on a specific date and automatically remove it when the promotion ends.
Compare different storefront experiences
A new theme or design may look great during internal reviews, but actual customers can interact with it differently than expected. Shopify Rollouts lets you introduce the changes to a smaller share of visitors before exposing it to your full audience. You can use this as a “test run” of the new design to make sure everything goes according to plan. You could send only 10% of eligible traffic to the new version while the remaining 90% continues to see the original theme. Based upon the results, you can edit the new design and make sure you are optimizing the new theme design for the best conversion rate.
This basic A/B testing lets you compare experiences without immediately replacing the live theme. You could test changes like:
- A different homepage hero
- A revised product promotion
- A new featured collection
- A reorganized homepage layout
- A seasonal storefront design
- A different navigation approach
- A redesigned landing page experience
Schedule changes in advance
Rollouts includes scheduling controls for the start and end of a storefront change. This helps your team prepare campaigns before they need to go live. For example, you could schedule a promotional theme to launch at the beginning of a sale and end when the promotion expires. This is the perfect tool for the holiday season when you’re pushing new messaging frequently to align with various sales.
Control the size of the test
Shopify Rollouts gives you control over both the split between the theme versions and the percentage of total site traffic included in the rollout.
This means you can decide:
- How much total traffic participates in the rollout
- How the participating traffic is divided between the control and treatment
That flexibility can be useful when a change is significant or when you want to begin with a limited audience.
How to Create a Shopify Rollout
The following walkthrough explains how to create a basic theme rollout using your current Shopify theme as the control and an edited version as the treatment.
Step 1: Open Rollouts From Shopify Markets
From your Shopify admin, go to the Markets section.
Select Rollouts to open the rollout management screen. This area shows your existing rollouts and provides the option to create a new one.
Click Create to begin.
Step 2: Name Your Rollout
Give the rollout a clear name that explains what you are testing or scheduling.
For example:
Homepage A/B Test
A descriptive name will make the rollout easier to recognize later, especially if your store runs several tests or scheduled changes.
Other possible names include:
- Holiday Homepage Rollout
- New Product Page Theme Test
- Summer Sale Theme
- Homepage Hero Test
- Updated Navigation Test
Step 3: Add a Change
After naming the rollout, select the option to add a change.
The walkthrough presents options for changing the online store theme or making a change associated with the checkout or customer account experience. The video focuses on creating an online store theme rollout.
Select the option to change the online store theme.
Step 4: Choose Your Control Theme
Shopify identifies your currently published theme. This live theme becomes the starting point for the rollout.
In an A/B test, this version acts as the control. It represents the existing storefront experience against which you will compare the edited version.
Shopify then asks what edits you want to make to the theme.
Step 5: Edit the Treatment Theme
Click Edit to open the Shopify theme editor.
The editor should look familiar if you have previously customized a Shopify theme. The important difference is that the changes you make here belong to the rollout draft. They do not immediately replace your currently published theme.
Make the changes you want to test.
For example, you might:
- Change the homepage heading
- Add a featured product section
- Replace the homepage banner
- Move a collection higher on the page
- Adjust the order of homepage sections
- Create a different promotional experience
The video demonstration makes an obvious text change so the edited version is easy to distinguish from the live theme.
Once you have completed the changes, click Save. Exit the theme editor to return to the rollout configuration.
Step 6: Review the Control and Treatment
The rollout should now display two versions:
- Control: Your original published theme
- Treatment: The edited theme created for the rollout
Review the changes before continuing. You can reopen the treatment in the theme editor if you need to make additional adjustments.
You can also replace the edited version with an entirely different theme. This is helpful when you want to compare your current theme against a larger redesign rather than a small set of edits.
Step 7: Choose the Traffic Split
Next, decide how traffic should be divided between the control and treatment.
For a standard A/B test, you might use an even split:
- 50% to the control
- 50% to the treatment
Shopify then distributes traffic between the two versions during the rollout.
A 50/50 split can be appropriate when the change is not especially sensitive and you want both versions to receive a similar amount of exposure.
For a more cautious rollout, you could use:
- 90% to the control
- 10% to the treatment
This allows you to test the new experience with a smaller share of visitors while most customers continue to see the existing storefront.
The right split depends on the size of the change and your comfort level. A minor homepage adjustment may support a 50/50 test, while a major redesign may be better suited to a more limited initial rollout.
Step 8: Set the Start Date
Choose the date when the rollout should begin.
Scheduling allows you to prepare the theme changes in advance rather than manually publishing them at the start of a campaign.
For example, you could schedule a test to begin on Monday so it runs for a complete week.
Before selecting the date, confirm that:
- The treatment theme is complete
- The design has been reviewed
- Links and buttons work correctly
- Mobile layouts have been checked
- Promotional content matches the campaign schedule
- Your team knows when the rollout will begin
Step 9: Set the End Date
Choose when the rollout should end.
The demonstration schedules the rollout for one week. During that period, traffic is divided between the original theme and the edited theme.
The appropriate test length will depend on your store traffic and the purpose of the rollout. The Rollouts interface can provide an estimate of how many visitors may be affected based on the percentage of traffic included.
Avoid making decisions based only on a very small number of visitors. Your test needs enough traffic to provide useful information.
Step 10: Decide What Happens at the End
Shopify gives you options for what happens after the rollout reaches its end date.
You can return the treatment to 0%, which sends traffic back to the original control theme. You may also have the option to apply the treatment after the rollout ends.
For most A/B tests, returning to the control is the safer approach. This gives your team time to examine the results and decide whether the treatment should become permanent.
Automatically applying the treatment can make sense when you already know the new version should replace the control. However, you should use that option carefully. An automatic change may become permanent before your team has fully reviewed the storefront experience.
Step 11: Select the Total Traffic Allocation
Rollouts also allows you to decide how much of your store’s total traffic will participate.
For example, you could include 100% of your traffic in the rollout and divide those visitors equally between the control and treatment.
You could also include only 50% of your total traffic. Shopify would then apply your selected control-treatment split within that participating group.
Using all available traffic may help you collect information faster. Limiting the rollout may be appropriate when you are testing a major change or want to reduce exposure during the first phase.
Review the estimated number of affected visitors before saving the rollout.
Step 12: Save and Review the Scheduled Rollout
After configuring the themes, traffic allocation, dates and end behavior, save the rollout.
Return to the Rollouts screen to confirm its status.
A scheduled rollout should show important information such as:
- The rollout name
- Its scheduled status
- The start date
- The end date
- The markets included
- The online store customization
- The percentage of traffic allocated
Review these details carefully. A scheduling mistake or incorrect traffic allocation could expose the wrong experience to customers.
You can reopen the rollout if you need to edit the treatment, replace the theme or adjust the configuration before launch.
Shopify Rollouts Best Practices
Rollouts makes testing and scheduling easier, but the quality of the result still depends on how you plan the change.
Test one clear idea
When possible, make the treatment focus on one primary hypothesis.
For example:
Hypothesis: Featuring the best-selling collection closer to the top of the homepage will encourage more visitors to view products.
If you change the heading, navigation, promotional offer, product order and homepage layout at the same time, it may become difficult to understand which change influenced customer behavior.
Review the treatment on mobile
Theme changes can behave differently across desktop and mobile devices. Review the treatment on common screen sizes before scheduling it.
Check navigation menus, banners, text wrapping, product cards and calls to action.
Start cautiously with major changes
A 50/50 split may work well for a simple change. A major theme redesign may warrant a smaller treatment audience at first.
Starting with 10% of traffic can help you identify obvious problems before expanding the rollout.
Use rollback as the default for experiments
Returning traffic to the original theme at the end of a test gives your team a chance to review what happened before adopting the change permanently.
This is especially important when the rollout includes a new design, unfamiliar functionality or a large number of theme edits.
Put Shopify Rollouts to Work for Your Store
Shopify Rollouts gives merchants a more controlled way to test storefront ideas, schedule design changes and introduce new theme experiences.
The feature can help you avoid the all-or-nothing approach of publishing a new theme directly to every visitor. You can create a treatment, determine how much traffic sees it, schedule when it runs and choose whether the store returns to the original theme afterward.
However, a successful rollout requires more than moving a traffic slider. You need a clear testing strategy, a properly configured treatment, thorough quality assurance and a plan for evaluating the outcome.
Sunrise Integration can help you plan, build and manage Shopify Rollouts for your store. Our Shopify experts can assist with theme customization, rollout strategy, traffic allocation, testing and post-rollout implementation.
Contact Sunrise Integration to discuss how Shopify Rollouts can help you test changes safely and create a better shopping experience for your customers.
Frequently Asked Questions About Shopify Rollouts
What is Shopify Rollouts?
Shopify Rollouts is a Shopify feature that lets merchants schedule storefront changes, test different theme experiences and control how traffic is divided between them. You can use your current published theme as the control and compare it against an edited or replacement theme.
Where can I find Shopify Rollouts?
You can find Shopify Rollouts in the Markets section of your Shopify admin. Open Markets, select Rollouts and click Create to configure a new rollout.
Can Shopify Rollouts be used for A/B testing?
Yes. Shopify Rollouts can be used to divide traffic between your current theme and an edited theme. For example, you can send 50% of participating visitors to your original homepage and 50% to a redesigned homepage.
Can I schedule a Shopify theme change in advance?
Yes. Shopify Rollouts lets you select a start date and an end date. This is useful for scheduled sales, seasonal campaigns, limited-time promotions and planned storefront tests.
What happens when a Shopify rollout ends?
You can configure what happens at the scheduled end of the rollout. For an A/B test, you can return the treatment to 0% so all traffic goes back to the original theme. You may also choose to apply the new version after the rollout ends.
Can I replace the treatment with a completely different Shopify theme?
Yes. Shopify Rollouts can compare your current published theme with a different theme available in your store. This gives you the flexibility to test a larger redesign rather than making only small edits to the existing theme.
How long should a Shopify rollout run?
The rollout should run long enough to expose both versions to a useful number of visitors. A one-week test may be suitable for a store with consistent traffic, but lower-traffic stores may need more time. Review the estimated number of affected visitors before choosing the start and end dates.
Does Shopify Rollouts automatically measure which theme performs better?
The transcript demonstrates how to create the rollout, divide traffic and schedule the test, but it does not explain a built-in performance reporting process. Before launching, decide which business metrics you will use to compare the control and treatment, such as conversion rate, product views or customer engagement.
