Updated June 10, 2020
Theme Scientist is an A/B testing app for Shopify.
We launched this app as a tool for for testing 2 theme versions. Then our community requested a product price test app for Shopify, so we built it. ✊
Product A/B Testing
Question: Would your product sell more or less if you increased the price by 20%?
Answer: Run an A/B test and let actual data show you the truth.
A couple years ago I ran an A/B test on a product selling at $23.
Most competitors priced in the range from about $12 to $30.
I said, “the brand is strong and the product is great, let’s try raising the price to 28”.
After 1 month the data showed that sales of the higher priced version were about 20% greater. The new price became permanent.
That 1 experiment increased profit of that product by about $1,200 per month. Nice.
That experiment made me start taking A/B testing seriously.
Every ecommerce store should always be running at least 1 A/B. If not, you’re leaving money on the table.
💡 Every A/B experiment is a win because you learn.
Even if your split test variant performs weaker than your control (current version), now you know that you had the right version already. Next test.
Theme Scientist Version 2.0
Here are some examples of product A/B tests you can try:
Try a standard product image vs a lifestyle image of someone using your product.
Try pricing your product higher and see if it has a significant effect.
Test 2 different names for your product, maybe 1 will resonate stronger with your audience.
Maybe making your product “on sale” will help sales. Maybe not. Who knows until you try.
How It Works
With Theme Scientist you just set up the experiment in the app, and then every day at 00:01 the app swaps the version so each version gets a full day of exposure.
This gives you a good split test because the same user might visit your site from different devices throughout the day. Keeping 1 version for a full day helps keep a consistent testing environment.
The app tracks the stats like:
- page views
- adds to cart
- conversion rate
You can run the test as long as you’d like. I recommend 2 weeks as a general rule, since this will have each day of the week getting a full view.
My preferred test length is 1 month, unless it’s a high traffic site or I can see a clear winner earlier then I might do a shorter test.
How to get started
Theme Scientist has a 14 day free trial.
I use this app and add new ideas + we take feature requests, so don’t be shy.
Our goal is to help you increase your conversion rate so you can get more value from your hard earned traffic.
Get the app here 👉 apps.shopify.com/theme-scientist.
If you want some other ideas for A/B testing check out our Shopify AB Test Guide.
Feel free to send us a message if you want some personalized recommendations or need help setting up tests.