Split Test – Analysing the results
As soon as you’ve created your test you’ll start to see some data appear in the various charts.
You will see historic data (minimum 30 days) and current data. This will allow you to spot any significant changes in trend before you began running the tests. This is very valuable insight in order to understand if a change in performance is really linked to the test or if it’s simply a normal behaviour for your product.
Once the test is finished, the bottom chart will summarise the data seen in the above charts.
It will compare the average of each key metrics (Revenue, Sales, Sessions, Conversion and Conversion Rate) for both tests and will compare that a sample. This sample (also known as “control”) is made up of the 30 days prior to the test running. This allows you to only to compare A vs B but to also see if A has itself experienced other external factors that have influenced its performance.
This is really important as there are so many moving parts on Amazon and you cannot control everything that impacts your product. Therefore having this additional data driven insight will help you make smarter decisions about your A/B tests results.