Split Testing – Statistical Significance
What is Statistical Significance?
In short, it means this: Based on the data collected during the test, we can confidently state that “A” or “B” is the best choice.
Here are some more details about Statistical Significance from around the web:
Investopedia ( link ) :
What is ‘Statistically Significant’
Statistically significant is the likelihood that a relationship between two or more variables is caused by something other than chance. Statistical hypothesis testing is used to determine whether the result of a data set is statistically significant. This test provides a p-value, representing the probability that random chance could explain the result; in general, a p-value of 5% or lower is considered to be statistically significant.