There are dozens of different options in your digital marketing strategy. How do you know which one is best?
The answer lies in statistics.
Statistical confidence rating is a way of quantitatively measuring the probability that one option is better than another.
For instance, a store owner might have two different advertising posters and might want to know which poster is best at attracting customers.
The store owner puts one poster (poster “A”) in the window for a week, and counts both the number of people who walk by and the number of people who enter the store. The next week, the owner puts the alternative poster (poster “B”) in the window and measures the number of people who walk by and the number of people who enter.
At the end of the two weeks, the store owner calculates that 30% of the people who saw poster “A” went into the store while 80% of the people who saw poster “B” entered. Intuitively, it seems clear that poster “B” is the best.
What if only 3 customers went by on each of the two weeks? Now our confidence in the results is much lower.
Enter statistical confidence rating. Confidence rating allows you to determine a confidence interval associated with a result and the probability that the result reflects reality. Confidence rating comes in two parts: a confidence interval and a confidence level.
A confidence interval is typically expressed in terms like “30% +/- 3%” and describes the range within which we can expect results to fall.
The confidence level represents the chance that a real result falls within that range. For instance, our result of “30 +/- 3” might be calculated at a confidence level of 95%. In such a case, there is a 95% chance that a real result lies between 27 and 33.
Statistical confidence ratings are typically calculated using Fisher’s exact test or Welch’s t-test. These formulas allow our store owner to understand the meaning of his experimental results and to be sure that he puts the best poster in the window of his store.
Feel free to contact Hyphen if you have any feedback or comments.