Vladimir's answer

[nusaiba127]

4. Statistically analyze the data array with test results (Excel is absolutely fine, you can also use SPSS). Are the results within the margin of error, how much do they deviate, and how do they depend on time. If, for example, in the first point of the A/A test you get strong deviations from one option to another, this is a failure, and you cannot test further.

5. Don't test everything. This is not entertainment (unless you really have nothing else to do). It makes sense to test only what can lead to noticeable results from the point of view of marketing and business analysis. And also what the result of which can be really measured. For example, you decided to increase the font size on the site, tested a page with a large font for a couple of weeks - sales increased. What does this mean? It means nothing to me either (see the previous points).

6. You need to test entire paths. That is, it is not south africa consumer email list enough to just test the purchase page (or some action on the site) — you need to test those pages and steps that lead to this final conversion page.

7. You also need to test advertising channels (and not just the page on the website).

A question was asked in the comments:

"How do you establish a winner? We tested a headline on a page that sells "head-on". What difference in conversion should there be between A and B to recognize a winner?"



First, you need to conduct long-term isolated experiments (the basic rule of any statistical assessment). Second, everything inevitably comes down to statistics and mathematics (that's why I recommend Excel and SPSS or similar free ones). We need to calculate the confidence probability that the difference in values ​​means something. There is a good article (one of many). They take transactions from GA according to Optimizely tests t/uploads/ga_transactions.png , compare transactions (purchases) with a regular bell distribution and see if the average value falls within the confidence interval of the error