How To Test Websites, Part 2 - MultiVariate Testing

In part 1, we looked at Split testing. We showed that it is easy to implement but, since you can only test one variable at a time, it takes time before your web page becomes optimized. This results in missed sales.

In this part, we will look at multivariate testing.

In multivariate testing, different combinations of multiple variables are shown randomly to website visitors, and the multivariate process uses statistics to determine which combination is the most effective.

The advantage is that optimization happens much quicker, because you are not constrained to only test one part of your web page at a time.

One disadvantage of standard multivariate analysis is that it would take a lot of data before you could be statistically confident about the results - which will cause delays if you do not have a high traffic website. For example, if you tried to test 3 variables at the same time, and each variable could have four possible values, you are looking at 3^4 = 81 possible combinations.

However, there are 2 modifying algorithms that can counter this disadvantage. They speed up the analysis process through mathematical shortcuts. They are: Taguchi methods and the MuVar algorithm (proprietary to the MuVar software package).

Taguchi tries to approximate standard multivariate analysis by eliminating some of the possible combinations. MuVar, on the other hand, is more subtle. It tries to slightly bias the weight of combinations that prove successful early on, as well as cut down the testing of combinations as soon as they prove to be statistically significant losers. Testing has shown MuVar to be the faster and more accurate algorithm.

The biggest disadvantage to multivariate analysis is that it requires mathematics and statistics beyond the level of the average internet marketer.

However, the MuVar software package eliminates this disadvantage. It actually provides the best of both worlds - the simplicity of split testing with the quick optimization power of multivariate analysis.

In part 3, we will look more closely at MuVar.