Kinds Of Split Testing Software
I read once about a man who tripled his conversion rate by changing a single letter in his headline. While that kind of improvement is not common (not by a long stretch), you can quite probably double your conversion rate after testing 10 or 20 items on your page. With that said, there are many things to consider when getting started with split testing.
I can’t discuss every feature of every testing tool out there, but I can give you a major overview of the different kinds of split testing software.
One big distinction is between single variable and multiple variable (multivariate) testing.
With single variable testing you just pick one element on your page to test. Take your headline, for example. If your current headline says “Are you having trouble with tooth decay?”, you might want to see how “Do your kids have tooth decay?” works. In a single variable test, you just find a way to rotate each of those headlines on your page, and keep track of how many sales each generates.
Paul Hancox has a simple split testing tool. And Google has one that’s free, though it takes a bit to set up all your accounts to use Google’s tool. And the reporting is not real-time.
Single variable testing can be slow. How would you like to test 10 times as much with the same amount of traffic? Well, you can. You use a method called multi-variable, or multivariate testing. This allows you to test several elemens on your page simultaneously.
So there’s the main distinction between single variable testing and multivariate testing. But once you get into the realm of multivariate testing the options seem to multiply again. One difference among multivariate tools is between tools that are hosted on someone else’s server, and ones that you install on the same server as your webpages. Hosted solutions can be slower, and more expensive, but you don’t have to install any software.
The Split Test Accelerator is a good choice if you want to install the software on your server. Kaizentrack is another optoin. If you want a hosted solution, you can look at Sitespect and Vertster.
Taguchi testing is only one kind of multivariate testing. There are also “full factorial” tools, and random independent rotators. When you run a Taguchi test, you run only a small fraction of the possible combinations of your factor levels. You might, for instance, run only 18 pages to test 4374 combinations. This is made possible by orthongonal arrays. This allows you to find the best combination with out running through all of them. You do have to be careful about your factors interacting negatively with each other, but if you take care as you design your test, you can avoid the bulk of the problem with these. The Split Test Accelerator, Vertster, and Kaizentrack are Taguchi testing tools.
If you want to get around interaction effects you can with a full factorial tool. It will take more traffic to avoid the interactions — a lot more — but with a FFT, like Google’s Website Optimizer, you can just see how every combination plays out, and live with the interactions.
The Random independent rotators are a third option. These treat factors independently. So you can just plug in a new factor with as many options as you want at any time, and ignore the fact that other factors are being tested at the same time. This is the most flexible form of testing, but it is also plagued with the most interaction effects.
There are other distinctions and features worth noting. And each solution has a different mix of pros and cons. But this should get your oriented a bit before you make your decision.