Hi,

Here's a question for you statisticians / taguchi experts out there...

I'm currently using Multitrack Generator to test multiple variables on my sales page. I'm wondering if there is an optimal number of variables that one should be testing concurrently.

That is to say, does the math behind taguchi start breaking down at some point -- say if I were testing 15 or 20 different variables at once?

What, in your experiences, is the optimal number of variables that can be tested at once in order to still get meaningful data from a taguchi test?

Thanks,

Michael

12

And is that the optimal number, the number right before which the math breaks down, or both?

Should I make an effort to find 12 things to test concurrently, if possible?

Thanks,

Mike

Up to is fine. After 12, the margin of error grows.

I recommend you checkout http://www.vertster.com/ ... and their forum. Scott Miller, David Bullock, and yours truly, hang out ... all about Taguchi and multivariant testing.

Here's a freebie course on taguchi testing.

http://www.nathanshaw.name/recommends/STA_Course

I use this software and love it. Jim's support has been the best support I've ever had from anywhere, and when it's taguchi testing we're talking about, that's priceless.

Nathan

Hi Michael and Michael!

Fortin is correct in that the margin for error grows as you use larger and larger test designs. Huge tests can be done successfully. It just gets harder.... You will need massive amounts of traffic and a highly controlled test environment to make it work.

Take a look at http://www.york.ac.uk/depts/maths/tables/orthogonal.htm for a few possible Taguchi designs. You will see the L81 which uses *40* variables. Not really practical for online testing, but it is out there.

Personally, I think even 12 variables is too many for most websites. Particularly since most people don't really know how to design a test effectively to control noise and maximize the strength of their success metrics.

I agree with Scott.

My best tests have been where the variables are wildly different.

Hi Scott,

This is very interesting. Can you point me in the direction of any resources you know about relating to designing test effictively?

Cheers,

Michael

I thought the answer was 42...

no, wait, sorry -- I'm thinking of the Hitchhiker's Guide to the Galaxy.

Ah, yes. The secret to life...