Here is a very interesting presentation on the common pitfalls and suggested best-practices regarding online testing.

It is a recording of a live web-conference given by Dr. Flint McGlaughlin of Marketing Experiments. He has a wonderful speaking manner and goes through great lengths to address some of the more common anxieties novice researchers have about online testing; including framing the right questions, statistical validity, and determining metrics. While Dr. McGlaughlin does not go into great depth on any of the topics that he covers, the surface-level advice that he gives could improve most of the online testing that we see today.

Much of his presentation is centered around his copyrighted “formula” to prove the usefulness of a given research project.

The formula is “usefulness = 2(questions) + treatment + metric + 2(validity) + interpretation.” Dr. McGlaughlin doesn’t explicitly show values being put into this formula but it appears that he considers all the components important, with “asking the right questions” and “making sure the research is valid” to be twice as important as the rest. As all of the variables are subjective, I’m not sure of the formula’s overall value but maybe that’s not the point.

It’s a useful presentation to anyone who has wanted to test the output of a new website project or has struggled with coming up with a clear strategy of how to gather reliable data from an online initiative.

The case study that he uses is from a project with Britannica Online to improve the conversion rate of their free trial offer. This case study is a great example of how to correctly construct a solicitation form.

Link to presentation (flash based)