Why does Derek Halpern hate split testing?

Derek Halpern posted a video this week. Here it is:

What a headline! He certainly got me watching. 

I was shaking my virtual fists at him: "what are you TALKING about?!"



Well, well. It's part sensationalist bait and switch; part great advice.

First he reiterates how much he hates split testing. Then he admits to doing lots of split testing. He split tested when he started his business and then again as his business matured. He's also included the results of split tests as evidence in countless videos.

But he's actually saying something sensible. 

He's actually saying, "I hate bad split testing." 

Like any tool, split testing isn't necessarily the right thing to use right now. And like any tool, split testing can be used well or it can be used badly. 

Bad split testing is like throwing spaghetti at a wall to see what sticks. But you only throw one strand a week and give up after three or four throws.

On the other hand...

Good split testing is one of the most essential tools for any modern business. And it's not that hard to do it right.

Now there are people who really do hate split testing. There are two reasons: 

  1. Pain. They've tried it, done it wrong and got no conclusive results. When that happens, it SUCKS. The cycle of excitement and hope crashing into disappointment and confusion is like having your heart broken by "the one" over and over again. 

    Remedy: Do it better. Maybe get someone who knows what they're doing to help.
  2. Fear. They're subconsciously scared that split testing might prove that the service they offer doesn't actually add anything to anyone's bottom line. If you're making a killing peddling new hot trends or so-called "best practices" then it can be uncomfortable to find out that they don't work.

    Remedy: Time. Either they'll come round or the truth will out in the end anyway.

Split testing reveals the real effects of ideas, designs and best practices. Time does the same. But split testing beats time because it gives you early warning.

If you're pushing forward and treading new ground then you're going to take a few wrong steps. That's just part of life. What you need is a way to keep pushing forward whilst protecting yourself from getting burnt by the wrong steps. That's split testing.

Split testing is the safety net that lets wise businesses try bold ideas – and get all the upside without the risk.

So how do you do split testing right?

You're going to need a few things in place:

  1. A control. This could be a website, a landing page, an email, or an advert... pick the earliest critical step in your sales and marketing funnel.
  2. A metric. What are you going to measure that will tell you what's working? Often you'll measure some sort of conversion rate or revenue.
  3. Enough traffic. This can get a little technical, but there are simple calculators available that help you work out what "enough" means for you. (More on this another time.)
  4. Questions about your audience that you need to find answers to. These are what drive your create your hypotheses.

If you'd like more help with these steps, I might have the answer:

Request the worksheet: How to Split Test For The Win

I'm creating a worksheet to help you do split testing better. You'll find the best part of the funnel to test, develop a good hypothesis and get yourself the best chance of getting value from your test.

If you're interested, sign up to my UX insights mailing list on the right now and I'll let you peek at the beta release as soon as it's ready.