[Printable] Split Testing and Statistical Significance 1-Pager

ntestIf you’ve had the chance to read The The 4-Hour Chef by Tim Ferriss then you know that he’s kind of hooked on the idea of one-pagers.

In essence, a one-pager is a very condensed resource about a given topic. It’s meant to list only the essential, and only the bits that will give you the most results while at the same time requiring the least of your input/effort.

I’ve liked the idea of one-pagers right away the minute I saw them. That’s probably because I like structured information. I like structure in general. Okay, I’m a Structure Nazi (like a Grammar Nazi only less mainstream).

Seeing that one-pagers are a great tool to convey somewhat complex ideas in a relatively easy to grasp manner, I’ve decided to use them on this blog.

My first target, as you can see in the headline – split testing and statistical significance.

Split testing and statistical significance 1-pager


Check statistical significance

Checking statistical significance will let you know if the results you’re getting from your split test are by any chance accidental.

For that, you can use my statistical significance calculator >>.

If your results are significant, you can name the winner and the loser of your test.

loser winner

Next step: scrap the loser, create another version of your test subject and run it against your winner in a new test. In other words, start over.

Don’t over-interpret


The main thing to keep in mind when split testing is not to over-interpret your results. Just because there is a winner and there is a loser, doesn’t meant that you will be able to tell exactly why the test has turned out the way it did. The reasons can be many.

It’s a lot safer to just take the results as they are, and not build theories trying to explain them. Most of the time they turn out to be false anyway. Your time is much better spent creating a new version and running the test again.

Your downloadable copy.

What’s your relationship with split testing? Do you even lift split test?

[Printable] Split Testing and Statistical Significance 1-Pager | newInternetOrder.com

My Statistical Significance Calculator Relaunches

launchA while ago, actually something like a year and a half ago, I launched a little tool called the statistical significance calculator. The thing was working just fine until I changed my design a couple of months ago, and then it stopped.

Since I believe it still is a great tool for assessing your split testing campaign’s results, I’ve decided to relaunch it. Or more accurately, make it work again.

If you know what this is about and don’t want to read any further then just click this link and test it for yourself: Statistical Significance Calculator.

However, if the thing that’s on your mind right now is more like “what the hell is statistical significance?!” then here’s your answer:

A/B testing

Let’s start with the basics. A/B testing or split testing is simply a method of running two versions of something (usually a marketing message or copy) alongside each other, taking note of the results, and then pointing out the better performing version.

Now, this “pointing out the better version” part isn’t always that easy. Sometimes the raw numbers simply don’t convey any clear message, and that’s where statistical significance comes into play.

The deal with statistical significance

In a word, statistical significance is a math-thing.

Its purpose is to tell us if we’re getting significant split testing results or not. In plain English, it gives us an insight if using our split testing results for making any kind of decision about the test subject is a good idea or not. In other words, statistical significance tells us what’s the chance of our test results being purely accidental.

Let me give you an example. Let’s say you have a split test around a given sales message. Version (a) managed to generate 12 conversions out of 100 views, while version (b) got 15 conversions out of 102 views. The question: Which is the better version?

Our simple human mind tells us that it’s version (b) – it has a 15% conversion rate while version (a) has only 12%. However, math tells us that the results are not significant (try them out in my calculator), and therefore, can’t be taken seriously.

In the example above, the test sample is simply too small to be able to determine anything accurately (100 views and 102 views) in relation to the number of actions/conversions (12 and 15).

My statistical significance calculator has been designed to point out every such situation, so you can make better decisions based on your split testing results.

Well, that’s it for the theory part. Here’s the link again: http://newinternetorder.com/statistical-significance-calculator/. The exact how-to guide is also there.

By the way, do you split test various elements of your site a lot?

My Statistical Significance Calculator Relaunches | newInternetOrder.com

Funny Marketing Trick (the “Douchebag Ninja”)

If you’ve been interested in making money online for any amount of time then you’ve probably seen one of those sales videos where the narrating voice tells you to shut down all distractions so you can “receive the most value” when watching the video.

The instruction usually tells you to shut down Facebook, Skype, Twitter, close your other browser tabs, and even (believe it or not) turn off your phone and lock the door of the room you’re in right now.

… And it’s all so you can “receive more value” … yeah, right.


It’s actually a marketing technique I like to call the Douchebag Ninja.

By the way, have you noticed that I’m using the word douchebag an awful lot lately? Anyways.

The rules of the Douchebag Ninja

First of all, why the name… It’s all because the technique is kind of stealth in nature, and because it’s about deceiving the visitor that there’s something special waiting for them, only to make them more receptive to the marketing message.

The main goal of the Douchebag Ninja is simply to maximize the conversions that the offer is getting. And one of the things that work quite well in doing so is convincing the viewer to shut down all possible distractions around them.

It’s actually very easy to lose a sale when you’re an online marketer. All it takes is one single distraction and your prospective customer is no longer paying attention to anything you’re saying.

Therefore, if a marketer manages to convince the viewer to lock themselves in a room and throw away the key, they have much better chance to close the sale.

The problem with the Douchebag Ninja

The problem isn’t the technique itself but the deceptive instructions used by people trying to apply it.

The thing is that you don’t need to shut down distractions to “receive value” from anything, and if someone tells your otherwise, they just want to sell you something.

Picture this, can you imagine yourself in a crowded bus, standing, yet holding a great book and reading it with complete dedication despite the overall noisy environment? It’s probably no problem for you… The fact is that we can always tune ourselves for receiving value, no matter the scenery we’re in. And we certainly don’t need anyone to tell us what to do with our phone, Facebook, or other “distractions.”

The takeaway

Basically, there are two lessons to take away from this post.

  1. Don’t believe anyone who tells you that you “need to shut down all distractions because what you’re about to see will change your life forever” – it’s just the Douchebag Ninja technique in practice.
  2. If you’re trying to sell something yourself, don’t try to trick people into paying attention to you. Instead, make your message interesting enough so people will shut down distractions on their own.

I guess that’s it about the technique itself. Have you come across anything like this in the online marketing space recently?

Funny Marketing Trick (the “Douchebag Ninja”) | newInternetOrder.com