How to Improve Your Website Conversions with A/B Testing [INTERVIEW]

Successful website optimization is a delicate balance of art and science that can have a drastic impact on your ability to generate leads, convert new customers and increase sales. The following questions were posed to Pete Koomen, an ex-Googler and now Co-founder of Optimizely – a fast-growing startup helping companies increase their conversions with A/B testing .

What is A/B testing?

The idea of A/B testing is very simple: make a change to your website, (or an ad, or a marketing email,) and then measure how this change affects visitor behavior.  A really simple example is A/B testing the effect of a text changes on your landing page call-to-action button: does a red “Buy now” work better than a blue “Learn more”?  How about “Get started”? These are questions that A/B testing can answer quickly.

Optimizely has made it dead simple to run A/B tests on your website, whether you’re an engineer or a marketer.

Why is A/B testing important?

A/B Testing is important to anyone trying to communicate with specific goals in mind: are you trying to sell something with your website?  Trying to drive engagement on your blog? Trying to drive user signups on your landing page?  A/B Testing will help you optimize for these goals using your own visitor data.  For a real-world example illustrating how powerful A/B Testing can be, you can read about how the Obama presidential campaign raised $60MM through rigorous A/B Testing.

What kind of variables should be tested?

Every site is different, but there are two places we always recommend people start when A/B testing:  calls to action and drop-off points in your funnel.  Calls to action are pretty straightforward–if you’ve got a well-defined funnel, A/B testing is a great way to ensure that your calls to action are visible and resonate with visitors.  Drop-off-points are usually much more site-specific, but they generally refer to points along the funnel where visitors tend to drop off.  Take a look at your site analytics and try and identify the dropoff points; these areas usually represent the lowest-hanging fruit when it comes to improving a site’s performance.

On what type of page are A/B tests best applied?

Any page on which you’re hoping your visitors will take some action is a great candidate for A/B testing.  The best candidates are pages with a very clear goal, like a landing page, a signup page, or a product detail page.  You can optimize for any metric, though–some of our customers regularly test headlines on news stories in order to drive engagement, for example.

How do you determine if results are statistically significant?

Optimizely automatically displays the results of a T-test on your data, which is helpful, but we also display the cumulative conversion rate over time, which is a great visual way to verify that your conversion rates have stabilized over time.  I’ve attached a great example–these are actually the results of an experiment we ran on our own home page a while back.  We tested several different calls to action, and you can see that the winner outperformed the control by more than 25%.  The conversion rate for each variation is graphed over time, and you can see that even as the conversion rate fluctuates with traffic makeup, the _difference_ between variations remains relatively constant.

To see Optimizely in action, check out the video below: