A subtle tweak, an overhaul, a redesign — sometimes your website requires some changes but totally overhauling your site to one final version won’t give you the data you need to prove those changes worked, i.e. increased conversions and/or improved user interface.  Employing testing tools allows you to experiment with the options that best fit your customer needs, via either A/B or multivariate testing.

A/B and multivariate testing allow you to test two or more versions of a webpage to determine which changes produce the desired results. A/B testing allows you to simultaneously serve either version A of your page, often the current version, and version B, the new and possibly improved version, to site visitors and see which performs better.  A/B testing is best for testing either a drastic change or testing one element.  If you plan to test a combination of changes – that is, separate the changes into distinct components – multivariate testing is the better option.  For example, if your page contains a call to action, headline, and image, you would have two versions of each, giving you eight combinations totals (for you math nerds, that’s 2³) as illustrated below.

Variations of Each Element

• Call To Action:   OR  
• Headline: “Ponies for Sale” OR “12 Different Styles of Ponies Available”
• Image:  OR  
  

8 Testing Combinations

•  + “Ponies for Sale” + 
•  + “Ponies for Sale” + 
•  + “Ponies for Sale” + 
•  + “Ponies for Sale” + 
•  + “12 Different Styles of Ponies Available” + 
•   + “12 Different Styles of Ponies Available” + 
•   + “12 Different Styles of Ponies Available” + 
•   + “12 Different Styles of Ponies Available” +
  

A/B and multivariate testing tools vary greatly and offer a variety of features and functions.  Below, you’ll find a tool comparison matrix explaining the differences between all eight tools, along with an interactive tool finder which will make a recommendation based on your needs and price point.

Below is a key to the dimensions we compared for each tool.

A/B: In A/B Testing you create two (or more) different versions of your website or landing page. Then you split your traffic amongst those versions to analyze which one gets maximum conversion rate or sales

Multivariate: In multivariate testing you test different versions of elements on your website or landing page (such as headlines, images, buttons, etc). You then split your traffic amongst those combinations to analyze which produces the most successful outcome.

Max pages in A/B test: Some tools like Google Content Experiments have a maximum number of pages that can be A/B tested, while others allow for an unlimited number of pages.

Max number of active tests: Websites with a lot of traffic could benefit greatly from regular testing on different parts of the site, thereby requiring several active tests.

Option to run on % total traffic (traffic throttling): Controlling the percentage of traffic that is directed to each variation allows you to test some variations more heavily than others, e.g. variation A gets 10% of traffic and variation B gets 25%.

Option to run on % total traffic: The option to run on a percentage of total traffic enables users to limit testing to only a subset of their visitors, avoiding running the test on everyone.

Heatmaps & Clickmaps: Heatmaps and clickmaps allow you to see where visitors are clicking on a webpage.

WYSIWYG page editor: A WYSIWYG page editor provides a simple interface for creating variations, for which knowledge of HTML is not required.

Time to see results: A/B and multivariate testing results can be viewed as quickly as real-time or take as long as two weeks.

HTML/Javascript editor: HTML and/or Javascript editors are a more technical interface for creating the variations.

Ability to edit variations during test: Some tools allow you to edit the variations while the test is still running, before the experiments end date/time.

Revenue Tracking & Goals: In some cases, conversion goals have different monetary values, thereby making tracking various revenue metrics important.

Support for dynamic websites: Testing tools that feature support for dynamic websites allow you to create variations of dynamic content, including lightboxes or mouseover pop-ups.

Targeting or Pre-Test Segmentation: Targeting or Pre-Test Segmentation enables you to target content to different groups of visitors based on marketer-defined segments.

Predictive learning: Predictive learning enables you to store each visitor and click individually, so targeted offers can be served based on a visitor’s unique interests.

Mobile API: A Mobile API enables testing within mobile apps.

Integrate third party data: Some tools can integrate third party data including customer databases and surveys.

Segmentation of results: Segmentation of results, implemented post-test, is analyzing results separately for different segments.

Content Rotation: With a landing page campaign, the visitor is not locked into a single offer displayed – they can experience different offers as their profile changes. Content rotation allows you to pass data into the platform through the tags and further enhance the visitor profile stored by the platform so you can target variations of content to the visitor.

Method of testing: Based on how you distribute traffic to your combinations, there are several types of multivariate tests. Read article for definitions of the different testing types: http://bit.ly/e3MTdP

Campaign Preview: Campaign preview allows you to see previews your different variations before you start the test.

Still have questions on A/B and multivariate testing tools? Contact us – we’re here to help!