A/B AND MULTIVARIATE TESTING
The basic user course covers the five steps to using Web analytics. Step 4 is to validate your site modifications by using sample groups. The process of using these sample groups to help to help determine the effectiveness of your modifications is called A/B testing. This is, in a nutshell, testing your original page or site element versus a new one. This process is a very practical and powerful way to use the different segment discussed in this section.

  • Define A/B testing.
  • Explain different testing techniques.
  • Discuss validation of sample groups.
  • Understand the available tools for A/B testing.
WHAT IS A/B TESTING?
Imagine that we own a company that specializes in baking cookies. We have a solid customer base that regularly buys our cookies. We decide that we would like to improve our cookie recipe and make a few changes. The worst thing we could do is replace our old recipe without verifying that the new recipe is better. We would risk losing our customer base if the new cookie doesn’t taste batter then the original. In order to refine and prove our new cookie better, we would create sample groups or taste testing groups to test the new cookie and help us determine whether we should distribute them.

Internet marketing and business are a lot like this cookie company. Many companies make both subtle and major changes to their public Web sites and marketing without properly validating that the changes improve upon the original.

As you start to make changes to your campaigns and site, use sample groups (taste testers) to help you verify that your changes are returning the effect that you desire. This can be done by conducting A/B Testing.

If you attended the Omniture University Site Catalyst User Training, you will remember talking about the process of using Web Analytics. This is what A/B testing is all about. It is the process of changing your site for the better. Here are some main steps that are included in A/B testing:

1. Determine areas of the site that have opportunity to positively affect conversion.
2. Hypothesize on possible changes using the current data to validate your conclusion.
3. Test your hypothesis by running multiple versions of a page.
4. Use data to tell you which page had the best conversion ratio.

THE OMNITURE EXAMPLE
For example, Omniture’s corporate site (www.omniture.com) is constantly running A?B testing on its internal campaigns. Omniture’s site is a lead generation site, so completing a lead is the success event. On Omniture’s home page there is a large area dedicated to internal campaigns. This area houses offers for those that complete the lead form. This offer might be a new white paper on media tracking or another on how to increase conversions. Once visitors complete the lead form, they can download the white paper. Omniture often conducts A/B tests on this area of the site. Different sample groups will see different versions of the campaign (a process that will be covered in the next section). The visitors that act upon these campaigns are tracked using an eVar(Custom Commerce Variable). Then, Omniture can run an analysis to see which version of the campaign is resulting in the most leads and make it universal.

This same strategy can be applied to your site pages and elements using the techniques in this section.

A/B TESTING TECHNIQUES
After you decided what pages or site elements you want to test, the next step is to determine your preferred testing technique. The two major techniques are

1. Consecutive Testing.
2. Synchronous Testing.

CONSECUTIVE TESTING
Consecutive testing is the process of presenting all visitors with one version of the page (Version A) for a period of time and then switching the page to a new version (Version B) and presenting it to all visitors for the same amount of time.

For example, if you were testing campaign landing pages, you might test one across all of your traffic for one week and then test the next version across all your traffic for the next week.