How to Plan Your A/B Test

Crazy Egg makes it fast and easy to ship experiments that will help you improve your user experience. We do this by using the multi-armed bandit approach to A/B testing so that you can maximize your ROI. The value of our approach is that you can continue experimenting with different variants by changing copy, images, and calls-to-action, and always be assured that your website traffic will be directed to the highest converting version of your web page. 

While Crazy Egg automates the A/B testing process, it's key to plan out what you want to test before you launch a variant. Figure out your goals like increasing signups for a particular event, boosting trials of your software, sending increased traffic to specific pages, and/or selling more of your product. While our A/B testing tool allows you to iterate quickly, you can be even more targeted with your time by creating a hypothesis on how changing different elements will impact your conversion rate. 


Continuous A/B Testing and Multi-Armed Bandit Best Practices

Even though our A/B testing approach lets you run tests continuously so that you can reach your conversion rate optimization goals, one helpful approach is to launch up to three variants against the control at any given time. Once our algorithm has chosen a winner, retire the remaining variants so that the winning variant is the new control. You can then start the process anew by creating variants of the new control so that you can keep growing your conversion rate. 

To start off strong, follow these steps before launching your variants:

Step 1: Decide on an overarching goal like increasing signups, selling more products, getting higher page views, or all of the above. 

Step 2: Look for elements on your page that are most likely to tip the scale. For example, if you want to increase your signups, a good element to experiment on is a call-to-action button or banner. Change up the size, color, and/or copy to see what has the greatest impact. 

Step 3: Decide on a specific action like having visitors click on an element, submit a form, land on a certain page, or trigger a script that will mark an achievement that our algorithm can use to determine the winner. 

Sample Scenario

Say that you're offering a limited-time deal and you want to maximize your conversions for time period. In this instance, you could experiment with the copy on a large call-to-action button on your page. 

Control copy:

SIGN UP FOR 25% OFF
Join our mailing list and receive 25% off your first purchase.

Variant 1:

TAKE ADVANTAGE OF TODAY'S EXCLUSIVE OFFER! 
Join our mailing list to receive 25% off, plus other perks delivered straight to your inbox.

The new copy creates a sense of urgency, and may be a key factor in causing a spike in signups.

A/B Test Ideas for Inspiration

Not sure where to begin? We've put together some lists to help you brainstorm experiments that you can test on your site so that you can deliver a better user experience.

Pages to A/B Test

Try testing your:

  • Homepage
  • Landing pages like event registration or blog layouts
  • Pricing page
  • Demo page 
  • Product page(s) 

Elements to Test

The list of items you can experiment with using Crazy Egg's A/B Testing Tool include:

  • Calls-to-action: Wording, font-size, colors
  • Headlines: Copy, length, font-color
  • Product descriptions: Copy, style, number of words
  • Form fields: Add fields, hide fields, choices available
  • Navigation links: Order, number, variety
  • Buttons: Color, size
  • Images: Hide images, rearrange images, insert new images 
  • Page length: Increase or decrease the length of your page

Jump to:

Crazy Egg A/B Tests: Guide to Multi-Armed Bandit Testing

Still need help? Contact Us Contact Us