Will your popup land you with a penalisation from Google?
March 28, 2017

How I Improved Home Page Engagement by 20.6% with a Simple A/B Test

Headline Split Test Blog - Brodie Clark
Alpha Digital
Our values are at the core of our business and drive us every day. We love working with clients and partners with similar philosophies!

Brief

The client is a Brisbane-based manufacturer of high-quality outdoor umbrellas. The website’s home page currently explicitly states their product’s monetary value by including pricing in the headline. Although this immediately eliminates those who are unable to afford their products, we believe it could be hindering engagement and de-motivating visitors to explore the website (an important step in converting visitors).

Description and Hypothesis

Users wanted to view product images and know specifications, although weren’t clicking the primary home page calls-to-action (based on Google Analytics data). We want to know whether mentioning pricing as the first experience with a brand is a positive one, and whether not mentioning pricing forces users to explore and develop their own judgement of value. Additionally, we want to amplify the effects of our changes by developing a shorter headline that we know users are going to read. The combined effects of these changes will result in improved engagement.

Results

split-testing-comparison

A/B Split Test Results

Although all tracked goals displayed improvements, we were unable to reach a statistically significant result for all goals due to the sample size. We were however able to attain a statistically significant result for overall engagement on the page, displaying a 20.6% improvement.

Control = 49.67% Conversion Rate

Variation = 59.9% Conversion Rate (+20.6%, 98% sig.)Optimisely Split Testing Results

Crazy Egg Scroll Map Results

Results from the Crazy Egg Scroll Map revealed that users who were exposed to the Variation headline scrolled further down the page. Approximately 65% of users exposed to the new headline reached the primary product categories, compared to approximately 35% in the Control.Crazy Egg Scroll Map Results

Next Steps

Having a home page headline that states pricing is like trying to sell tickets to a concert without saying which band is playing. Why focus on the price when the band is enough of a draw card? Although the test was successful in improving engagement, we now need to optimise the user’s experience site-wide to ensure visitors are converting.