Guy Yalif, Intellimize CEO
     Guy Yalif, CEO/Co-founder


Personalization correctly applied always outperforms “one size fits all” approaches.

Why predictive personalization is replacing A/B testing

Performance marketers today face a conundrum. On the one hand, you’re tasked with creating marketing campaigns that appeal to wide—and sometimes very different—groups of prospects. On the other, those campaigns and efforts must deliver results. Why is it a conundrum? Because a single strategy, no matter how refined and researched, cannot optimally appeal to all of your visitors.

Performance marketers have turned to A/B testing to identify the single best page for all visitors. While A/B testing has helped marketers better quantify the impact of their ideas, trying to find the single “one size fits all” best page to show all of your visitors leaves money on the table and wastes time and effort.

Let’s explore an example to illustrate this. Say you are responsible for optimizing the conversion rate of a website and the current messaging that performs best (which we’ll call the baseline) converts 3% of your visitors. You decide you want to see if different messages can perform better, so you create two new test variations. In this example, you have three different, equally sized audience segments visiting your site: existing customers, prospective customers and competitors’ customers.

Conversion Rates for 3 Different Variations to 3 Audience Segments (with equal traffic) 

Baseline 1 2
Audience Segments A: Existing Customers 3% 2% 1%
B: Prospective Customers 2% 4% 1%
C: Competitors’ Customers 1% 3% 5%
Average conversion rate 2.0% 3.0% 2.3%

Pick one to show to all: 3% conversion rate overall
Personalize to each segment: 4% conversion rate overall (33% lift)

If you could only show a single variation to every site visitor, you would select Variation 1 because it delivers the best overall conversion rate across all customers (3%). However, if you could personalize the experience for each group and show each group the variation that performs best for them, you would show Baseline to existing customers (3%), Variation 1 to prospective customers (4%) and Variation 2 to competitors’ customers (5%). This personalized approach would result in a 4% conversion rate from the same ideas and the same group of visitors — a 33% improvement in performance.

No matter how individual variations perform personalization correctly applied always outperforms “one size fits all” approaches.

While rules based personalization is better than a “one size fits all” approach, you are required to set up static rules to deliver specific experiences to predefined segments of your audience. Predictive personalization uses machine learning to present the best experiences to each individual visitor to your site. Additionally, both A/B testing and rules based personalization optimize for a point in time, irrespective of how your visitors’ behavior changes or how your marketing efforts change in the future. Predictive personalization automatically adjusts to changes in visitor behavior over time, shifting traffic to your best performing experiences.

There are three advantages predictive personalization offers marketers over A/B testing:

Faster Results: Predictive personalization begins optimizing without waiting weeks or months for statistical significance, unlike A/B testing. Predictive personalization also allows you to test more experiences and variations at the same time than A/B testing so you can see results across more ideas sooner.

Better Results: Personalization outperforms “one size fits all” approaches by serving the best performing experiences to each user. Additionally, as visitor behavior changes over time, predictive personalization adjusts accordingly to deliver the best performing experience. A/B testing, on the other hand, picks a winner once and does not adjust again.

Less Work: A/B testing requires your ongoing attention, monitoring experiments, deciding when to call winners, and managing potentially large matrices of separate test cells. Predictive personalization automates experiment management and execution, freeing marketers to spend more of their time understanding prospects and developing new ideas to drive conversion.

In the year since we launched a predictive personalization system, we’ve seen that these benefits are real. Perkville, a referral and rewards program increased customer referrals by 70% in its first few months. Chime, a bank focused on helping consumers lead healthier financial lives, saw a 79% lift in new account signups four weeks after introducing their second round of ideas.

Any performance marketer who has a frustratingly long backlog of ideas to test and wants to see results quickly should be investigating predictive personalization. A/B Testing, gave us a way to understand the best single experience for all visitors at one moment in time, and predictive personalization now gives us a way to deliver the best experiences for each and every visitor as visitor behavior changes.