Case Study: Using Analytics to Optimize your Content Strategy

Mallory Mongeon | 2014-10-16T15:00:54+00:00 | email on acid logo

Do you really know your audience?

What does it take for a marketer to create a killer campaign that spurs their audience into action? The answer, simply put, is to “know your audience.” So what does “knowing your audience” actually mean? No, it doesn’t involve showing up at a random prospect’s door and asking ‘em out to lunch (that would just be weird). Instead, knowing your audience as a marketer boils down to leveraging advanced email analytics and reporting tools to uncover their interests, pain points and motivators. By utilizing reporting tools that will uncover real-time behavior, marketers can find out what incentivizes their audience so they can craft the most compelling CTA!

charity: water recently did a fantastic job of A/B testing using our email analytics to uncover their audience’s motivators. For those of you who are unfamiliar with charity: water, they are a non-profit organization with a mission to bring clean and safe drinking water to everyone on the planet. Part of being part of a non-profit means asking for money. Asking for money is an uncomfortable task but something charity: water knew they had to improve upon. To see which message most incentivized readers to donate, charity:water set up 4 different emails and landing pages with our analytics reporting. Read on for the findings!

charity: water’s Multi-Variant A/B Tests

charity: water wanted to see what type of messaging resulted in the highest number of donations to bring clean water to Bangladesh. To test their messaging, they identified two key elements they thought would increase their conversion rate within their copy:

  • Adding the specific country that will receive the clean water.
  • Matching every recipient’s donation.

Their initial hypothesis was that adding BOTH the elements mentioned would result in the highest open, click-through, conversion and engagement rate.

In order to uncover their “perfect” promotion, the following marketing materials were created below:

  • 4 email variants testing out different messaging
  • 4 landing pages, consistent with email messaging

Below are the 4 email/landing page variations.

Variant 1: Control Offer

The first email promotion with a dedicated landing page is what they called their “control” page. This is considered their control messaging because it severely downplays the two motivating elements they are testing. The control messaging focused simply around the gift of giving clean water, while breezing over the fact that the donations will be matched and the water will go to Bangladesh.

Variant 2: Matching Only Offer

The email and landing page prominently highlight the fact that every donation made will be matched and will directly fund the clean water projects. There is no mention of where the donated water will go.

Variant 3: Country Specific Offer

This email and landing page focus on where the clean water is being brought to. They hit hard on the lack of water and presence of waterborne diseases in Bangladesh as motivators for their audience to give. Only in one sentence is it mentioned that 75% of their donations will be matched.

Variant 4: Matching + Country Specific Offer

The copy highlights the fact that the money gathered from this fundraising event will bring clean water to Bangladesh, and they also let the reader know all donations will be matched, making both testing elements present.

Here’s what they found…

charity: water shared with us their average open, click-through and conversion rates they have seen from past campaigns. We used this average as a baseline to measure their results and the success they saw. All the variants in this case study outperformed the average rates they normally see, so all the statistics below reflect a percentage in increase from their baseline stats.

Open Rate Increase CTR Increase Conversion Rate Increase
Control Offer 2.26% 0.80% 1.91%
Matching Only Offer 3.70% 0.24% 6.71%
Country Specific Offer 2.46% 0.40% 6.29%
Matching + Country Specific Offer 2.48% 0.11% 2.90%

From the results above, the Match Only promotion dominated and came in with the highest open, read, CTR and conversion rate. Ding, ding, ding! Looks like charity: water has a winner.

Applying What They’ve Learned

By executing their multi-variant test with our analytics, charity: water was able to decipher that their most compelling offer is letting their audience know their donation will make twice the impact. They also uncovered that their audience doesn’t need to know what country receives the funds, proving their hypothesis wrong.

On top of run of the mill stats like opens and conversions, charity: water also leveraged engagement tracking. We define engagement as what happens between an open and a click (i.e. how long someone has an email open). From their engagement results, their matching-only campaign not only did well with typical stats like open rates, but it also garnered the most engagement from their audience. Their audience’s eyeballs stayed on the matching only offer more than any other variation. This matching-only campaign was deleted and skimmed the least as well.

These results allowed charity: water to decipher their most inspiring call to action before the holiday season rolled around. This was a critical discovery for them because the holiday season is also typically the biggest giving season in the US. Having identified the most successful incentive as matching donations, charity: water can create a holiday campaign that will be the most likely to increase click-throughs and conversions. All that hard work and analysis translates to more water for those in need and better ROI. If you are interested in learning more about charity: water, sign up for their newsletter to receive updates and stories from the field!

How are you measuring success?

What you don’t know can hurt you, and that’s why integrating advanced analytics with your email efforts is critical. According to Unica’s State of Marketing in 2011 report, 57% of marketers cited “measurement, analysis, and learning” as the biggest bottleneck they face within their organizations. If you fit into the majority of marketers, we may be able to lend you a helping hand. With our advanced reporting and tracking tools you could compare campaigns side by side, track how long people are reading your email, find which email clients have the longest engagement, get reports on geolocation, and so much more. Check out a sample analytics report here to see what kind of insights we can offer.


Give us a shout in the comments section below and tell us how email analytics have improved your bottom line!

Submit a Comment