Previously in parts 1 and 2 of this series, we covered the overall landscape of personalization in an effort to get marketers aware of the processes and options for personalizing a campaign. In this final post, we will review some of the challenges for measuring the effectiveness of personalized content and incorporating personalization into one’s analytics and CRO strategies. This post will provide some best practices to better understand the effectiveness of your personalized campaign.

By now, I’m sure marketers have read about the advantages of adding personalization to your site or campaign. How do we measure the benefits of personalization once one has implemented personalized content? The answer is not as complex or different from what you might expect. You will want to use many of the same analytics methodologies used in measuring the impact of your online marketing efforts but with a few adjustments. In order to evaluate the lift in performance of personalized content, we need to compare it using the same language used to analyze the performance of all the other campaigns and site performance. Thus, the main areas to focus on are KPIs, segmentation, and benchmarks. We’ll discuss the specific ramifications to consider when evaluating personalized content performance using these measures.

KPIs

Part 1 of this series discussed the importance of goal setting when setting up personalized content. This importance carries through into the analysis side of things because the original goals you set will provide critical context to evaluate how successful the campaign is. Whether your focus is a revenue increase, higher brand awareness, or higher engagement, you’ll want to decide what KPIs best describe your goals for a personalized experience, as early as possible. This will allow you to ensure your analytics systems are properly set up and integrated to capture the data to support your KPIs. The most common KPIs to consider include conversions (purchases, downloads, etc.), revenue/purchases per visitor, and site/campaign engagement metrics such as bounce rate, pages/visit(session), AOV (average order value), cart abandonments, email opens etc. These KPIs will start to give you a sense of how your personalized efforts compare to your previous, static efforts at a high level. Note, when setting KPIs, try to think about how personalized content will specifically have an impact, and not just think in generalities, such as “I’d like x to increase by 100%.”

Segmentation

Carefully defining the target of your personalized effort is essential to accurately measure performance in a given vertical or channel. Here, the 4 W’s are applicable: who, what, when, and where.

Who: The most essential, top-level segments to make sure you are tracking are which visitors engaged with personalized content and which did not. Also important to consider is which responders are relevant to your underlying business goals. Are your personalization efforts directed at shoppers of a particular category, brand, or location? This is important, for example, if you’re trying to boost sales within the western region, but you’re mainly converting east coast users. Staying mindful of who you are targeting will keep your success in perspective.

When: This is a critical metric in any campaign but especially when dealing with personalization. Providing content and/or recommendations that accurately address seasonal trends is one of the most effective uses of personalized content. The date ranges of these events must be kept top of mind when analyzing the success of users’ engagement.

What: Once you’ve identified which users interacted with personalized content, break your analysis down by which elements were interacted with. For example, did product recommendations, video content, or custom landing pages move the needle the most on your main KPIs? Lack of movement for a particular type of content might need more refining on the modeling or algorithms that are driving the content presented to customers. This could also indicate the CTAs associated with your personalized content might not be aligned generating the proper sense of urgency especially at lower points in the funnel.

Where: This question mainly looks at which point within the conversion process did personalized content have the biggest impact, and is is closely related to the previously mentioned “what” question. Rather than simply looking at what was personalized, it is important to prioritize where in the user flow those elements appear. There are a number of factors that impact conversion rates and knowing at what point consumers are within the engagement flow of your campaign will help you better assess and contextualize the performance of each individual element of personalization.

Benchmarking

This is one of the most essential aspects of any campaign/site analysis but is especially critical when dealing with personalized elements. I recommend using both internal and external benchmarks when assessing performance. Internally, as with any analysis, compare your results to annual to date, YoY (year over year), monthly, and rolling benchmarks. As much as possible, incorporate benchmarks for each of the KPIs you are tracking, regardless of how granular. The more context you can provide, the better you can analyze not just if your personalized elements are successful, but start to answer the question of why. Externally, it is always useful to see how your campaigns are converting versus others within your vertical. For example, according to recent Forrester Research, the average conversion rate for ecommerce stores is 2.9%. How does your site compare? How did personalization move you toward or past this overall average?

Conclusions

This series has covered a lot of ground, from the basics of personalization, an overview of personalization products and solutions, and a rundown of the analytics best practices for assessing the performance of your personalized content. Each of these topics are complex enough to yield a series of posts on their own, so this series is intended to simply be a general introduction to each of these topic areas and get you asking some essential questions to start incorporating and measuring personalization within your campaigns and sites. The hope is that by learning more about this process via this post, other posts, tutorials, etc., you can also start to understand the potential value and positive impact personalization can have on your marketing efforts.