Skip to content
All Blogs

You built it, but they never came – Are your metrics lying to you?

Author: Tiffany Staples

magnifying glass metrics data numbers

Have you heard of the “last mile” in personalization? It refers to the time between when a decision to deliver content is made, and when that content or experience change is presented. Unfortunately, this is where many marketing technologies struggle because it’s very difficult for solutions to deliver on. With the complexity and speed of digital today, there are three main concerns with delivering real-time personalization:

  1. Was the content replaced after the default content had already loaded?
  2. Was the decision made quickly enough to deliver the content at all, or was it never presented to the individual?
  3. If it was placed on the page/screen, was it actually seen or had the user already scrolled beyond where it was placed or moved to a new page before it was loaded?

With the intricacy of digital across all channels, organizations often don’t even realize how few of their personalized offers or experiential changes ever make it to the consumer due to poor coding, data time-outs, and other common challenges. In addition, even if the content was delivered, there’s no real reporting on the visibility of those changes - i.e., did anyone actually see it?

Standard marketing solutions consider content as “viewed” when it’s delivered – meaning the page it’s supposed to be on is loaded. They don’t factor in whether the content was actually visible to the visitor. In addition to creating skewed performance reporting and inaccurate marketing attribution models, inaccurate visibility measurement impairs machine learning models. Machine decisioning relies on data and when that data is wrong it negatively impacts the resulting predictive models used for marketing.

The same goes for real-time personalization campaigns. You work hard to create the right message, for the right person, at the right time - you need to understand if those in-the-moment changes are effective or not. Visibility detection ensures your campaign analysis is concise and provides relevant, actionable data about your real-time personalization efforts.

The advantages of visibility detection within a customer data platform span the entire organization. Here are some of the most common benefits:

  • More accurate Predictive Analytics models for real time decisioning and outbound marketing, leading to increased sales
  • Marketing Mix Modelling or Multi Touch Attribution Models are improved, which drives better return on marketing spend
  • Marketing Performance/Funnel reporting will accurately reflect the relative performance of different pieces of content and messaging

Visibility detection improves machine learning (ML) and prevents bias

Decisioning engines rely on machine learning models to determine which messages work best for particular customers on different channels. The models are trained with data that includes if content was loaded, and whether a customer engaged with that content. If a customer doesn’t engage, machine learning assumes the message didn’t work, but what if the message wasn’t even visible?

The impact of poor data quality can be far reaching in terms of machine learning models because tests and decisions are designed to adapt based on updated information. The ultimate goal is to deliver messages quickly and efficiently, but decisioning models need accurate data to do this. If the content is counted as delivered simply because it's sent to the channel, it leads to an inaccurate analysis of what's working and what's not. When content is delivered but not seen, and the machine learning model uses that to analyze engagement, it creates a false positive. Likewise, if your models are being trained based on whether the offers are converting, but the offer was never seen, you may wrongly train the models to believe the offer isn’t effective. This is called machine learning bias.

Visibility detection generates the most accurate insights into how visitors interact with targeted content and provides protection from bias in machine learning models. This includes measuring when content is displayed ‘above the fold’, and standard visibility measurement according to Internet Advertising Bureau standards (at least 50% of the banner or creative must display on screen for more than one second). Generating visibility timings and coverage metrics also provides more accurate measurement beyond industry standards. To prevent false positives, visibility detection must pause measurement when content moves off the page (i.e., not visible as a user scrolls) as well as during ‘tab-over’ events in the browser. Incorporating true visibility metrics into your data capture and decisioning process provides accurate campaign results and improved machine learning models.