All the technical detail, expertise and advice


Real-Time Data Feeds

Published: Thursday, 05 June 2014 10:18 by Paul Siddall, Senior Pre-Sales Consultant
Big Data Analytics

Beyond Reporting to Real-Time Marketing

An increasing number of  systems say they offer real-time data feeds, allowing a company to consume and then action data in real-time. When looking at the system intended to consume this data and turn it into actions however, it’s important to understand the quality and richness of data in the feed and how immediately actionable it is.

Increasingly web analytics companies are offering real-time data feeds of the data collected from clients’ websites. Events within those data feeds contain single actions with some segmentation details included, such as the referring campaign. Whilst this level of data is good for building real-time dashboards and tracking actions with limited segmentation, it does not offer enough detail for fully understanding an individual, for example, what has been entered into a form which was subsequently abandoned or changing values shown within a page e.g. stock availability.

If a full understanding is required at an individual level, including a history of previous interactions, then some form of historic access database must be built which has the ability to access and update data as required. Moreover if that understanding is required across the whole session to drive analytics and actions in real-time, then the incoming data feed must be modelled and stored in memory.

Once a system has been built to understand the user in real-time, the next step is to take action on the data. This means some form of monitoring system to watch for certain “states” within the data which would then trigger an action. For example, if on the mortgage calculator a visitor indicated that they are a male aged between 18-25 then went to the loans page you may wish to take a different action than if they said they are a female aged 40-50 and then went to the home insurance page.

The typical actions taken from real-time data are to trigger another system to take an appropriate action, or to make a communication with the user directly via the web page. Most web analytics solutions do not support the ability to send a message back in to the web page, thereby removing the ability to communicate with all users by limiting the functionality to only communications with known users for whom they have an email address or other form of communication.

Celebrus builds an in-memory session state alongside a historic individually focussed datamart of contextualised data. The system continually processes and queries the data to determine when actions should be taken, both by evaluating the current session state as events occur and via regular timed assessments. Actions are then driven through integrating with an external system – for example a real-time decisioning engine or communications tool – with options to determine what aspects of that session state should be sent and what format the data should be sent in. The external system receives that important data, along with the relevant context, in a format that it supports and then chooses which action is the most appropriate to take. The fact that Celebrus has provided contextualised individual-level data about current and previous sessions in a format that is easy for that external system to understand and action means the data processing is faster and easier.  

Additionally, Celebrus’ own real-time software can execute the decision chosen by the engine and push the appropriate content back into the web page in real-time. This enables true one-to-one real-time personalisation and turns real-time data feeds into the best possible actions that deliver results and not just high level insights to build reports.

Paul Siddall is a Senior Pre-Sales Consultant