All the technical detail, expertise and advice
Really real-time expansion with expanded Celebrus data streamsPublished: Thursday, 14 September 2016 by Ant Phillips, Senior Developer
Celebrus has support for a wide range of data warehouse systems, everything from the traditional database vendors (MySQL, Oracle, SQL Server, Teradata et al), document-centric systems like MongoDB, through to the latest business intelligence platforms like Hadoop and Teradata Aster. These are all ideal for storing and processing the detailed and complete individual-level detailed data which Celebrus collects and generates. In Celebrus terminology, it is the Data Loader which is responsible for integrating with these data systems and feeding them the Celebrus data via the appropriate Celebrus Connector. Furthermore, the Data Loader can pretty much guarantee that the data is available in your target data system within 60 seconds of an event happening in a website or mobile application (for example, a visitor logging in or completing a purchase).
In recent years we’ve witnessed a really critical move towards real-time processing and streaming analytics. In these deployments we had previously been challenged with streaming Celebrus data to third party applications just as soon as it happened i.e. in true real-time of milliseconds. However, with us being increasingly asked to deliver our data into streaming applications to support uses including streaming analytics, real-time personalisation and fraud detection, we set about fixing that.
So, to support this real-time industry trend, we introduced Data Streams. A Data Stream is configured in much the same way as the Data Loader: simply specify which events you want to send (and the attributes thereof) and where to send them (via the Connector information). You are then good to go; it really is that simple and fast. It’s likely that you’ll identify specific data that you need in real-time so you’ll only pick the data and attributes which are important to have within milliseconds. However, using the complete data set for analytics can also inform which those key data points that you want in real-time are, (and might throw up some surprises), so the ability to adjust the Data Stream quickly and easily is really useful.
Streaming applications can also return content decisions to Celebrus which are used to personalise the website or mobile application. A content decision is typically an identifier which specifies a piece of content which should be displayed to the visitor (for example: Buy2Get1Free!). The actual content to display (HTML) is either configured in Celebrus or within your own Content Management System (CMS). What’s really compelling is that Celebrus orchestrates all these moving parts: website and mobile data collection, decisioning, content management and data loading, thereby enabling clients to integrate best-of-breed solutions for each part of their architecture.
The introduction of Data Streams adds another powerful string to our client’s proverbial bow, enabling them to decide what systems should integrate with Celebrus and how quickly they need the data. And with uses that are highly complementary to those enabled by our sub-60 second Data Loader technology, Data Streams enable our clients to use their Celebrus data foundations in even more innovative and valuable ways.