Aqueduct: for big and fast data

EOH Big Data Lab has created an open standards-based fast data platform, called Aqueduct, that can capture, store and analyse multiple streams of data to perform Complex Event Processing.

Whereas traditional methods of data management have struggled with the high volume, velocity and variety of big data, Aqueduct can identify significant patterns – such as opportunities or threats – and then suggest actions to take advantage of these insights.

Aqueduct supports 4 distinct frameworks that allows rapid development and deployment for industry-specific applications:

 

  1. Aquifers – The Ingestion Layer
  2. Castellum – The Storage Layer
  3. Eductor – The Extraction Layer
  4. Fontana – The Presentation Layer

1. Aquifers

This layer of our big data platform ingests huge amounts of un-structured and semi-structured data, ad infinitum, in a fast and cost-effective manner for monitoring, real-time analytics and reaction, and in order to store data for later use. Once the real-time data analytics is done, it is then presented as consolidated information so that value can be extracted.

3. Eductor

This layer of our big data platform is the space from which data scientists extract and interrogate the vast quantities of data stored, ad infinitum. The technology underpinning this layer enables data scientists to query historical data by means of models and algorithms in order to, for example, establish trends in space and time.

2. Castellum

This layer of our big data platform stores semi-structured and un-structured data, continuously, and makes infinite amounts of data available. The platform’s storage capacity is easily and automatically scaled depending on your data storage needs, and is able to recover immediately in case of disruptions; thereby providing a seamless operation.

4. Fontana

This layer of our big data platform presents aggregated data from both the Ingestion and Extraction Layers via different methods including heat maps, histograms, dials and graphs. This layer allows the end-users to find value in the information presented and enables them to make more accurate predictions and informed decisions.

1. The Ingestion Layer

A primary function of the zenAptix Big Data Platform is to ingest huge amounts of un-structured and semi-structured data ad infinitum in a fast and cost effective manner for real-time data analytics and in order to store it for later use.

The zenAptix Ingestion Layer can ingest a variety of data streams, with different data-types, at an extremely high volume.

In addition, specific data changes can be monitored as the data is ingested. The system can then react to any change in real-time, for example through sending a notification or alert.

The Ingestion Layer is also the space in which real-time data analytics is done and presented as consolidated information in the Visualisation Layer from where value can be extracted.

3. Eductor – The Extraction Layer

This layer of our big data platform is the space from which data scientists extract and
interrogate the vast quantities of data stored, ad infinitum. The technology underpinning this
layer enables data scientists to query historical data by means of models and algorithms in
order to, for example, establish trends in space and time.

To complement these frameworks, Notebooks provide a collaborative interface to perform
exploratory data analytics, backed by Spark.

2. Castellum – The Storage Layer

This layer of our big data platform stores semi-structured and un-structured data, continuously,
and makes infinite amounts of data available. The platform’s storage capacity is easily and
automatically scaled depending on your data storage needs, and is able to recover immediately
in case of disruptions; thereby providing a seamless operation.

4. Fontana – The Presentation Layer

This layer of our big data platform presents aggregated data from both the Ingestion and
Extraction Layers via different methods including heat maps, histograms, dials and graphs.
This layer allows the end-users to find value in the information presented and enables them to
make more accurate predictions and informed decisions.

Aqueduct’s growing pool of data frameworks include:
1) GeoTemporal Indexing Service to relate disparate datasets across time and space
2) Monetisation of data supported by distributed ledger technology
3) Integration with blockchain-related smart contracts