The processing industry is home to a growing trend towards processing production data close to the sources of data. This on-site processing in the manufacturing facility, also known as “Edge Analytics”, superbly complements a selective, centralised storage of this data for further processing on premises or in the Cloud. In certain deployment scenarios, Edge Analytics gateways such as the HARTING MICA can also be configured as a local “MicroCloud” in the manufacturing environment. This gives the customer near-realtime access to its data as well as optimal control over data sovereignty in its production environment.
The central requirements in this context are the processing and storage of data streams and time series.
The Edge Analytics container is the perfect tool to get a quick success in the project.
In order to enable the user to efficiently develop appropriate solutions, the required tools are now pre-configured in packetised fashion and seamlessly integrated into the new analytics container:
- Near-realtime “in-flow” processing of sensor data via use of Apache Edgent
- Local, optimised storage and analysis of sensor, geo and JSON data based on IBM Informix
- Optional implementation of local MICA Analytics Clusters (“Manufacturing Cloud”) using IBM Queryplex
Why were these particular technologies bundled, and how can they be used in a specific project? The following paragraphs provide some additional information.
Apache Edgent for "In-Flow" processing
The OpenSource “Apache Edgent” project launched by the IBM Streams development team enables high-performance and parallel processing of sensor data on the MICA. These data can e.g. be aggregated via a sliding time window, and filter conditions can be applied and/or value-dependent actions can be triggered. Edgent can also be easily linked to the IBM Informix database locally on the MICA, for example to compare incoming sensor data with data from e.g. the previous day or week. In addition to the connection to Informix (via JDBC), Edgent comes with additional connectors which are simple to configure, e.g. a direct connection to MQTT, Kafka, HTTP, Web Sockets, IBM's IoT platform, etc.
IBM INFORMIX ALS SENSORDATEN HISTORIAN
IBM positioniert die Informix Datenbank besonders im Bereich IoT und Industrie 4.0, sowohl für den Einsatz im Bereich Edge Analytics, aber auch in der Cloud und On-Premises. Informix bietet als eine der wenigen relationalen SQL Datenbanken eine integrierte und optimierte Unterstützung von Zeitreihendaten (d.h. Sensordaten).
IBM positioniert die Informix Datenbank besonders im Bereich IoT und Industrie 4.0.
IBM informix as sensor data historian
IBM is positioning the Informix database in particular in the IoT and Industrie 4.0 area, both for use in the field of Edge Analytics as well as in the Cloud and on-premises. Informix is one of the few relational SQL databases to provide integrated and optimised support for time-series data (i.e., sensor data). Time series data are stored in Informix similar to a vector, so that e.g. the measurement values for one sensor are physically next to one another. In addition, Informix does not save the time stamps when saving sensor data which occurs at regular intervals, which means a considerable saving in occupied memory space – a decisive advantage when used on resource-limited Edge Devices such as the MICA. Last but not least, less data volume also means faster operations on these data. JSON-formatted sensor data can also be stored in an Informix time series, which allows a very high degree of flexibility when integrating a wide range of sensors without any subsequent modification of the data model.
If necessary, Informix can be configured as a round-robin database so that sensor data are automatically deleted by the MICA after a freely configurable time frame (for example, 30 days).
Particular emphasis was placed on the rapid processing of these data. In addition to more than 100 predefined SQL functions for time series data (e.g., aggregation functions, moving averaging, pattern recognition, spatiotemporal time-series data processing, etc.), there is also the possibility to perform custom functions directly on the data via a C programming interface in the database server.
Naturally, appropriately optimised data transfer interfaces in Informix also ensure that the large amounts of data collected in e.g. IoT projects can be transferred very quickly to the database. Also, linking to products for stream processing (e.g., with Apache Edgent or IBM Streams) is easily achieved – and often necessary – in such situations. Since the basic data model in Informix is relational, existing master data can be easily combined with the sensor data in queries without having to sacrifice the performance of the sensor data. For applications that can only access classic relational tables, Informix offers the ability to define virtual table views while maintaining Informix sensor data performance. In order to facilitate application development in the IoT context, Informix offers a variety of interfaces such as MongoDB, REST, MQTT, DRDA. JDBC, ODBC, .NET, SQL, and more.
IBM Queryplex is an innovative technology that allows multiple MICAs to be linked to a type of micro-cloud or to a MICA cluster. One application can be to obtain a holistic view of all the sensor data already collected on the MICAs in Informix for more complex analysis, without having to consolidate all the data in a central location beforehand. Another possible application is to use the MICA cluster as a computing cluster for more complex machine learning models.