Industry 4.0/IIoT setups are often distributed and consist of hundreds of sensors, devices or machines. Nowadays, powerful edge devices enable the execution of software services like Artificial Intelligence close to production machines to reduce latency. Although several supporting Industry 4.0 platforms do exist, industrial stakeholders aim for more open, interoperable and vendor-neutral solutions. In this paper, we focus on the monitoring of runtime properties in Industry 4.0 for services and (edge) devices. We propose a realization based on existing components like Micrometer, MQTT and upcoming standards such as the Asset Administration Shell. We discuss patterns for their integration, the impact of the patterns on performance in terms of an experiment as well as experiences that we made so far.
The digitization of industry increases the performance of technical systems and their processes, but also their complexity. Particular complexity arises as Industry 4.0 installations are inherently distributed. In such settings, edge devices allow for handling the high data frequencies and volumes provided by sensors and production machines. Moreover, modern edge devices enable the safe execution of hard real-time machine control operations and (soft real-time) IT functionality, such as Artificial Intelligence (AI).
In the BMWi-funded project IIP-Ecosphere1, we research approaches to address typical shortcomings in Industry 4.0 systems regarding openness, interoperability, or vendor-neutrality. At the heart of IIP-Ecosphere, we develop a prototypical AI-enabling Industry 4.0 platform that allows for exploring the approaches developed in the project. As a core functionality, the platform shall automatically deploy and manage (AI-)services in a uniform manner on heterogeneous resources such as edge devices, on-premise servers, or cloud resources.
Within this context, we focus in this paper on a runtime monitoring approach for (AI-)services.
Our research question is: How to design and realize eficient resource and service monitoring in
an open, interoperable and vendor-neutral manner? Our approach combines a recent Industry
nEvelop-O
LGOBE
CEUR
Workshop
Proceedings
4.0 information/interface model, the Asset Administration Shell (AAS) [
For these components, we introduce three integration patterns and analyze them in a performance experiment. While a local integration of monitoring and AAS performs best, it implies also higher resource usage, a potential obstacle for resource-constrained edge devices. In contrast, a distributed installation increases response times. We show that a combination with publish-subscribe protocols helps balancing resource usage and response time.
Structure of the paper: In Section 2, we provide background information on used standards and technologies. We present our approach to Industry 4.0 resource and service monitoring in Section 3 and discuss initial experimental results in Section 4. In Section 5, we briefly outline related work, and in Section 6 we conclude the paper and give an outlook on future work.
In this section, we introduce standards and technologies that we selected to meet the goals of IIP-Ecosphere. For the realization, we aim at maximizing the use of Open Source.
Micrometer is an Open Source vendor-neutral application metrics facade with support for monitoring tools like Prometheus or Dynatrace. Micrometer defines so-called meters such as timers, gauges, or counters, which also carry their unit of monitoring, e.g., items/s.
A range of protocols is used in IIoT, including (legacy) machine protocols such as Fieldbus (the
ifeld level is not in our scope as stated in [
! " #$ % &
Recently, the Asset Administration Shell (AAS) [
partners opted for an extensive exploration. Essentially, as illustrated in Figure 1, an AAS is a nested structure containing typed properties, operations or references between elements. An AAS can be static, dynamic regarding values or structures, or active through callable operations.
Eclipse BaSyx4 is the de-facto AAS reference implementation. In BaSyx, an AAS is represented as a REST structure that can be served by a local process or deployed to a remote server. Properties and operations can be realized through functors attached to an AAS that perform local or remote method invocation, e.g., via the BaSyx Virtual Automation Bus (VAB) protocol.
Our goal is to collect runtime monitoring data reflecting the resource usage and performance
of devices and services in Industry 4.0 installations. A realization shall provide access to the
monitored information for individual devices and services, but also serve as a basis for eficient
aggregation over all elements in an installation. Moreover, IIP-Ecosphere imposes further
requirements [
To realize the monitoring approach, we made some basic decisions (cf. [
The mentioned components can be combined in several ways, implying individual performance impacts. Figure 2 illustrates some alternative patterns, all including services with defined
5https://www.eclass.eu/anwendung/verwaltungsschale.html meters on the device side, either a local or remote AAS to represent the monitored data, and a client reading the remote meters via Micrometer proxies. Our patterns are: a) A local AAS executing the Micrometer environment, which, in turn, observes the services.
The AAS polls the monitored data when a property is accessed, i.e., its functors perform local calls. To publish the AAS, an HTTP server process must be running on the device. b) A remote AAS communicating through a (light-weight) AAS implementation server on the device. As in a), the data is polled, here through (VAB) remote method calls. c) Remote pub-sub AAS: A scheduled task on the device publishes the data on a channel . The AAS subscribes to and holds the received data in a local cache bound to the AAS property functors. If communication from the AAS to the device is needed, e.g., to reconfigure the monitoring, a protocol as in b) can additionally be used.
As a BaSyx AAS is represented as a REST structure, all patterns access the AAS from the client through REST. Patterns a) and b) exclusively rely on AAS functionality and particularly follow the examples provided by BaSyx. In contrast, in pattern c) we connect device and AAS/client using asynchronous communication and transform the polling from pattern b) into an approach that relies on local calls to a cache instead of remote method invocations. In this pattern, further components can subscribe to the channel and operate even without access to the AAS. Patterns a) and b) may also be combined with some form of pushing, which we do not detail here.
To analyze the eficiency of the patterns introduced in Section 3, we present now an experimental evaluation of the individual response times. We realized the patterns as variants of the same Java code6. As basis, we use a simple workload in terms of two interconnected Spring Cloud Stream services (Spring version 3.1.1). The Micrometer (version 1.6.4) instances represent more than 50 default Spring and custom meters. A BaSyx AAS (github version of January 2021) defines a representative structure of properties and operations for all variants. A client reads the AAS through our Micrometer proxies and records the individual response times.
We use the local AAS pattern as baseline. However, Spring and BaSyx conflict when being
executed in the same JVM. Thus, we use corresponding meters without service workload as
fallback baseline. For the remote AAS pattern, the client polls meter data via VAB. As the meters
are published by Spring in terms of REST by default, we opted for a VAB implementation server
that accesses the meter information through an internal REST client [
As experimental environment, we use Windows 10 (Dell 7490, Intel i7-8650U, 1.9MHz, 32
GByte RAM, Open JDK 13+33) to emulate a development setting and Ubuntu Linux 20.04
(VMWare VM with 6 vCores on a Xeon E5-2650v4, 2.2GHz, 12 GByte RAM, Open JDK 13.0.7)
as a server environment. Both setups use local network only. A single iteration of a variant
reads 62 AAS properties representing gauges, counters and timers. Initial experiments in [
indicated, that the measurements stabilize at around 175 iterations, probably due to caching or JVM/JIT settling operations. Here, we perform 200 iterations for each variant. To prevent warm-up outliers, we exclude the results of first two iterations per variant.
local a) Gauges VAB b)
AMQP c) local a) Timer VAB b)
AMQP c) local a) Counter VAB b)
AMQP c)
Windows avg [ms] 1,3 5,5 1,2 0,8 5,5 1,6 0,6 3,6 1,1
We addressed a combination Industry 4.0 protocols, Micrometer and AAS. For IIoT protocols,
a body of work is published with a certain focus on MQTT. Two interesting related works
are [
Industry 4.0 systems do not only require complex, distributed setups, but sometimes also the use of prescribed protocols and standards. Moreover, future Industry 4.0 systems aim at increased openness, interoperability and vendor-neutrality. In this context, we analyzed whether Asset Administration Shells, Micrometer and protocols such as MQTT or AMQP can be combined to realize eficient runtime monitoring of resource and service properties. We proposed three basic integration patterns and discussed the impact on the performance in terms of an experiment. In essence, a more complex publish-subscribe implementation can outperform techniques that are (mostly) shipped with the utilized implementation frameworks. Moreover, eficient updates of meters may require specific attention, e.g., asynchronous schedules.
In future, we plan to explore the scalability of the approach when monitoring a larger set of devices. On the one side, an overview of all monitored values through an Asset Administration Shell is demanded by our stakeholders and can be provided as shown in this work. On the other side, we expect that a realization of further platform components, e.g., a monitoring aggregator, can eficiently be achieved through an integrated publish-subscribe approach.
IIP-Ecosphere is partially supported by the German Ministry of Economics and Energy (BMWi), grant 01MK20006D.