-The nowadays industrial digital revolution demands for software driven solutions where reconfiguration is one of the key enablers to achieve smart manufacturing by easy deployment and code reuse. Despite existing several tools and platforms that allow for software reconfiguration at the digital twin / edge level, it is most of the times difficult to make use of state of the art algorithms developed in the most popular programming languages due to software incompatibility. This paper presents a novel framework named Dynamic INtelligent Architecture for Software MOdular REconfiguration (DINASORE) that implements the industrial standard IEC 61499 based in Function Blocks (FB) in Python language for Cyber-Physical Production Systems' implementation. It adopts the 4DIAC-IDE as graphical user interface (GUI) to ease the design and deployment of FBs to quickly and on-demand reconfigure target equipment. The proposed framework provides data integration to third party platforms through the use of OPC-UA. The test scenarios demonstrate that the proposed framework 1) is flexible and reliable for different applications and 2) the CPU and memory workload linearly increases for a large amount of FBs. Index Terms-Cyber-Physical Systems, IEC 61499, Smart Manufacturing, Machine Learning
One of the key aspects of the nowadays fourth
industrial revolution is the digitization of shop-floor entities like
processes, equipment and components to increase their
interoperability with users and information systems. In order
to achieve digitization, an increased effort of standardization
is required to create uniformed interfaces that promote a
transparent communication among a set of heterogeneous
entities. This standardization is often attained with the concept
of digital twin (DT), and is often seen as a wrapper used
to integrate any device or process into a network, where
information can be easily accessed and shared [
With a change in paradigm from closed programmable logic controller (PLC) implementations to industrial PCs that allow a more flexible information sharing and storage, new opportunities taking advantage of this transparency can be explored. From multi-agent systems to artificial intelligence applications, democratizing data storage and sharing is key for the new advances in manufacturing systems, where machine learning is one of the key enablers of industry 4.0. This way, applications like artificial vision to detect production defects, predict when and why a certain component will fail or even explore new energy efficiency solutions are some examples of the well established importance of artificial intelligence in manufacturing.
Albeit not specific for manufacturing applications, there are
several platforms developed in order to ease the development
of such intelligent systems in a modular fashion, where the
main idea is to accelerate the implementation of an
end-toend solution without the need to know the technical details
of certain techniques. Some examples of this modular design
and execution are Rapidminer Studio [
However, the use of such platforms in industrial applications, mainly at the level of control systems, is not straightforward. On the one hand, some of these are Cloud-based solutions, which is still an obstacle for specific industries nowadays due to the industry’s policy, restricting the data access only to local network components. On the other hand, these modular designs do not implement any industrial standard, such as IEC 61499 adopted in 2005 and based in the function block (FB) definition that abstracts both software and hardware modules suitable for manufacturing requirements.
Based on this, the present paper proposes a framework
called DINASORE for the execution of digital twins in
Cyber-Physical Production Systems that is able to support
the latest advances in machine learning. DINASORE stands
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
for Dynamic INtelligent Architecture for Software MOdular
REconfiguration and is compliant with the 4DIAC platform
[
The same way FORTE is used together with 4DIAC for the low level interaction and execution of C/C++ applications in industrial equipment, DINASORE is a similar implementation but for Python FBs, which makes possible the use of important machine learning packages such as TensorFlow, PyTorch, Keras and Theano.
From this perspective, with DINASORE it is possible to develop a distributed control system for industrial applications which is machine learning-enabled. This framework allows to overcome the difficulty to use and develop state of the art algorithms due to the C/C++ (FORTE) requirement. Therefore, all the latest developments in the Python community, from deep learning to optimization, can be used in industrial applications with DINASORE. It is capable of online-reconfiguration of Python FBs, where an easy to understand Python FB template is provided for customized developments. On top of that, and due to industrial requirements, each DINASORE environment has an embedded Open Platform Communications - Unified Architecture (OPC-UA) server with a data model to facilitate the integration with third-party platforms. The OPC-UA data model abstracts the concepts of equipment and device for fully industrial integration with other devices or information systems. In sum, the main contributions and differentiation of this work lie in: 1) A Python implementation of a digital twin that is inte
grated with 4DIAC and compliant with IEC 61499; 2) A Python template to build FBs compliant with DINA
SORE; 3) OPC-UA data availability using an OPC-UA server;
The remainder of the present work is organized in five more sections. A literature review is made in Section II, and the DINASORE implementation explained in Section III, from its architecture to the Python template definition to develop new FBs. The following section presents the test case scenarios defined and the main results obtained. Finally, Section V will discuss the results obtained and draw some conclusions and future work for the current implementation.
One of the most widely used platforms for component-based
design and execution of distributed control systems is 4DIAC
[
There is already a great motivation in using these
approaches, mainly the development of CPPSs using the IEC
61499 standard, as can be seen in literature. A use case that
implements such an approach is presented in aluminum cold
rolling mill plant demonstrator where the authors design and
test an event-driven process based on IEC 61499 standard
using OPC-UA [
In [
Regarding the technologies used to support the development
of CPPS based on the IEC 61499, there are a set of
platforms already available that can be used. Some examples are
the Archimedes [
Some effort has been applied to the integration of UML
with the IEC 61499, where a case scenario composed by
a distributing and sorting process using FESTO FMS-200
FORTE platform was built resulting in a CPPS [
Regarding all the works previously presented, the DINASORE framework presents an additional, but important, step towards the implementation of the IEC 61499 standard using Python language, and consequently, the use of state of the art machine learning algorithms in CPPSs. Together with the OPC-UA for vertical integration, DINASORE can be seen as a powerful framework that can accelerate and strengthen the next generation of smart industry.
III. IMPLEMENTATION
Similar to FORTE (4DIAC-RTE) portable implementation in C++ of IEC 61499, DINASORE1 shares the same philosophy but in Python language. With all the latest artificial intelligence advances and current implementations in Python, DINASORE can be seen as a tool that enables advanced machine learning systems to execute as close to shop-floor equipment as possible. The further integration with OPC-UA for sharing a simple and intuitive data model allows for each digital twin DINASORE implementation easy to integrate with most information systems.
The graphical user interface (GUI) integrated with the DINASORE is the 4DIAC-IDE, which enables drawing and deploying distributed configurations, based in FBs. The 4DIACIDE uses the Eclipse Project as a core development framework, providing a GUI based in a desktop application, with the typical Eclipse IDE appearance. The process of development of a new CPPS based in FBs has as main steps: 1) the definition of CPPS network configuration, specifying for each device its IP address and port used for 4DIAC communication, 2) the drawing of the FB pipeline, drag&dropping FBs and linking them through specific connections, modeling the required software architecture, 3) the mapping of specific FB to devices, and 4) the deployment of the FB pipeline to the corresponding DINASORE devices. Therefore, the 4DIACIDE GUI has several views directed to the user to implement different steps, e.g. the network configuration, the development and the deployment views. After the development and deployment of the FB pipeline, 4DIAC-IDE enables to monitoring (watch) the whole system for real-time visualization of the current state of each data and event inputs and outputs in the configuration. Additionally, the 4DIAC-IDE allows the interaction with the actual pipeline, triggering events, stopping the configuration, or cleaning the actual DINASORE runtime environment resources.
The standard IEC 61499 defines several rules at the industry level, including the composition and structure of FBs, which is the graphical representation of a set of functionalities. Each FB contains events and variables to communicate with other FBs, where each event allows to trigger the execution of a certain FB, while each variable stores the data (e.g. sensor measurements, algorithm outputs). The 4DIAC-IDE uses FBs as elementary components that connect among themselves, using both events and variables forming a functional pipeline. Considering that, there are three types of FBs defined by IEC 61499, namely: 1) Basic Function Blocks (BFBs): In simplistic terms, they are state machines that according to specific events are able to execute the corresponding algorithms; 2) Composite Function Blocks (CFBs): It is a composition of BFBs making up a network of FBs to model more complex system behaviors; 1Available Online: github.com/DIGI2-FEUP/dinasore 3) Service Interface Function Blocks (SIFBs): It allows to specify how FBs should interface in terms of both events and data connectors to other FBs that execute (mainly) in different physical platforms (Machine-2Machine communication).
Additionally, the developer has the autonomy to implement their own FBs and integrate them in both 4DIAC-IDE and DINASORE runtime environment. The main type of FBs adopted in the DINASORE is the Basic Function Block (BFB), characterized by two files, an XML metadata file containing the FB structural information and a Python file implementing the code functionalities.
The main goal of DINASORE is to serve a gateway to machine learning into distributed control systems based on IEC 61499. As we believe the future design and deployment of CPPSs will definitely pass through the use of block-based technologies, such as 4DIAC-IDE and IEC 61499 as depicted in the related work, DINASORE can be seen as a key complementary technology to enrich the area of CPPS with artificial intelligence algorithms. Although integrated with 4DIAC-IDE, DINASORE is not designed in its root to execute in embedded systems, complying with real-time constraints as FORTE. The idea behind using Python language is to enable the latest advances in machine learning to integrate at the edge level with existing shop-floor equipment, without the need for cloud based processing. With the embedded implementation of OPCUA in the DINASORE function blocks (FBs), any external data is automatically provided in a OPC-UA server for further system integration.
As for DINASORE implementation, there’s the need to classify the used execution model type framing the technology into the IEC 61499 guidelines. One of the most well known categorization of Execution Control Chart (ECC) execution model was proposed by Ferrarini in [26], where 7 different classes were defined, from A0 to A6, exploring two dimensions, namely scan order and multitasking. The scan order refers to execution models that can have either fixed (predefined order), or dynamically (order calculated on-thefly) FB execution during the ECC, while the multitasking dimension refers to no controlled ways of multitasking, where FBs execute in a multi-threading fashion; done by time slice allocation to each FB execution (preemptive scheduling) and done by FB slice, where each FB executes at a time (nonpreemptive scheduling).
Additionally, there’s a small and brief survey of run-time
environment (RTE) platforms for IEC 61499 where 4 types of
execution models are introduced [
The DINASORE execution model implementation, Figure 1, uses a producer-consumer pattern, where each FB performs in a different thread, which is both producer and consumer. The data transmitted between FBs addresses both events and variables, where the FB object stores the input events in a queue and the variables in register attributes. Thus, each FB object has an internal queue for the input events, waiting until it receives an input event in the data structure, reading after the variables’ actual value, and processing the event’s functionality. After completing the functionality execution, the FB object pushes the corresponding output events in the queue of the following connected FBs. The same thing happens to the variables where the FB updates their output variables and consequentially the input variables of the following linked FBs. The actual value of each FB event and variable is available through the monitoring/communication interfaces, i.e., using the OPC-UA server or 4DIAC-IDE watch option.
Fig. 1. DINASORE Architecture.
The FB thread-object requires two external files to execute that compose the FB itself. The first file is XML-based and contains the meta-information about the FB, e.g. the FB type, the FB class and the FB structure composed by input/output events and variables, and their details, including the data type and the OPC-UA role (variable, method or none). Besides the DINASORE usage, the metadata file enables the 4DIAC-IDE to render the FB with their events and variables in the GUI. The second file composing the FB is a Python script encoding the FB functionalities, which requires the implementation of a class, assuming the Object-Oriented (OO) paradigm. That class requires the implementation of the schedule method, which receives as arguments the event name and respective value, triggering the FB execution, with the current input variable values. Then, according to the event received, the method selects the functionality to execute. After executing the functionality, the schedule method returns a list of output events and variables. Both schedule method arguments and output variables should follow the order specified in the metadata file.
Concerning the need for external communication between the DINASORE and other applications, e.g. 4DIAC-IDE and third party information systems, there are two different and independent communication interfaces integrated in the DINASORE, 1) the 4DIAC interface, which uses TCP/IP, and 2) the OPC-UA interface, which uses a data model XML file as a reconfiguration file. These two interfaces allow the reconfiguration of the current runtime workflows and the monitoring of each FB. The 4DIAC interface communicates with the 4DIAC-IDE using a TCP/IP connection, where the DINASORE interface executes a TCP server, receiving the commands from the 4DIAC-IDE enabling the creation, stop, and deletion of the configuration workflow and the runtime configuration monitorization. The interaction between 4DIACIDE and DINASORE for the configuration creation starts with several messages instantiating every FB (Create FB); each message contains the FB type and instance name. Then the 4DIAC-IDE sends the commands to create the connections (events and variables) between FBs (Create Connection and Write Connection), and finally the IDE requests the start of the FB threads (Start Configuration). After starting the workflow, the DINASORE enables the monitoring of each FB variable and event, through the watch option (the Create Watch message to activate the subscription; the Read Watch message to request the variable/event current value; and the Delete Watch message to unsubscribe). Additionally, the DINASORE allows from the 4DIAC-IDE to stop and reset (Stop Configuration) the configuration workflow, terminating the respective working threads and the remote trigger of an event (Trigger Event), performing a FB functionality in the DINASORE.
As an alternative, the DINASORE can use the OPC-UA Data Model as reconfiguration channel. This way, the DINASORE stores the configuration locally in the Data Model XML file, where registers all the used FBs, and their respective connections (events and variables). The FB XML metainformation classifies each FB in different sets, grouping them in 1) devices, 2) equipment, 3) services, 4) endpoints, and 5) start-points: 1) The device abstraction represents sensors, like sensors integrated using Modbus protocol; 2) The equipment representation uses a more complex structure allowing the aggregation of sensors and actuators, using events as relational connections; 3) The service type maps to FBs as a method and when an event is received, its execution is triggered. This is suitable for machine learning algorithms (e.g., Random Forest, SVM) or statistical operations (e.g., moving average, moving standard deviation); 4) The start-point type represents protocols interfaces that receive data, e.g. subscribing a MQTT topic or requesting TCP/IP data; 5) The endpoint type provides data through the defined interface, e.g. publishing in a MQTT topic and replying to a TCP/IP request.
Those representations make the development of new FBs easier by providing templates and specific behaviors to maximize the effectiveness of the Python code written in each FB. The (1) device, (2) equipment, and (4) start-point scripts adopt a loop template, which enables a cyclic execution of the FB. The (3) service and (5) endpoint types use the typical asynchronous approach executing when triggered.
Both OPC-UA and TPC/IP (4DIAC-IDE) interfaces are essential to DINASORE operation, enabling different features and capabilities. The 4DIAC communication interface allows the combination with a graphical user interface (GUI) for the development of workflows based on FBs. The OPC-UA Data Model, besides enabling the workflow reconfiguration, the primary purpose is to monitor the execution of a specific pipeline monitoring using the OPC-UA industrial protocol and the storage of the device configuration in an XML format. The configuration storage transforms the DINASORE in a more reliable and fault-tolerant platform, enabling the memorization of the current state of the workflow, being tolerant of power cuts restarting the runtime environment in the previous state. The Data Model XML file updates every time, when the DINASORE receives commands from the 4DIAC-IDE, for the creation, stop, and removal of configuration workflows.
The DINASORE evaluation focuses on the validation of the tool in different scenarios. Those scenarios point out the platform advantages and disadvantages, besides establishing the most suitable applications, like sensing, control, or data processing. The first scenario focuses on the use of machine learning (ML) algorithms, in particular classification methods, for the detection of collisions in a real mini-robotic arm based in servo motors. The second example consists of a typical distributed control architecture composed of two components (gripper and robotic arm) that have to synchronize between them to perform the required operation. The third case shows the simulation of a manufacturing production line. Based on the simulation scenario, several experiments with increasing amounts of FBs allow to assess the scalability of the framework in terms of processing and memory usage.
The main goal of the collision detection implementation is to transform a typical robotic arm into a collaborative one. That scenario uses a robotic arm based in servo motors that provide load metrics able to infer about possible collisions with obstacles. Based on this, the process 1) monitors each servo motor, 2) detects when one servo motor is in overload, and 3) stops the robotic arm if it collides with an obstacle. The network architecture contains two devices, both Raspberry’s Pi: 1) responsible for monitoring the servo motors and check the motors overload (Figure 2 rose FBs), and 2) controlling the robotic arm, sending instructions to perform a particular task, and waiting to receive an overload alert to stop its execution (Figure 2 green FB).
The servo motors sensing component uses as hardware an Arduino Uno board (ATMega328P microcontroller), which collects data of the voltage and current for each motor, using a potential divider to measure the voltage and an amplifier to obtain the current. The Root Mean Squared (RMS) uses the last 250 samples to calculate the voltage and current RMS value, which allows the computation of the real and apparent power. This data processing is performed at the microcontroller level, transmitting the eight features (4 metrics of 2 servo motors) through the serial port to the Raspberry Pi, which uses a FB, implementing the device template, to read and parse the data. After parsing the data and calculating the respective lag features (moving average and standard deviation) for the last 10 samples, the overall features (total of 24 variables) feed a classification method. The model predicts if the robotic arm is performing with regular efforts (output at zero) or in overload/collision situations (output at one). The classification model training uses an offline dataset, collected from the serial port, containing the robotic arm performing different operations with and without collisions. Several classification algorithms, including Support Vector Machine (SVM), Random Forrest (RF), and Artificial Neural Networks (ANN), were validated using the precision, the recall, and the f1-score as performance metrics. The more accurate model was the RF, exported to perform online on the second pipeline FB, that implements the service template. The model output predictions generate a stop event sent, through multicast sockets, to the second Raspberry Pi, which controls the robotic arm. The communication between the two Raspberry Pis, due to the usage of UDP multicast sockets, adopts a paradigm producersubscriber, where the collision detection component produces stop events when in overload, sent to the robotic arm control component. The robotic arm control component (green FB, using the device template), in a different Raspberry Pi, reads from a text file, the sequence of motor positions to perform, and sends them sequentially through the serial port to the embedded controller. The robotic arm performs, in a cycle, the list of instructions until it receives a stop event from the monitoring component; then, it freezes, waiting to receive a continue event from the user, using the 4DIAC-IDE, to restart the operation.
The synchronized control between a Universal Robots 5 (UR5) robotic arm and a 3D printed gripper requires the usage of a CPPS composed of two Raspberry Pi, similar to the previous scenario. Considering that architecture, one component controls the servo motor that opens and closes the gripper (Figure 3 blue FBs), and the other sends commands to the UR5 robotic arm (Figure 3 purple FBs). The operation performed by both devices consists in 1) catch one object at position A, 2) go with the piece to position B, 3) go back to position A, 4) leave the object in position A, 5) go without the item to position B, 6) go without the object to position A, and restarts the cycle.
Fig. 3. UR5 Robotic Arm and Gripper Workflow.
The UR5 robotic arm uses an API to send commands through a TCP/IP communication channel with the physical controller, which enables the continuous flow of instructions (X, Y, Z positions and Rx, Ry, Rz rotations) to the UR5 robotic arm. The FB, using the service template, wraps the API function to move the robotic arm to a specific location (X, Y, and Z) with the rotation of the joints (Rx, Ry, and Rz). The gripper developed to pick up objects contains three different parts 1) a Raspberry Pi, 2) a servo motor that opens and closes the gripper, and 3) a 3D printed Polylactic Acid (PLA) casing. The movement of the servo motor opens and closes the 3D printed fingers, according to the instruction sent from the Raspberry Pi interface using the General Purpose Input/Output (GPIO). Thus, a FB, adopting the service template, encapsulates the control of GPIO pins using an input variable to establish the operation to perform (open or close). The communication in the CPPS uses multicast sockets to send events between FBs that are in different devices (different colors), as used in the previous scenario.
The simulation of a manufacturing production line addresses several challenges, like the material tracking on the shopfloor or equipment sensorization. Regarding the flow of materials along the production line, the developed simulation includes each station of the process, with its attributes, like the current manufactured material and the respective transporter, the sensors associated, and the operation time. The simulated representation mainly serves as an advantage for the process industries, which have a sequential set of productive steps, being able to simulate different layouts to optimize productivity.
The manufacturing line simulation scenario serves as a basis for performance and workload evaluation for the DINASORE framework. By using a series of three FBs that represent a station it is possible to assess the CPU and memory usage with a varying number of FBs. The htop monitoring tool (linux) was used enabling the profiling of each process in the operating system. Typically, the memory usage has a constant value for the same input parameters; however, the CPU usage has high variability, which requires more samples to obtain an accurate estimation. The higher number of collected samples for the same scenario, generates a more substantial variability, computed in the form of standard deviation. Figure 5 considers both metrics, representing the collected samples for the memory and the average and standard deviation for the processing usage.
Figure 4 presents partially the FB pipeline of the simulation implementation, where each simulated station uses three FBs to model its behavior. The main FB, that adopts the equipment template, represents a station and implements a state machine with the following states: 1) unscheduled, waiting for new material to produce; 2) standby, arrives a material and transporter associated and waits to start the operation; 3) productive, working in the operation of the station; and 4) error, a stochastic state to simulate an extra time producing the material. The additional two FBs, implemented using the service template, allow to simulate the production time and error time (if any). All the series of three FBs represent the entire production line with its characteristic sequential U shape and bifurcations. Additionally, this approach enables the integration of real components with in this simulation environment, turning the pipeline more accurate.
Fig. 5. DINASORE Performance Results.
The main property to evaluate on the DINASORE is how an increasing number of FBs influences the framework usage of computational resources. This kind of test requires a large number of FBs (up to 200) focusing on the DINASORE management of the runtime environment. The hardware specifications of the host machine are 16GB of RAM and an Intel Core i7 processor, with 12 cores and a frequency of 2.20GHz. The analysis of the results indicates a growing trend in both metrics with an increasing number of FBs. That trend follows the natural behavior of computational systems: more complexity causes more resources consumed, with the CPU curve following a rate of approximately 18 FBs per 1% of CPU usage. Those values prove the reliability and scalability of the developed solutions, considering the target hardware, which varies from low computational power devices (e.g., Raspberry Pi) to high performance machines (e.g., servers). Such experiments’ main intention is solely to validate the performance of DINASORE, not considering CPU and memory usage in terms of FB functionality (e.g., training a deep neural network).
Looking in perspective, the DINASORE framework enables the deployment of powerful Python algorithms for CPPSs, following industrial standards globally adopted, like the IEC 61499 and OPC-UA protocol. The proposed framework increases the flexibility of the traditionally closed and hard to re-configure industrial systems, being a step forward the highmix low-volume paradigm. The validation scenarios prove the flexibility and reliability of the DINASORE, giving the necessary freedom to developers for a multitude of different application implementations, like sensor integration, equipment control, data processing, or communication protocols. The performance evaluation demonstrates the scalability in a DINASORE runtime environment with an increasing amount of FBs. Nevertheless, the DINASORE transforms a heavyweight local application into a distributed solution performing in a clusters of devices. Considering that, the platform provides capabilities to scale up the solutions for a larger amount of devices, isolating the applications into FBs, and communication through popular IoT protocols, like MQTT or OPC-UA.
As for the future work, and assuming that DINASORE enables the use of computation-intensive machine learning solutions, the exploration of speculative computation concept, as implemented in [30] for FB-based systems, will be explored. The central idea is to implement this concept as a background process in DINASORE so the execution can be accelerated transparently. An additional objective is the continuous implementation of new methods using the FB structure to easily experiment with them in different industrial applications due to the FB portability between systems.
INDTECH 4.0 - New technologies for intelligent manufacturing. Support on behalf of IS for Technological Research and Development (SI a` Investigac¸a˜o e Desenvolvimento Tecnolo´ gico). POCI-01-0247-FEDER-026653 Symposium on Industrial Electronics (ISIE). IEEE, 2019, pp. 1306– 1311. [26] L. Ferrarini and C. Veber, “Implementation approaches for the execution model of iec 61499 applications,” in 2nd IEEE International Conference on Industrial Informatics, 2004. INDIN’04. 2004. IEEE, 2004, pp. 612– 617. [27] G. Cengic and K. Akesson, “Definition of the execution model used in the fuber iec 61499 runtime environment.[in:] industrial informatics, 2008. indin 2008,” in 6th IEEE International Conference on, 2008, p. 301. [28] C. Sunder, A. Zoitl, J. H. Christensen, V. Vyatkin, R. W. Brennan, A. Valentini, L. Ferrarini, T. Strasser, J. L. Martinez-Lastra, and F. Auinger, “Usability and interoperability of iec 61499 based distributed automation systems,” in 2006 4th IEEE International Conference on Industrial Informatics. IEEE, 2006, pp. 31–37. [29] G. Cengic and K. Akesson, “On formal analysis of iec 61499 applications, part b: Execution semantics,” IEEE Transactions on Industrial Informatics, vol. 6, no. 2, pp. 145–154, 2010. [30] D. Drozdov, V. Dubinin, and V. Vyatkin, “Speculative computation in iec 61499 function blocks execution—modeling and simulation,” in 2016 IEEE 14th International Conference on Industrial Informatics (INDIN). IEEE, 2016, pp. 748–755.