Manufacturing Data Analytics for Manufacturing Quality Assurance Luis Lourenço1, Paulo Figueiras1, Ruben Costa1, Myrsini Ntemi2, Benjamin Mandler3, Athina Tsanousa2, Ilias Gialampoukidis2, Ana Gomez4, Leturiondo Urko4, Stefanos Vrochidis2, Santiago Gálvez-Settier5 and Raul Poler 6 1 UNINOVA, Campus da FCT NOVA, Caparica, 2829-516, Portugal 2 Centre for Research and Technology Hellas (CERTH), Information Technologies Institute (ITI), 6th km Charilaou-Thermi Rd, Thessaloniki, 57001, Greece 3 IBM Haifa Research Laboratory, Haifa, 3498825, Israel 4 Ikerlan Technology Research Center, Basque Research and Technology Alliance (BRTA), Pº J.Mª. Arizmendiarrieta 2, 20500 Arrasate, Spain 5 Instituto Tecnológico de Informática, Av. Juan de la Cierva, 27, Valencia, 46980, Spain 6 Research Centre on Production Management and Engineering (CIGIP), Universitat Politècnica de València, Camino de Vera, s/n, Valencia, 46022, Spain Abstract Nowadays, manufacturing companies are eager to access insights from advanced analytics, without requiring them to have specialized IT workforce or data science advanced skills. Most of current solutions lack of easy-to-use advanced data preparation, production reporting and advanced analytics and prediction. Thanks to the increase in the use of sensors, actuators and instruments, European manufacturing lines collect a huge amount of data during the manufacturing process, which is very valuable for the improvement of quality in manufacturing, but analyzing huge amounts of data on a daily basis, requires heavy statistical and technology training and support, making them not accessible for SMEs. The European i4Q Project, aims at providing an IoT-based Reliable Industrial Data Services (RIDS), a complete suite consisting of 22 i4Q Solutions, able to manage the huge amount of industrial data coming from cheap cost-effective, smart, and small size interconnected factory devices for supporting manufacturing online monitoring and control. This paper will present a set of i4Q services, for data integration and fusion, data analytics and data distribution. Such services, will be responsible for the execution of AI workloads (including at the edge), enabling the dynamic deployment industrial scenarios based on a cloud/edge architecture. Monitoring at various levels is provided in i4Q through scalable tools and the collected data, is used for a variety of activities including resource monitoring and management, workload assignment, smart alerting, predictive failure and model (re)training. Keywords 1 Zero-defect manufacturing, data quality, data analytics, artificial intelligence 1. Introduction The European manufacturing sector key challenge, is to transform cost-based competitive advantages into those that rely on sustainable, high-value-added production. An important lever to address this challenge is to enable companies to achieve superior product quality with highly efficient, Proceedings of the Workshop of I-ESA’22, March 23–24, 2022, Valencia, Spain EMAIL: lcl@uninova.pt (L. Lourenço); paf@uninova.pt (P. Figueiras); rddc@uninova.pt (R. Costa); nmyrsini@iti.gr (M. Ntemi); mandler@il.ibm.com (B. Mandler); atsan@iti.gr (A. Tsanousa); heliasgj@iti.gr (I. Gialampoukidis); ana.gomez@ikerlan.es (A. Gomez); uleturiondo@ikerlan.es (L. Urko); stefanos@iti.gr (S. Vrochidis); sgalvez@iti.es (S. Gálvez-Settier); rpoler@cigip.upv.es (R. Poler) ORCID: 0000-0002-6068-1982 (P. Figueiras); 0000-0002-6142-1840 (R. Costa); 0000-0002-5081-6863 (M. Ntemi); 0000-0001-6599-4446 (A. Tsanousa); 0000-0002-5234-9795 (I. Gialampoukidis); 0000-0001-5720-0183 (A. Gomez); 0000-0003-4913-6438 (L. Urko); 0000- 0002-2505-9178 (S. Vrochidis); 0000-0003-3479-8367 (S. Gálvez-Settier); 0000-0003-4475-6371 (R. Poler) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Wor Pr ks hop oceedi ngs ht I tp: // ceur - SSN1613- ws .or 0073 g CEUR Workshop Proceedings (CEUR-WS.org) smart production processes. This approach creates new challenges, though, because of the many, highly heterogeneous, and intensely interconnected manufacturing resources and their digital counterparts. A successful smart factory needs to manage data-related processes along the entire data life cycle, including data collection, storage, distribution, analysis, use, and deletion, to ensure high data quality at all times. This includes processes related to i) the design, deployment, and use of hardware and software; ii) the planning, implementation, and monitoring of intra-organizational procedures; and iii) the inter-organizational practices in the value chain. The comprehensive quality control of all important factors is an effective measure against unfit, erroneous, unintelligible, or otherwise unreliable data. Since the third Industrial Revolution, which was characterised by the emergence of the digital information age, that manufacturers all over the world are embracing the notion of convergence of the digital and physical worlds [1]. Mainly due to this convergence and to technological advances achieved throughout the last two decades, manufacturing-related data is being generated at exponentially growing rates [2]. Still, there are few manufacturing sectors that truly capitalize on such amount of collected data, by extracting meaningful insights for supporting improvements on their businesses, processes and products [3]. Recently, the application of Data Analytics to manufacturing data has been presented as a solution for the issue of capitalizing on ever-growing manufacturing data [4]. Manufacturing Data Analytics can be defined as the process of finding useful information from analysing manufacturing-generated raw data, whether for decision-making support or for optimisation of business and production processes, among other objectives [5]. [6] present the main objectives for applying Big Data Analytics (BDA) in smart manufacturing. It is envisioned that future BDA applications will be able to assist enterprise managers to learn everything about what they did today and to predict what they will do tomorrow. This future vision is based on a taxonomy of data analytics approaches for manufacturing, which entails four types of analytics processes: descriptive, diagnostic, predictive and prescriptive analytics [2, 7]. Both descriptive and diagnostic analytics methods are reactive while predictive and prescriptive analytics approaches are proactive. Descriptive analytics is an exploratory analysis of historical data to tell what happened. During this stage, most of data mining and statistical methods can be used to reveal the data characteristics, recognise patterns and identify relationships of data objects. Diagnostic analytics is a deeper look at data to attempt to understand the causes of events and behaviours. The diagnostic analysis of machines and other equipment can help to identify the possible faults and predict the failures to reduce the machine down-times. Predictive analytics mainly utilises historical data to anticipate the trends of data (i.e., what will occur in the future). Finally, prescriptive analytics extends the results of descriptive, diagnostic and predictive analytics to make the right decisions in order to achieve predicted outcomes. The prescriptive methods typically include simulation, decision-making, optimisation and reinforcement learning algorithms. Although the three first types of data analytics are not new research trends, the fourth, prescriptive analytics, is seen as a future challenge in Manufacturing Data Analytics [2, 7], and is closely linked to simulation (digital twins) and optimisation. Data analytics is a wide term that encompasses many methodologies and procedures. It is hard to define a specific time of appearance, since there are different processes included in the term, however data analytics is a field quite old, e.g., Pearson’s correlation coefficient has been introduced since 1920s, and widely exploited. Data analytics have already reached a high level of maturity and the constantly increasing need for data driven solutions, may lead to further advancements in tools and algorithms. Regarding the maturity of data analytics in the manufacturing sector, an empirical study of 2017, that researched 100 manufacturing companies, located mainly in Germany and Switzerland, found that in most companies the amount of data that was exploited, was quite smaller that the available amount of data. The problem for this focuses mainly on the processing step of the data and on the information extraction [8]. Considering the data landscape in the manufacturing industry, the i4Q [9] project is focusing on the prescriptive analytics challenge, which entails several smaller challenges, such as close-loop integration between data analytics and simulation processes (in order to bring simulation and digital twin models the closest to reality as possible and to capitalize on the insights gathered from such models) [3] and by leveraging data analytics workloads between edge and cloud computing (so as to implement an hybrid cloud/edge computing scheme, to not only exploit the strength of cloud computing to process the complicated tasks but also harness the benefit of edge computing in short latency, consequently obtaining the better performance) [7]. This paper is organized as follows: section 1, gives an introductory overview on the quality assurance in the manufacturing sector and data analytics as driver; section 2 provides the concept for manufacturing data analytics for manufacturing quality assurance, and also a description of the services pool, finally, section 3 provides the concluding remarks and points out future work, and section 4 gives the acknowledgements to the i4Q consortium. 2. Concept The aim of i4Q project is at turning data into information and actionable insights. Ways to achieve this are to move analysis workloads close to the data sources, thus contributing to the reduction of the Big Data challenge, in addition to helping with security / privacy aspects, while maintaining a low latency response time which may be crucial for example in production facilities, as well as feeding such insights to the end consumer, via smart monitoring and alerting mechanisms or through the integration with digital twins and other simulation models. The objective of the i4Q data analytics core services, is to tackle the analysis of manufacturing data by combining simulation and real data, while employing data fusion techniques to help with the final analysis task. In addition, this set of services will handle AI Model Life-Cycle Management, from a central cloud to an edge-based environment, in order to support the distribution, deployment, and monitoring of AI models. This way, Microservice applications will be able to use manufacturing data processing services, data streams and AI models in an efficient way, scaling up resources in a transparent way. One of the i4Q goals is to design and build a set of management tools for cloud/edge lifecycle of manufacturing related AI models. Summarizing, the main objectives for the data analytics for manufacturing quality assurance services, can be described as follows: • To develop services for data integration and fusion, so that higher level services can process it in an efficient way • To implement data analytics services for categorisation and classifications of manufacturing data and derive actionable insights. • To design and deploy a scalable policy-based Model Distribution from cloud to edge • To develop modules for AI workload placement and deployment on the edge • To develop an efficient monitoring and predictive alerting platform in manufacturing • To develop Manufacturing Digital Simulation Models, such as Digital Twins, enabling virtual validation /visualization and productivity optimization using pre-existing and real time data from different factory levels. Figure 1 depicts a conceptualization of the services for manufacturing data analytics for manufacturing quality assurance. The objective of these set of services, is to achieve the objectives previously identified. The set of services are divided grouped into the following categories: (i) “Data Integration and Transformation Services”, responsible for preparing manufacturing data, so that microservice applications can process it in an efficient way; (ii) “Manufacturing Data Analysis for Quality Qualification Services”, responsible for implementing several incremental algorithms (i.e. operating on data streams with fast incremental updates) suitable for analytic processing of high- speed data streams; (iii) “AI Workload and Model Distribution and Deployment from Cloud to Edge Services”, addressing the management of AI-based workloads in a hybrid cloud edge manufacturing environment; (iv) “Smart Manufacturing Monitoring and Alerting Services”, addressing the development of scalable monitoring tools designed for the manufacturing edge environment; and (v) “Manufacturing Digital Simulation Models Services”, aiming to control the full traceability and enabling partners to streamline and digitalize the entire production process. A more detailed overview of each service categories, is provided in the following sub-sections. Figure 1: i4Q Manufacturing Data Analytics for Manufacturing Quality Assurance 2.1. Data integration and transformation services The Data Integration and Transformation solution (i4QDIT) will be responsible for reading and preparing manufacturing data for further analysis. The solution will be server-based and will contain micro-services that will import, prepare and export the data to other i4Q solutions. In more detail, i4QDIT will be able to read manufacturing data, mostly coming from sensors, whether they are stored in a database or they are real-time streaming. The data will go through some cleaning and filtering functions and afterwards they will be pre-processed according to the needs of other solutions. The pre-processing stage includes the application of some suitable transformation services, like Fast Fourier Transformation (FFT) and Wavelet Packet Decomposition, and feature extraction, which is quite important in sensor data analysis. Another function of this solution will be the harmonization of heterogeneous data, in terms of timestamps’ visualizations or in data format. Special attention will be given on the function of fusion that will combine data from heterogeneous sensors using sophisticated methods. Fusion intends to maximize the usage of the obtained informatio1n from the sensors and therefore improve the performance of the prediction algorithms that will be later applied on these data. The current solution will interact with and be useful to all the solutions related to data analysis. It will be implemented in the early stages of the project’s architecture and will provide datasets ready for further analysis. The micro-services comprising the solution will be written in Python language and i4QDIT solution will be implemented as a function in the Big data analytics Suite. 2.2. Manufacturing data analysis for quality qualification Data analytics for quality qualification, aims to tackle the analysis of manufacturing data by combining simulation and real data, while employing data fusion techniques to help with the final analysis task. In addition, it will handle AI Model Life-Cycle Management, from a central cloud to an edge-based environment, in order to support the distribution, deployment, and monitoring of AI models. This way, Microservice applications will be able to use manufacturing data processing services, data streams and AI models in an efficient way, scaling up resources in a transparent way. One of the goals, is to design and build a set of management tools for cloud/edge lifecycle of manufacturing related AI models. Currently the manufacturing environment is heterogeneous and dynamic, including nodes with different characteristics. These services will implement specialized analytic functions on top of the data infrastructure from “Data Integration and Transformation”. In particular, the goal is to implement several incremental algorithms (i.e. operating on data streams with fast incremental updates) suitable for analytic processing of high-speed data streams. The core functions to be implemented are related to clustering, regression, classification, anomaly detection, temporal correlation – the corresponding incremental versions of these algorithms are known from the literature but are typically not available as a part of database systems. The services are decomposed into the following categories as described below: Data analytics services (i4QDA) is defined as a system whose main functions are the provision of Data Analytics services, supported by the integration of several state-of-the-art tools, methods, and libraries, ranging from Big Data Processing and Analytics to Machine Learning, Data Mining and Deep Learning. The services will be provisioned through two main channels: i) Open Application Programming Interfaces (APIs) like RESTFul-based, pub-sub, socket-based for the collection of the necessary data to execute the selected services and for the provision of results coming from the Data Analytics services, or ii) Deployment bundles with the necessary tools, methods, and libraries to deploy and run the selected services on premises or on cloud environments. Big Data Analytics (i4QBDA) is defined as a system whose main function is to deliver on- demand deployment bundles that are easily configurable, deployable, and executed. This Suite will be able to provide custom-built deployment bundles that can contain all the necessary tools, methods, libraries, and code to deploy and run the selected Data Analytics tasks in a panoply of environments, from centralized, distributed on-premises or Cloud. This solution will be supported by containerization technologies such as Kubernetes or Docker. Analytics Dashboard (i4QAD) is defined as a system whose main function is to provide visual analytics tools and methods to the i4Q project. The i4Q Analytics Dashboard can be used via a Web Application or through the provision of a deployment bundle that can be deployed on premises or on the cloud, and will be based on state-of-the-art visual analytics tools, such as Apache Superset, Grafana or Jupyter Notebooks. 2.3. AI workload and model distribution and deployment from cloud to edge A hierarchical Cloud/edge architecture provides efficient and flexible management of edge workloads, to gain from the advantages of the edge environment. Edge technology enables several desirable characteristics for a smarter factory scenario, such as low latency, enhanced security and privacy, reduced network bandwidth utilization, and a fist line of defense in a big data architecture (data reduction, by keeping data local and not necessarily transmitting all data to the cloud). The edge service provides such capabilities by distributing AI workloads and models to edge nodes, enabling processing at the edge, taking advantage of the associated low latency for enabling faster reaction to data in real-time. Two complementary services are envisioned to tackle the lifecycle of AI workloads and the underlying models they use, to run adequately in an edge environment Knowledgeable deployment and execution of AI workloads on an edge-computing environment is required to efficiently operate on the edge. Proper placement and deployment services are required to take advantage of the edge environment. This component shall enable workloads to execute efficiently on the edge, including placement and deployment services. The edge service provides interfaces and capabilities to run different AI workloads on different devices; primarily edge devices. For AI at the edge automated workload management is essential to address scale and dynamic heterogeneity of workloads. Placement, deployment, execution and monitoring cycles shall optimize AI workloads operation on the edge. Target deployment environments may be very heterogeneous and dynamic; thus, deployment needs to take a variety of criteria into consideration. Re-deployment of the entire workload or the adaptation of the underlying model may be required while the workload is running. A second service provides infrastructure for managing AI-based models in a hybrid cloud edge- manufacturing environment, addressing scalability requirements. A policy-based distribution mechanism is envisioned to ease the distribution task by enabling the specification of rules for eligible targets in a simplified manner. The edge environment presents several challenges for model lifecycle management, such as scale, constrained resources, and disconnected mode. Both services shall be Integrated and coordinated to ensure workloads and their AI models meet at the edge. 2.4. Smart manufacturing monitoring and alerting The infrastructure monitoring solution (i4QIM) incorporates an ensemble of monitoring tools and predictive failure alerting mechanisms, aiming to provide efficient alerts when a problem is detected. i4QIM monitors manufacturing lines and processes by tracking and analyzing industrial sensor signals. It also monitors the real-time analysis results derived by other i4Q solutions to provide feedback about machines’ optimal functionality. i4QIM rely on state-of-the-art ML algorithms to detect or predict imminent problems in manufacturing lines. Specifically, light gradient boosting machine is applied for classification tasks, while recurrent neural networks are applied for regression tasks [10, 11]. In every case, specific thresholds determine the occurrence a problem and if certain conditions hold an alert is triggered on-the-fly. A principal advantage of i4QIM is its ability to suggest in time a possible solution to an underlying problem. This purpose is served by exploiting advanced statistical techniques, as well. Specifically, Pearson correlation and Granger causality analysis are applied on sensor signals to infer which parameters influence the most the dependent variable. This crucial latent information is then fed into ML algorithms to estimate a forthcoming failure of a machine. An example could be that of tool wear detection in CNC machining industries, where cutting parameters are analyzed, the most influencing ones are detected, and i4QIM estimates the presence of tool wear. If tool wear has begun, an alert is triggered. i4QIM analysis results are provided to i4QLRT solution to reconfigure optimally machine parameters, and to take corrective actions if a problem is detected. This real-time intelligent mechanism reveals latent failure patterns, which are usually not detectable by humans. Consequently, machine malfunctions, permanent damages, and frequent manufacturing operations interruptions (i.e., machine stops) can be efficiently prevented. That is, the production rate is increased, while cost is reduced. 2.5. Manufacturing digital simulation models The Digital Twin solution i4QDT allows industrial companies to achieve a connected 3D production simulation, with a digital twin for manufacturing enabling virtual validation/visualization and productivity optimization using pre-existing and/or simulated data and data from different factory levels (small cell to entire factory). Thus, the user of i4QDT is able to launch simulations of the manufacturing asset/plant based on, on one hand production/machine data and on the other hand, the digital twin obtaining results that are visualized in a 3D environment and other data visualization formats such as graphs or tables. In this sense i4QDT provides a virtual representation and contextualization of all the assets present in a manufacturing line. The virtual representations of the different physical devices are accessible through APIs, creating a framework of consistent interoperability that allows the building of the digital twins in a more focused digital twin domain, and reducing the complexity of IoT deployments. For that purpose, data-driven approaches are used exploiting the capabilities of machine learning algorithms [12]. Additionally, when virtual sensors are to be obtained, physics-based models are developed and included in the Digital Twin. An industrial model exchange standard is used for facilitating the integration of models with monitoring algorithms [13], protecting the model intellectual property, and making the framework independent from the modelling source. Each Digital Twin developed allows calls to perform simulations that can be managed through REST APIs. Thanks to these simulations, the machine owner or manufacturer, or a system operator could test several different scenarios and decisions on optimum parameters assessed in advance, especially when used in conjunction with other solutions that help the exploitation of the results, with different purposes like optimize operations or predict failures [14]. 3. Conclusions This paper provides an overview about how i4Q project is addressing data analytics for manufacturing quality assurance. More specifically, i4Q Project aims to provide a solution consisting of IoT-based Reliable Industrial Data Services (RIDS), which will be a complete package consisting of 22 i4Q Solutions. These solutions will be able to manage the huge amount of industrial data coming from cheap cost-effective, smart, and small size interconnected factory devices for supporting manufacturing online monitoring and control. The i4Q concept addressing data analytics for manufacturing quality assurance, is grouped into five capabilities around data management: “Data Integration and Transformation”, “Manufacturing Data Analysis for Quality Qualification”, “AI Workload and Model Distribution and Deployment from Cloud to Edge”, “Smart Manufacturing Monitoring and Alerting”, and “Manufacturing Digital Simulation Models”. i4Q concept will be validated in pre-defined use cases. Six Pilots are envisioned for the validation of i4Q services, namely; Pilot 1: Smart Quality in CNC Machining, Pilot 2: Diagnostics and IoT Services, Pilot 3: White Goods Product Quality, Pilot 4: Aeronautics and Aerospace Metal Parts Quality, Pilot 5: Advanced In-line Inspection for incoming Prime Matter Quality Control, Pilot 6: Automatic Advanced Inspection of Automotive Plastic Parts. It is clear that the selected pilots represent different industrial sectors and activities, such as white goods, wood equipment, metal machining, ceramics pressing, plastic injection, metal equipment. Future work should be focused upon ways to improve the proposed concept by considering a wider range of applications/services of data analytics in product lifecycle for sustainable production and cyber-physical systems. In addition, other key technologies related to smart manufacturing should also be investigated. With the support of cloud technology and IoT, cloud manufacturing can transform various manufacturing resources into services so that end users can request services on demand in a convenient pay-as-you-go manner. Moreover, cyber-physical systems integration into cloud manufacturing enables remote monitoring and execution of manufacturing operations. Thus, physical machinery and virtualized services are implemented to support manufacturing activities and decision-making. The networked manufacturing services allow for smart decision making through a collaborative and intelligent full sharing and circulation of manufacturing capabilities and services. 4. Acknowledgements The authors acknowledge the European Commission for the support and funding under the scope of Horizon2020 i4Q Innovation Project (Agreement Number 958205) and the remaining partners of the i4Q Project Consortium. 5. References [1] F. Tao, Q. Qi, A. Liu, A. Kusiak, Data-driven smart manufacturing, Journal of Manufacturing Systems 48 (2018) 157-169. doi: 10.1016/j.jmsy.2018.01.006. [2] G. Shao, S.-J. Shin, S. Jain, Data analytics using simulation for smart manufacturing, in: Proceedings of the Winter Simulation Conference, IEEE, New York, 2014, pp. 2192-2203. doi: 10.1109/WSC.2014.7020063. [3] S. Jain, G. Shao, S.-J. Shin, Manufacturing data analytics using a virtual factory representation, International Journal of Production Research 55 (2017) 5450-5464. doi: 10.1080/00207543.2017.1321799. [4] P. Lade, R. Ghosh, S. Srinivasan, Manufacturing Analytics and Industrial Internet of Things, IEEE Intelligent Systems 31 (2017) 74-79. doi: 10.1109/MIS.2017.49. [5] S. H. Bang, R. Ak, A. Narayanan, Y. T. Lee, H. Cho, A survey on knowledge transfer for manufacturing data analytics, Computers in Industry 104 (2019) 116-130. doi: 10.1016/j.compind.2018.07.001. [6] S. Ren, Y. Zhang, Y. Liu, T. Sakao, D. Huisingh, C. Almeida, A comprehensive review of big data analytics throughout product lifecycle to support sustainable smart manufacturing: A framework, challenges and future research directions, Journal of Cleaner Production 210 (2019) 1343-1365. doi: 10.1016/j.jclepro.2018.11.025. [7] H.-N. Dai, H. Wang, G. Xu, J. Wan, M. Imran, Big data analytics for manufacturing internet of things: opportunities, challenges and enabling technologies, Enterprise Information Systems 14 (2020) 1279-1303. doi: 10.1080/17517575.2019.1633689. [8] S. Groggert, M. Wenking, R. Schmitt, T. Friedli, Status quo and future potential of manufacturing data analytics—an empirical study, in: IEEE International Conference on Industrial Engineering and Engineering Management, IEEE, New York, 2017, pp. 779-783. doi: 10.1109/IEEM.2017.8289997. [9] i4Q, Industrial Data Services for Quality Control in Smart Manufacturing, 2021. URL: https://www.i4q-project.eu/ [10] N. Aziz, E. Akhir, I. Aziz, J. Jaafar, M. Hasan, A. Abas, A Study on Gradient Boosting Algorithms for Development of AI Monitoring and Prediction Systems, in: International Conference on Computational Intelligence, IEEE, New York, 2020, pp. 11-16. doi: 10.1109/ICCI51257.2020.9247843. [11] J. Lee, Y. Lee, J. Kim, Fault detection based on one-class deep learning for manufacturing applications limited to an imbalanced database, Journal of Manufacturing Systems 57 (2020) 357-366. doi: 10.1016/j.jmsy.2020.10.013. [12] J. Friederich, D. Francis, S. Lazarova-Molnar, N. Mohamed, A framework for data-driven digital twins of smart manufacturing systems, Computers in Industry 136 (2022) 103586. doi: 10.1016/j.compind.2021.103586. [13] S. Sinisi, V. Alimguzhin, T. Mancini, E. Tronci, Reconciling interoperability with efficient Verification and Validation within open source simulation environments, Simulation Modelling Practice and Theory 109 (2021) 102277. doi: 10.1016/j.simpat.2021.102277. [14] T. Lin, G. Shi, C. Yang, Y. Zhang, J. Wang, Z. Jia, L. Guo, Y. Xiao, Z. Wei, S. Lan, Efficient container virtualization-based digital twin simulation of smart industrial systems, Journal of Cleaner Production 281 (2021) 124443. doi: 10.1016/j.jclepro.2020.124443.