The SmarTwin project aims to investigate an innovative service model for cost optimization, risk reduction, micro-traceability of a product's processing steps, certification of the ecological footprint and, finally, financial support for complex supply chains [2], with a focus on production chains dealing with perishable products (i.e. vegetables).
The project has several innovative aspects. The first is represented by the approach to the problem, which is developed on three parallel and integrated logical levels: one methodological, one technological Ital-IA 2024: 4th National Conference on Artificial Intelligence, organized by CINI, May 29-30, 2024, Naples, Italy and one financial. The methodological aspect is aimed at providing normative and operational support to supply chain actors for the implementation of the new hypothesized service model.
The technological aspect is aimed at automatically detecting events, typical activities, quality levels (of products) and operating conditions (of the various contexts in which activities take place) through the use of IoT technologies, Artificial Intelligence (machine learning [3] and computer vision [4]) and predictive analytics systems (e.g., with artificial neural networks and big data analytics systems). This information properly collected and analyzed will 0000-0001-8904-180X (M. Di Capua); 0000-0002-6589-9323 (E. Di Nardo); 0000-0001-5592-7995 (A. Ciaramella); 0009-0003-5962-8302 (G. Iannuzzo); enable the feeding of a much more extensive product certification and tracking system than those currently used. The financial aspect, on the other hand, represents an absolute novelty for this type of solution and will allow for versatile and secure management of economic relations between the various supply chain partners. In particular, within the system, in addition to supporting with specific functions the process of evaluation and analysis of each step of the economic stage, a virtual currency will be introduced (to be connected to the Digital Twin of the supply chain [1] model) to manage payments, anticipate liquidity within the circuit and to securitize credit. Overall, the presence of the financial "dimension" in the governance of supply chain processes allows us to enhance the new level of "trust" that we aim to achieve with the application of the proposed service model and that not only involves the economic operators in the supply chain but also extends to end consumers.
The proposed service model is characterized by its innovative features and, in particular, by its focus on defining a digital twin of the entire supply chain ecosystem to achieve a holistic view and a new level of detail and pervasiveness of process monitoring, tracking, quality control and certification activities to address and resolve the typical complexity of contexts such as productive supply chains.
The digital model aims to represent the various organizational, economic and regulatory dynamics of a modern supply chain, analyzing the huge amount of data collected by the network of integrated sensors and devices (e.g. cameras), in order to suggest optimization actions aimed at cost reduction (e.g. avoidance of waste), prevention of risks to people (e.g. timely control of the correct arrangement of goods in the warehouse), constant monitoring of product quality (e.g. verification of consistency of storage and transport conditions with what is required by the specific specification), and reduction of environmental impact (e.g. minimization of energy consumption). These objectives are essentially pursued by realizing a technological platform that combines the dynamic functionalities required to manage the configuration and constant evolution of the proposed service model and those of descriptive and predictive analysis on the Digital Twin of the supply chain ecosystem (see figure 1), with a series of specific technological components, based on AI, for the recognition of events, activities occurring in the operational context, potential critical scenarios and for the constant verification of consistency between what is detected in the real process with what is provided for at the contractual and regulatory level.
As already mentioned, the main idea of the project is to approach the problem of managing a complex supply chain ecosystem in a holistic way, constantly monitoring every aspect, every phase of work, every relationship between partners and every event that may be useful to ensure the quality specifications expected by the customer and to maximize the benefits for the operators in terms of optimizing costs, time and the objectives set. Obviously, the model must be adaptable to any supply chain context and be able to follow the natural evolutions that occur in these contexts to manage seasonality, workload discontinuities and any contingencies that may arise. It will therefore be necessary to define all the information relating to the actors, the processes, the quality objectives to be guaranteed (including sustainability objectives, specific production specifications, etc.) and, of course, the commercial relationships established between the various actors.
Furthermore, for each "measurable" element to be detected in the process, it is assumed that an IoT sensor network or different types of cameras (e.g. thermal), whose data streams are constantly analyzed by the AI components of the system, can be plugged in and used. All the configuration information required to build the model should be defined, captured and managed by the software platform in such a way that its acquisition is easy and immediate and does not create functional access or operational cost barriers to adoption. Therefore, it is necessary to provide functions for defining the configuration logic and the rules behavior of the system that are, as far as possible, driven by automatic processes and are dynamically adaptable and reusable in other phases of the supply chain. On the other hand, as far as the IoT sensor network and image stream acquisition sources are concerned [6], the system must be able to census the devices, identify their location and usefulness for the purpose of assessing the operating conditions and the associated rules for normalizing, aggregating and analyzing the collected data.
The goal of the system in this case is to be able to verify continuous compliance with the optimal operating conditions and, at the same time, to certify the consistency of what is detected in the real environment with what is stipulated in the agreements and work processes defined in the service model.
Finally, the monitoring of all events in the supply chain (micro-traceability) and the certification of processes and activities carried out, together with the digitalization of contractual agreements through smart contracts, will also enable the designed platform to automatically settle economic aspects between parties, both by generating payment flows and by offering operators the possibility of securitizing their acquired (and certified) receivables for use within the system as an alternative form of liquidity.
The software platform, that implements the service model, represents the basic technological infrastructure into which are integrated both the dynamic system configuration functionalities (necessary to digitize the model), the services offered by the specific components (such as the IoT platform, Blockchain and AI components), and the operational functionalities and those that make the Digital Twin of the overall supply chain ecosystem usable to various types of users. The different components of the platform will be detailed below and are shown in figure 2.
It presides over all activities of continuous acquisition of data from the IoT sensor network that is used to detect the status of activities and operational contexts both referring to working environments (e.g., warehouses for the storage of goods) and to the mobile network of multimodal transportation (e.g., sensors on board refrigerated trucks for the transport of perishable goods).
These data, appropriately normalized and aggregated, can already represent an element of interest for the purposes of tracking and monitoring the operating conditions of the supply chain ecosystem, but at the same time they also constitute a knowledge base that can be used for the creation of predictive models suitable for recognizing characteristic behaviors or events, that can be associated with the statements of supply chain operators or for evaluating elements of optimization and risk prevention. From this point of view the IoT platform, integrated into the SmarTwin system, provides for collecting data, aggregating them on a time basis, doing systematic checking of predefined rules and triggering any alerts, making them searchable by the application functions that implement the supply chain Digital Twin and making them available to the various AI-based analytics tools and optimization components working in the background.
This subsystem is in charge of the camera network census activities and the management of video streams, from the various devices installed in the supply chain environment (in places where typical activities take place such as warehouses, loading and unloading pallets, etc.), and for each of them it is concerned with submitting the streams to the analysis and image processing components to identify events, typical activities or situations of potential interest.
The final components (see figure 3) that will be in charge of image stream analysis (thus based on machine learning and computer vision), will be designed as Artificial Neural Networks (ANN) with different tasks, each of which is concerned with detecting particular types of events (e.g. the presence of people in the area) or capturing particular information (e.g. the license plate of a vehicle that is in a loading/unloading area). Obviously due to the specificity of the technology used, the context and the purpose of the analysis, it is not always possible to successfully use pre-trained or fully reusable components in different similar contexts. We will therefore proceed on the one hand with the identification of a set of typical events or actions that can be generalized and, on the other hand, with the identification of technological solutions that can allow the integration into the system of specific components trained ad hoc for a particular task. For example, it may be necessary to monitor the state of preservation of a particular type of product in a warehouse, and to achieve this type of monitoring it may be necessary to employ two cameras of different technology (e.g. a thermal camera and an RGB camera) and to train a specific neural network to process these video streams and detect the state of preservation at a specific frequency. Obviously, this component will have to be purpose-built and can only be used in the same environments, conditions and for the same type of product for which it was originally designed. In the context of the final prototype, we will try to imagine some case scenarios where such conditions might occur, to try to integrate purpose-built AI components into the system for a specific function.
As a whole, the subsystem that deals with the acquisition of image streams collects images at a set frequency from the various devices and submits them to the network of specialized AI components for recognition of typical events and activities, it collects all outputs towards the Digital Twin, in order to make data available (API) to the monitoring and alerting component of the service platform.
In order to integrate predictive task capabilities into the project, recent machine learning techniques have been explored, with a focus on the use of deep artificial neural networks. The main objective here is to predict potential problems and recurring data patterns, to improve supply chain performance, from the data collected by the IoT platform envisaged by the project. From a design perspective, it's necessary to develop versatile and reusable machine learning modules that can be "plugged" into the platform to provide information about potential problems that may arise along the supply chain being analyzed. The models will learn the normal dynamics of information flows that characterize one or more processes and will be able to recognize "anomalous" patterns of data to prevent risks and consequent process-related damage or waste, with a view to making the supply chain more effective and sustainable. The theoretical approach adopted for this purpose is the time-series analysis, which focuses on understanding data that vary over time. It is particularly relevant in IoT, as much data produced by connected devices is organized into time series, such as sensor data and measurements at regular intervals.
In the more recent landscape, transformer models, which were initially introduced for natural language processing problems, are also showing good results in this area. Transformers can handle data sequences of varying lengths and can be used for time series prediction and analysis tasks. Finally, it is useful to mention also autoencoders, which are deep learning models used for dimensionality reduction and feature representation of data, and which can be used in time series analysis to extract relevant features and reduce dimensionality of data, which is a potential issue to be addressed in the application scenarios of the project. Our analysis is confirming that these deep learning techniques can be used for various time series analysis tasks related to a supply chain, such as short or long-term forecasting, anomaly detection, trend and pattern analysis. Relative to the transformer models identified in the first part of the state-of-the-art analysis of the project, some preliminary tests were conducted on literature datasets to verify the effective ability of these models to also be used in the context of time series forecasting. Preliminary results confirmed that transformer models can be used with good effectiveness also with these types of data.
In the specific area of IoT sensor data analysis, an autoencoder-based neural network was tested to identify vibration anomalies from sensor readings installed on a series of bearings. The aim of the preliminary test was to be able to predict future bearing failures before they occur.
In the next stages of the project, possible automated approaches will be analyzed, to support quality control processes within warehouses and the quality verification of perishable products, through the adoption of computer vision techniques. The analysis referred to should be non-invasive, i.e. based on the placement of fixed cameras in places where operators normally carry out visual inspections. Digital images from cameras and videos can be used to train computer vision models dedicated to the inspection and analysis of products/goods. The computational vision tasks that will be implemented next will cover both general and specific tasks in the area of supply chain process control [5]. For example, machine learning models will be trained to detect specific machinery (e.g. forklifts) moving in logistics aisles, rather than the presence of people in storage areas. It will also be possible to define and integrate more specific and advanced neural network models that instead perform Human Activity Recognition (HAR) tasks, i.e. the recognition of specific human activities (e.g. picking) according to specific logistics use cases.
Within the designed platform of the project there are several components that adopt Artificial Intelligence (AI) or Machine Learning (ML) techniques to perform tasks traditionally reserved for the human operator and in particular: for the automatic classification and analysis of information acquired from the supply chain activities, the performance of automatic checks, to make predictions about operating conditions and finally to suggest possible optimizations. In this case, the technological issues to be addressed are essentially: the recognition of typical events and activities (HAR -Human Activity Recognition) using image processing techniques and methodologies based on deep architectures, the creation of predictive models from the large amount of data collected from the IoT sensor network (using machine learning techniques), and also the implementation of a decision support system (DSS), which is necessary to be able to apply the expected automatic control rules.
In this regard, to mitigate the distrust that often arises in adopting AI components as mere functional black boxes, whose actual behavior in terms of decisions made is not always clear and justifiable in detail [7], development approaches, based on the emerging branch of eXplainable, AI will be adopted. The set of tools and techniques used in XAI aims to According to recent research made by Forrester, XAI represents a phenomenon capable of generating numerous tangible benefits for those who regularly adopt it within their process management:
• a reduction in model monitoring efforts ranging from 35 percent to 50 percent; • an increase of up to eight times the number of models in production; • an overall accuracy of the model itself that can be estimated at 15 percent to 30 percent; • an increase in the profit range that can even triple.
Multimodal data, which characterize control functions, will be treated with innovative computer vision techniques, like image classification, visual object tracking, and reliable and interpretable AI techniques (ante-hoc and post-hoc methods). In addition, methodologies to improve resilience will be used (see figure 4). For the design of decision support models adopted for risk assessment [8] from information flows that characterize one or more supply chain processes, XAI methodologies will be used to improve multimodal data aggregation and decision logic (e.g., Fuzzy Logic). Data mining methodologies will be explored for preprocessing multimodal data (e.g., intrinsic dimension) along with signal processing and computer vision techniques. Data for learning and testing the designed models will be acquired from information flows, images and videos obtained during the operational phases of the supply chain. In the early design stages of the SmarTwin system, synthetically created realistic data will be used and data augmentation, concept drift and Active Learning methodologies will be adopted to make the models more robust. Among the possible methodologies adopted for explainability purposes, Neuro-Symbolic (NeSy) AI has been explored in this early design stage of the project.
The goal is to create a system that can learn from large amounts of data, much like neural networks, but also has the high explainability and provable correctness of symbolic systems. One of the key differences between symbolic and neural systems is how they represent knowledge. Symbolic systems use explicit representations that humans can understand. They structure knowledge in a logical way, often using logic-based languages. Neural systems, on the other hand, use distributed representations that are hard to interpret. They learn representations from data in an implicit way, which makes them highly adaptable but also opaque. It's seen as a promising approach to overcoming the limitations of purely symbolic or purely neural methods and to advance towards more intelligent and human-like AI systems. However, NeSy AI also faces several challenges. These include the difficulty of integrating symbolic and neural methods, the need for large amounts of data to train neural networks, and the lack of interpretability of neural representations.
This subsystem deals with the recording on Blockchain of any information, event, transaction or activity that needs to be tracked or is involved in the certification processes under the service model. This will make it possible to make such records "public," transparent and unchangeable. Of course, some of the information recorded on the Blockchain, although public, may also not be made visible to all users (in which case it would be encrypted) to implement different access policies and differentiate levels of visibility based on products and supply chain rules. Each record will include, where possible, elements to certify the information entered also by including possible references to further records made in other related Blockchains. Regarding the latter aspect, considering the needs and operations [9] that will be envisaged on the Blockchain, we will evaluate in the next future, as part of the specific research activities that will be carried out, which implementations and how many Blockchains to use and how to link them together. Recall, that in addition to the needs for tracking and certification of events and activities, we plan to manage by means of Smart Contract all contracting between supply chain actors and all economic transactions. A further use that will be made of Blockchain and Smart Contract will be that necessary for the virtualization of accrued credit and for the eventual securitization and transfer of the same to a secondary market.
This paper presents the SmarTwin project, which aims to integrate the latest machine learning and blockchain technologies into production supply chains in order to improve the quality of their component processes, as well as their associated economic and environmental impacts. The project is still at an early stage and will be completed in 2026, but the preliminary results of the research activities outlined here confirm the validity of the initial idea.
The project was funded by the Italian Ministry of Economic Development (MISE): it started in 2023 and it will end in 2026. Project reference no. F/310218/01-02-03-04-05/X56.