AI in Industry: Activities of the CINI-AIIS Lab at University of Naples Federico II Alessandro Del Prete1 , Sofia Dutto1 , Antonino Ferraro1 , Antonio Galli1,∗ , Vincenzo Moscato1 , Gabriele Piantadosi2 , Carlo Sansone1 and Giancarlo Sperlì1 1 University of Naples Federico II, via Claudio 21, Naples, 80125, Italy 2 ENEA, Centro Ricerche Portici, 80055 Portici (NA), Italy Abstract Artificial intelligence (AI) is reshaping the manufacturing landscape, offering opportunities for efficiency improvements and innovation. Through Machine Learning (ML) and Deep Learning (DL), AI enables predictive maintenance, anomaly detection, and image analysis in industrial settings. ML algorithms empower systems to learn from data, facilitating predictive maintenance by predicting optimal equipment servicing schedules based on operational conditions. DL techniques, including Convolutional Neural Networks (CNNs), are revolutionizing industrial image analysis by extracting intricate features for quality control and defect detection. Moreover, the integration of DL with natural language processing (NLP) streamlines tasks like document analysis and inventory management. At the University of Naples Federico II’s CINI-AIIS Lab, cutting-edge AI projects are underway, showcasing the transformative potential of AI in the industry sector. Keywords Predictive Maintenance, Energy Forecasting, Anomaly Detection, Remaining Useful Life. 1. Introduction the CINI-AIIS Lab, highlighting their innovative contri- butions. Artificial intelligence (AI) is transforming various indus- tries, including the manufacturing sector, by emulating human intelligence to tackle complex challenges. In the 2. Prediction and Forecasting for industrial domain, AI holds significant promise, enhanc- railway rolling stock equipment ing operational efficiency, optimizing processes, and driv- ing innovation. AI-powered systems can analyze exten- The manufacturing industry is currently undergoing the sive datasets to detect anomalies, predict equipment fail- so-called Industry 4.0 revolution, characterized by the ex- ures, and improve overall productivity. Machine Learn- tensive integration of physical and digital realms within ing (ML), a subset of AI, enables systems to learn from production settings. Key technologies driving this revo- data and make informed decisions, such as predicting lution include the Industrial Internet of Things, Big Data, optimal maintenance schedules based on equipment con- Artificial Intelligence, and advanced telecommunications ditions and operational context. like 4G and 5G, which profoundly influence the trans- Deep Learning (DL), another ML subset, leverages Ar- port sector. These innovations enable the gathering of tificial Neural Networks (ANNs) to process complex data vast data from diverse onboard devices and equipment patterns and make accurate predictions. DL, particu- installed on train vehicles and along railway tracks. larly through Convolutional Neural Networks (CNNs), is This wealth of data, acquired through smart sensors revolutionizing industrial image analysis by extracting and relayed to diagnostic systems either onboard or in meaningful features from images for quality control and control rooms, holds immense potential. By employ- defect detection. Additionally, DL, combined with Nat- ing appropriate techniques, it can unveil patterns of ural Language Processing (NLP), is streamlining tasks degradation in components and anticipate failures in like document analysis and inventory management. The a timely manner, facilitating optimal maintenance de- versatility of AI underscores its pivotal role in the manu- cisions. Traditionally, players in the railway transport facturing industry, driving efficiency gains and fostering sector have relied on planned maintenance, often result- innovation. ing in unnecessary actions and inflated operating costs. In this paper, we showcase AI projects in the industrial However, the evolution towards Condition-Based Main- sector from the University of Naples Federico II node of tenance (CBM) offers a proactive alternative. CBM, an extension of planned maintenance, assesses equipment Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- conditions through direct measurements, enabling timely nized by CINI, May 29-30, 2024, Naples, Italy ∗ Corresponding author. repairs or replacements when specific conditions are met. Envelope-Open antonio.galli@unina.it (A. Galli) Predictive Maintenance takes CBM a step further by © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). leveraging monitoring data and effective predictive tech- CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings Figure 1: The proposed failure prediction methodology. niques to anticipate fault occurrences. This approach Mann-Kendall) and correlation, to uncover relationships enables companies to schedule maintenance operations between dataset features and failure patterns. Subse- precisely when needed, leading to cost reductions, de- quently, LSTM networks are utilized to both predict and creased mean time to failures, and overall profit enhance- forecast failures, offering valuable insights to mainte- ment. The key advantage lies in conducting maintenance nance personnel responsible for rolling stock equipment. preemptively, averting prolonged downtime without re- Our methodology achieves an accuracy exceeding 99% sorting to premature interventions, thus minimizing the for both prediction and forecasting tasks, surpassing ex- unavailability of rolling stock and infrastructure equip- isting ML models and techniques in the predictive main- ment. tenance literature. Moreover, the error rates for pre- Machine Learning (ML) algorithms emerge as power- diction and forecasting tasks are notably low, with a ful tools in maintenance across diverse domains, owing false alarm rate of approximately 0.4% and a mean abso- to their support for predictive techniques. ML techniques lute error on the order of 10−4 . These results are highly have been applied extensively, from predicting light bulb promising compared to previous studies. Additionally, failures to early detection of machine failures, rotating the methodology is validated against real-world systems machinery failure prediction, and estimating the Remain- by our industry partner, confirming the soundness of our ing Useful Life (RUL) of various assets like hard disks assumptions and approaches. and wind turbines. In the railway domain, ML is gain- ing prominence in enhancing operations and reliability. However, the efficacy of ML algorithms hinges on select- 3. Forecast and Anomaly Detection ing the appropriate technique, especially considering the in Photovoltaic System gradual deterioration or sudden failure characteristic of rail systems. Thus, ML approaches for predictive main- In the context of the global energy landscape, the Interna- tenance must account for such data dynamics to ensure tional Energy Agency (IEA) highlights the pivotal role of accurate failure prediction and forecasting. photovoltaic (PV) energy in driving the ongoing energy We introduced a deep learning-based methodology transition, as indicated in the World Energy Outlook 2023 which enables the prediction and forecasting of failures, [1]. Despite the adoption of the Stated Policies Scenario allowing for proactive maintenance interventions to be (STEPS), it is observed that the utilization rate of solar planned before they occur, thereby optimizing both the energy markets lags behind the expanding production cost and duration of maintenance activities. Utilizing capacity of PV technologies (Figure 2). Long Short-Term Memory (LSTM) networks, an exten- Recognizing the imperative for a paradigm shift within sion of recurrent neural networks (RNN), the methodol- the PV market, encompassing both domestic and indus- ogy is adept at learning long-term dependencies in data trial installations, there is a pressing need for active in- that change gradually over time. An overview of the volvement from network and infrastructure stakehold- methodology is provided in Figure 1. ers, energy producers, and consumers [2]. These entities, The proposed approach analyzes data collected from frequently coalescing into energy communities, aim to numerous sensors distributed across the various subsys- foster sustainable energy exchange paradigms, necessi- tems of a railway vehicle, with a particular focus on the tating the development of novel tools. These tools must critical train traction converter cooling subsystem. Oper- render photovoltaic production economically viable for ational data from a train fleet spanning several months energy trading while ensuring environmental sustainabil- is examined. The framework employs classic statistical ity through reductions in energy storage requirements data analysis techniques, such as trend estimation (e.g., and effective planning for grid load and dispersion/uti- architecture. This result allowed us to test an anomaly detection pro- cedure by comparing the predicted result with the power produced by each individual inverter. When consider- ing a single plant and computing the prediction error, it is possible to put into practice a simple but effective anomaly detection technique as shown in Figure 4. Figure 2: Global solar module manufacturing and solar PV capacity additions in the STEPS, 2010-2030 [1]. lization. Figure 4: A single day of PV Power production (scaled to Inverter Nominal Power) as continuous lines and the relative model predictions as dotted lines. Highlighted a single faulty inverter showing a transient power loss. The bar plot of the model prediction error for each inverter allows to easily rec- ognize the faulty one by applying a threshold. Figure 4 shows that an individual inverter has suffered a transient power loss that can be appreciated visually by comparing the predicted vs. produced power curves, Figure 3: The proposed approach for Power Forecasting. The training dataset is composed by weather data, irradiance but also quantitatively by thresholding the prediction model and PV configuration (red-dashed PV Pant Data). Tar- error. It is not yet possible to quantify an unambiguous get data is composed by real daily AC Power Production and threshold, and future work is investigating the possibility Solar Irradiance (bottom left graph). of a local adaptive approach. The proposed approach, as presented in Figure 3, en- 4. Predictive Maintenance in IoT tails the integration of a selected provider from among scenarios global numerical and commercial models. This choice is informed by preliminary results. The integration in- We are currently immersed in the Industry 4.0 era, volves coupling this provider with a mathematical and marked by the continual automation of traditional manu- physical model of irradiance [3], with the objective of facturing and industrial processes through modern smart effectively propagating irradiance contributions through technologies like Internet of Things (IoT) and Artificial suitable machine learning models. Long short-term mem- Intelligence (AI) ([4]). This evolution demands an increas- ory (LSTM) and Transformer Neural Networks (NN) mod- ing integration between physical and digital systems in els will be considered, with comparisons drawn against production environments, enabling the collection of vast classical machine learning techniques. To optimize the data from various smart equipment and sensors. model’s performance, an appropriate weighted combina- Smart sensors, devices generating data on physical pa- tion of losses will be employed during the training pro- rameters (e.g., temperature, humidity, or vibration speed), cess. The training is conducted using real data sourced offer functionalities ranging from self-monitoring to man- from managed photovoltaic systems situated at five dis- aging complex processes ([5]). tinct Italian sites. By leveraging this combination of Analyzing such data yields insights into machinery numerical, commercial, and machine learning models, health and production levels, driving strategic decision- the proposal aims to enhance the accuracy and predic- making for benefits like reduced maintenance costs, tive capability of the system, paving the way for a more fewer machine faults, optimized inventory, and increased effective utilization of solar energy resources. production. Maintenance procedures are a key focus, Preliminary results shows up to 0.536 ± 0.015% mean given their significant impact on industrial production absolute percentage error (MAPE) on power yield fore- and service availability. Industries are investing heavily casting (99% CI validated) using a seq2seq Transformer in equipping themselves with the tools for data-driven connected network module. maintenance strategies. The experimental results demonstrate how the pro- In the literature, two main approaches support main- posed approach effectively meets the demands of mod- tenance: model-driven and data-driven methods, with ern embedded AI applications, particularly benefiting hybrid-driven approaches gaining traction. While model- smart manufacturing systems where reliability, low la- driven techniques rely on expert theoretical understand- tency, privacy, and low power are critical. These findings ing, data-driven techniques leverage the vast information have significant management implications for optimizing available to detect machinery anomalies ([6, 7]). Hybrid- production line operations. driven solutions merge model and data fusion ([8, 9]). Future research could explore further applications of Maintenance management approaches, as categorized the attention mechanism in predictive maintenance. Ad- by ([10]), include Run-to-Failure (R2F), Preventive Main- ditionally, investigating what aspects the model priori- tenance (PvM), and Predictive Maintenance (PdM). Our tizes (i.e., receives more attention) could be insightful, focus is on PdM, which relies on data-driven analysis potentially leveraging eXplainable Artificial Intelligence to predict machinery failures, optimizing maintenance (XAI) tools to provide explanations. procedures and increasing machine longevity ([11, 12]). In many domains with complex data, machine learn- ing and deep learning techniques stand out for predictive 5. Forecasting Remaining Useful maintenance ([13, 14, 15]). These approaches use histori- Life in Aerospace Maintenance cal datasets to train models for predicting failures, such as Remaining Useful Life (RUL) estimation. The aerospace industry is known for its strict safety stan- However, deploying deep learning in real-world IoT dards, regulatory requirements, and the importance of ef- scenarios faces challenges due to computational limi- ficient maintenance to keep operations running smoothly. tations. Edge/fog computing solutions are favored but While traditional maintenance methods like preventive influenced by network connectivity ([16]). Embedded AI and reactive maintenance have their drawbacks in terms techniques are increasingly proposed for efficient, cost- of cost and accurately predicting failures, predictive main- effective data-driven analysis on industrial equipment tenance driven by AI and data analytics is a more proac- hardware ([17]). tive and cost-effective solution that can help overcome This proposal presents a deep learning approach for these challenges. predictive maintenance, leveraging a multi-head attention PdM uses data from sensors, operation logs, and main- mechanism for high RUL estimation accuracy and low tenance records to monitor equipment health and predict memory requirements, suitable for hardware implemen- failures. By forecasting the remaining useful life of key tation. Experimental results demonstrate its effectiveness components, it prevents unplanned downtime, cuts main- and efficiency, making it a promising solution for real- tenance costs, and improves safety. world PdM scenarios. The focus is on using advanced algorithms such as Figure 5 presents a high-level overview of the proposed LSTM networks and Transformer models to improve model architecture for the described PdM task, along maintenance schedules in the aerospace industry using with the data analysis pipeline necessary for generating the C-MAPSS dataset, which contains sensor readings estimated RUL values. and remaining useful life (RUL) values for turbofan jet en- The input comprises historical data from sensors pro- gines under different operating conditions. These types viding crucial information about the monitored machin- of deep learning models have shown great success in ery’s conditions, which includes a temporal component understanding intricate time-based patterns and distant crucial for detecting degradation trends. connections in data sequences. Once the input data is processed, it’s fed into the model The C-MAPSS dataset is highly valued in the research capable of capturing temporal dependencies between fea- community and widely used. Many researchers consider tures. By setting an appropriate time window, the input it a valuable resource for studying intelligent mainte- data fed into the model forms a matrix of size (𝑇𝑤 , 𝑁𝑥 ), nance and machine health prognosis as shown in the where 𝑇𝑤 represents the length of the input time window figure 6. and 𝑁𝑥 denotes the number of considered features. The The LSTM network, with its gating mechanisms (for- model output is a real number representing the remain- get, input, and output gates) and capability to selectively ing useful life of the machinery. The main components of retain or discard information over time, proved adept the proposed architecture include: positional encoding at modeling long-term dependencies intrinsic to time- block, accounting for the relative or absolute position of series data. Conversely, the Transformer model, orig- the time-steps in the input sequence; the attention mod- inally designed for natural language processing tasks, ule, comprising two sub-layers with residual connections employed a self-attention mechanism and position-wise between them: the multi-head attention block and fully Figure 5: Proposed AI architecture. performance and address specific challenges associated with time series data. Acknowledgments This work was supported in part by the Piano Nazionale Ripresa Resilienza (PNRR) Ministero dell’Università e Figure 6: Datasets used for RUL prediction della Ricerca (MUR) Project under Grant PE0000013-FAIR References feed-forward networks to capture distant relationships within the input sequences effectively. [1] IEA, World energy outlook 2023, in: International Through rigorous testing and analysis using various Energy Agency, IEA Paris, France, 2023. metrics like mean absolute error, loss function and 𝑅2 val- [2] J. Antonanzas, N. Osorio, R. Escobar, R. Urraca, F. J. ues, our findings show that LSTM networks outperform Martinez-de Pison, F. Antonanzas-Torres, Review Transformers in accurately forecasting RUL 1. of photovoltaic power forecasting, Solar energy 136 (2016) 78–111. [3] P. G. Loutzenhiser, H. Manz, C. Felsmann, P. Stra- Table 1 chan, T. Frank, G. Maxwell, Empirical validation of Comparison of metric results for LSTM and Transformer mod- els models to compute solar irradiance on inclined sur- faces for building energy simulation, Solar Energy MAE Loss 𝑅2 81 (2007) 254–267. LSTM 20.51 922.80 0.729 [4] E. B. Hansen, S. Bøgh, Artificial intelligence and Transformer 29.11 1609.31 0.517 internet of things in small and medium-sized enter- prises: A survey, Journal of Manufacturing Systems The results analysis indicates that while the Trans- 58 (2021) 362–372. former model shows good performance in predicting [5] W. Zhang, D. Yang, H. Wang, Data-driven meth- engine performance, it falls short compared to the LSTM ods for predictive maintenance of industrial equip- model, likely due to dataset limitations rather than ar- ment: A survey, IEEE Systems Journal 13 (2019) chitectural constraints. The Transformer’s self-attention 2213–2227. mechanism and lack of an explicit recurrent structure [6] A. Petrillo, A. Picariello, S. Santini, B. Scarciello, like LSTMs may require more diverse data to accurately G. Sperlí, Model-based vehicular prognostics frame- capture sequential patterns, especially in complex time- work using big data architecture, Computers in series tasks. Understanding and predicting engine per- Industry 115 (2020) 103177. doi:https://doi.org/ formance over time relies heavily on modeling long-term 10.1016/j.compind.2019.103177 . temporal dependencies, which can be challenging with a [7] P. Liu, L. Wang, R. Ranjan, G. He, L. Zhao, A survey limited number of representative examples in the dataset. on active deep learning: from model-driven to data- The Transformer’s ability to effectively handle such tem- driven, ACM Computing Surveys (CSUR) (2021). poral contexts is influenced by the quality and compre- [8] W. Luo, T. Hu, Y. Ye, C. Zhang, Y. Wei, A hybrid hensiveness of the training data it receives. Further re- predictive maintenance approach for cnc machine search could explore new model architectures, ensemble tool driven by digital twin, Robotics and Computer- techniques and optimization strategies to improve model Integrated Manufacturing 65 (2020) 101974. [9] M. Zhang, N. Amaitik, Z. Wang, Y. Xu, A. Maisuradze, M. Peschl, D. Tzovaras, Pre- dictive maintenance for remanufacturing based on hybrid-driven remaining useful life prediction, Applied Sciences 12 (2022) 3218. [10] G. A. Susto, A. Schirru, S. Pampuri, S. McLoone, A. Beghi, Machine learning for predictive mainte- nance: A multiple classifier approach, IEEE Trans- actions on Industrial Informatics 11 (2015) 812–820. doi:10.1109/TII.2014.2349359 . [11] M. Mohammadi, A. Al-Fuqaha, S. Sorour, M. Guizani, Deep learning for iot big data and streaming analytics: A survey, IEEE Communi- cations Surveys Tutorials 20 (2018) 2923–2960. doi:10.1109/COMST.2018.2844341 . [12] B. Sharma, L. Sharma, C. Lal, Anomaly detec- tion techniques using deep learning in iot: A survey, in: 2019 International Conference on Computational Intelligence and Knowledge Econ- omy (ICCIKE), 2019, pp. 146–149. doi:10.1109/ ICCIKE47802.2019.9004362 . [13] T. P. Carvalho, F. A. A. M. N. Soares, R. Vita, R. da P. Francisco, J. P. Basto, S. G. S. Al- calá, A systematic literature review of machine learning methods applied to predictive mainte- nance, Computers & Industrial Engineering 137 (2019) 106024. doi:https://doi.org/10.1016/j. cie.2019.106024 . [14] Y. Ran, X. Zhou, P. Lin, Y. Wen, R. Deng, A survey of predictive maintenance: Systems, purposes and approaches, arXiv preprint arXiv:1912.07383 (2019). [15] T. Rieger, S. Regier, I. Stengel, N. L. Clarke, Fast pre- dictive maintenance in industrial internet of things (iiot) with deep learning (dl): A review., in: CERC, 2019, pp. 69–80. [16] Y. K. Teoh, S. S. Gill, A. K. Parlikad, Iot and fog computing based predictive maintenance model for effective asset management in industry 4.0 using machine learning, IEEE Internet of Things Journal (2021). [17] M. Brandalero, M. Ali, L. Le Jeune, H. G. M. Hernan- dez, M. Veleski, B. da Silva, J. Lemeire, K. Van Beeck, A. Touhafi, T. Goedemé, et al., Aitia: Embedded ai techniques for embedded industrial applications, in: 2020 International Conference on Omni-layer Intelligent Systems (COINS), IEEE, 2020, pp. 1–7.