Exploring a Hybrid Case-Based Reasoning Approach for Time Series Adaptation in Predictive Maintenance

Abstract

Predictive Maintenance (PredM) is a vital concept within Industry 4.0, focusing on proactive machine maintenance through analysis of sensor data to uphold quality standards and prevent downtime. PredM traditionally employs data analysis methods or Machine Learning (ML) algorithms for anomaly detection in time series data from sensors. Despite ample error-free data, the occurrence of errors is rare. Case-Based Reasoning (CBR) ofers an adaptive artificial intelligence approach efective in domains with limited fault data. The sub-research area of Temporal Case-Based Reasoning (TCBR) explores the processing of time series data based on CBR methods. Integrating TCBR methods into PredM leverages human involvement, addressing data privacy concerns and facilitating knowledge transfer. While the retrieval in TCBR has already been investigated, the adaptation of the time series contained in the retrieval results has not yet been considered. On this basis, however, it is possible to determine the further course of the time series as an alternative to ML prediction approaches. For the PredM use case with rare fault data, it is important to determine the further course of the time series and how much time remains before a possible fault case occurs. This research summary therefore investigates a hybrid CBR approach that uses deep learning methods like transformers for adaptation. The aim is to predict the course of a time series as accurately as possible, which is evaluated for the PredM use case. Such a hybrid CBR model should also extend an explanatory component for the predicted time series.

Keywords

• eol>Temporal Case-Based Reasoning
• Internet of Things
• Time Series Data
• Hybrid Case-Based Reasoning
• Explainable Case-Based Reasoning
• Predictive Maintenance

1. Introduction

Predictive Maintenance (PredM) is a concept in the context of Industry 4.0 that aims to analyze machine and production data to proactively take care of machines [ 1, 2 ]. The aim is to prevent the occurrence of failures that could lead to breakdowns, downtimes, or safety concerns by identifying them in advance using analysis methods. Traditionally, Machine Learning (ML) methods are used for data analysis to detect PredM issues [ 3 ], such as remaining useful lifetime, fault diagnosis and fault prediction [ 4 ]. This analysis is based on the available data measured by the sensors. These are collected over a time period so that they are available in the form of time series [ 5 ]. Such a time series can be used to track the course of the fault in the use case of PredM. A model that describes this progression is the pf curve [ 6 ]. This is a concept which, in the context of PredM, shows the diferent phases of the service life of a system. The proactive phase ends at a point (p) at which the existence of a fault can be determined for the first time. From this point, an interval begins that ends with the occurrence of the error (point f), after which the error has serious consequences such as a production stoppage. During this interval, the fault can be addressed and rectified so that a system failure can be prevented. Once a fault has been identified and classified, it is uncertain how far away the current time point is from the upcoming point the error will occur. To estimate this, the further course of the time series must be predicted. Common methods for predicting time series also originate from the ML methodology [ 7, 8 ].

Deep Learning (DL) is a sub research area of the data-driven ML methodology that enables automatically learning by using neural networks [ 3 ]. The DL methods require suficiently large data sets to be trained on them. However, the availability of this data poses a challenge in the PredM domain [ 2 ]. Although a lot of data is available, these data sets are usually unbalanced, as there is a lot of good data but only a little fault data available. For this reason, the application of DL methods in this domain is often limited to pure anomaly detection. The integration of domain knowledge based on which further failure relevant information can be derived in DL methods requires high efort. Therefore, instead of a data-driven Artificial Intelligence (AI) method, the usage of a knowledge-driven approach is suitable where this data is available by default. Case-Based Reasoning (CBR) [ 9, 10 ] is such an adaptive technique that is based on previous experience and reuses their knowledge for new problem-solving situations. Instead of attempting generalization based on the limited data, experiences are directly reused [ 11 ]. This makes the application of CBR suitable for domains in which only little empirical knowledge is available, as in the PredM use case. Like the DL methods, CBR can support the human in the loop [ 12 ] in analyzing the PredM faults. In contrast to the use of neural networks, however, CBR ofers a fundamental explanation by solving problems based on past problems contained in a case [ 13 ]. This enables a domain expert to understand why a fault could be found in a new time series. Furthermore, the transfer of a developed CBR approach to other domains is more easily, by using approaches of abstraction [ 14 ] or generalization [ 15 ]. Such a transfer enables models to be passed on without sharing the underlying data, which is particularly advantageous in the industrial context due to protected production knowledge and desired data privacy [ 16 ].

Addressing domains such as PredM where time series data is processed is investigated in the subresearch area of Temporal Case-Based Reasoning (TCBR) [ 17, 18 ]. There is examined how temporal relationships can be expressed in cases and extended with further domain knowledge. While the retrieval phase has already been investigated in the TCBR area [ 5 ], only a few works [ 19, 20 ] address the reuse phase and the associated adaptation of time series. A use case for this adaptation of time series in CBR occurs in the PredM context of the further prediction of time series and an approximation to the pf curve. DL methods also exist for this purpose, but their functionality is limited due to the limited data available, and also take little or no domain knowledge into account.

For this purpose, an adaptation can be carried out based on the most similar cases identified in the retrieval to predict the further course of the time series on this basis. A DL procedure that works on the time series available in the most similar cases like transformers [ 21 ] can again be integrated into the CBR system as a methodology for this. Such techniques can be applied for the adjustment of time series so that one time series is transformed into another, considering the other case attributes and possible dependencies within the case (cf. [ 22 ]). The training can be carried out based on all available time series that are to be transformed into another, so that a suficient amount of necessary data can be made available for this DL method. By integrating such a model, a hybrid CBR approach is to be explored. Hybrid AI describes a system that consists of several subsystems that together form an intelligent system [ 19 ]. Thus, the hybrid CBR approach should include the phases of the CBR cycle [ 9 ] with focus on case adaptation. For the use case of PredM, a hybrid CBR method is developed that first classifies an error case and predicts the further course of the time series based on this experience. To evaluate the suitability of the adaptation in the revise phase, this CBR approach should also provide an explanatory component. This increases transparency of the prediction and therefore enables experts to understand the suggested further course of the time series.

To introduce the idea and approach for this contribution, the reminder of this paper is as follows: in Sect. 2 the overall objective of this PhD thesis is specified and divided into three research questions. The methodology for addressing the overall contribution as well as the individual research questions is presented in Sect. 3, where research artifacts are derived. In Sect. 4, a conclusion is drawn and the current progress in the processing of these artifacts is presented.

2. Aims and Research Questions

Based on the described problem, the following objective for this PhD thesis results: A hybrid CBR approach is to be investigated, which uses DL methods to enable time series adaptation for the PredM use case. This hybrid system should be based on the current state of research and, if possible, supplies an equal or even better performance than pure DL models. As a further improvement, the approach to be researched should ofer explainability to provide comprehensible results. The following three Research Questions (RQs) arise from this objective: RQ1: What is the current state of research in the field of TCBR? What related work exists on DL for time series prediction or their adaptation?

This question aims at identifying the state-of-the-art in the research area of TCBR and giving an overview of already explored approaches. In particular, existing approaches that can be used or extended for the PredM use case are investigated. Likewise, the state-of-the-art of DL methods is examined. RQ2: How can case knowledge be adapted in TCBR to make the best possible prediction of the further course of a time series in the reuse phase?

The aim of this question is using adaptation to predict the further course of the time series as accurately as possible based on the most similar cases identified in the retrieval. The result of this adaptation should enable to approach the phase of the pf curve for the PredM use case and how close a fault is. As no traditional adaptation methods are suitable for this, research is needed into how DL methods can be integrated into the adaptation phase in such a way that the most accurate prediction possible is made. In addition to reuse, this question also addresses the retain phase by investigating the learning adaptation knowledge.

RQ3: How can an explanation of the results of the hybrid CBR system be provided that enables domain experts in the revise phase to understand the adaptation result?

This question addresses the research area of explainable AI and the specific field of Explainable Case-Based Reasoning (XCBR) [ 23 ], with the objective to increase the explanation of the results of a CBR system by investigating diferent aspects of the explanation [ 13 ]. This should provide a transparent understanding of the predicted time series by the CBR system and therefore, strengthen confidence in its correctness.

3. Methodology and Research Artifacts

The Design Science Research (DSR) methodology according to Hevner [ 24 ] is used as the overarching methodology of this PhD thesis, based on which research artifacts are created and validated. The overall problem definition is already established in the context of this contribution, by PredM being addressed as the domain. A major issue in this context is the origin of suitable fault data. This can be provided by the Fischertechnik Smart Factory from the Internet of Things (IoT) Lab Trier1 [ 25 ]. Suitable data for this use case is already collected as part of preliminary work [ 2 ], which this thesis is building upon, and can be additionally generated if necessary. Based on the DSR methodology, new knowledge should always build on existing knowledge available in the knowledge base. In addition, problems and opportunities must be named, which is only partially done by the use case of the PredM. Accordingly, these areas are to be filled by a literature study on preliminary work in TCBR as well as on DL for time series according to RQ1. This study is conducted using established methodologies [ 26, 27, 28 ]. This lays the foundation for creating the hybrid CBR system by extending established approaches within the scope of this work and provides the answer for RQ1. In addition, the technical realization for the prototypical implementation and evaluation of the artifacts are performed based on the CBR framework ProCAKE [ 29 ]. This already supports a representation and similarity measures for temporal cases [ 30 ], which are extended in the context of this work. The CBR applications created with this framework as part of the research artifacts will also be published.

In the following is described which Research Artifacts (RAs) address the research questions presented in Sect. 2 and the methodology used to investigate them. RA1 – Hybrid CBR Adaptation Method for Predicting the Further Course of the Time Series: The modeling of the knowledge containers [ 31 ] of the vocabulary, the case base and the similarity measures is investigated as part of the literature survey, on which this artifact is based. Suitable methods for filling these knowledge containers for the PredM use case are already identified [ 5, 32, 33 ]. The fourth container of adaptation knowledge is not yet considered in the preliminary work, which is why it is aimed in the context of RQ2. Since traditional methods reach their limits here, as with the similarity measures [ 32 ], DL methods can also be used. Based on the results of the retrieval, the fault classification as well as the set of most similar cases are available. The time series contained therein are to be adapted in the reuse phase in such a way that the further course of the time series of the new problem can be predicted as well as possible. For this purpose, DL methods, like transformers [ 21 ], are investigated that adapt a prediction based on the most similar cases.

Similar approaches already deal with adaptive forecasting models [ 7 ]. In contrast to most ML research, in this hybrid CBR system the models are not adapted themselves, but the most similar time series are adapted for the new problem. A related approach by Corchado and Lees [ 19, 20 ] learns an adaptation in a hybrid CBR system with a neural network based on the most similar cases. Methods such as this are examined in this artifact, as well as possible other suitable approaches. Here again, the similar methods already collected in the literature review are used as a basis. Based on requirements for the use case of time series prediction in PredM, these are evaluated, and the best possible methods are prototypically evaluated in ProCAKE. To do this in the CBR framework, the related DL-based retrieval approaches [ 32 ] must first be made available there, so that the classification on which the adaptation is based is available as the first result of the reuse phase. Depending on the scope of this integration, this may result in a separate artifact. Based on the prototypical implementation of the complete hybrid CBR approach, this is evaluated based on the test data, e.g., by cross-validation. Depending on the quality of the results, it may be investigated whether an adaptation-guided retrieval can improve the results [ 34 ]. The methods are then compared with common ML models for time series prediction [ 7, 8 ] as a baseline. The criteria used to assess the quality of the results in the PredM use case will also be investigated on literature basis [ 35 ].

RA2 – Integrated Explanation Component: This artifact addresses the XCBR target of the justification [ 13 ] for the hybrid CBR approach investigated in RA1. The intention is to explain how a CBR application achieved a specific response. This means that the similarity value and how it came about should be explained, as well as the adaptation steps carried out and why these were selected. The problem of explainability is already a research subject for traditional CBR applications [ 23 ], although the CBR methodology ofers a basic explanation in itself through the reuse of past cases [ 13 ]. The integration of DL methods in both the retrieval and reuse phases increases the need for explanation for this hybrid system to ensure that domain experts have confidence in the predicted time series and thus in the predicted time points. Therefore, a combined approach that explains both, hybrid similarity calculation and hybrid adaptation, needs to be explored in the context of RQ3. To investigate this, the state-of-the-art for explanations in CBR as well as for DL methods must first be examined, and an explanatory approach selected based on requirements. In preliminary work, visualizations for structural [ 36 ] and process-oriented cases [ 37 ] are already examined to explain the similarity calculation. Such approaches can be transferred to the time series retrieval and extended for the reuse phase. These may be combined with suitable explanation methods for DL approaches [ 38 ], such as for transformers [ 39 ]. To evaluate the developed methods for their suitability, a requirements’ analysis is carried out for the application area of TCBR and maybe also in particular for the PredM use case, based on the literature results from the literature study. On this basis, suitable procedures are designed and prototypically integrated into the ProCAKE framework. A user study will evaluate these implemented approaches to show that this increases the explanation of the hybrid CBR system.

4. Conclusion and Progress To Date

This paper presents a research summary of a PhD thesis that addresses the adaptation of time series by using a hybrid CBR approach. The overall problem is motivated based on the use case of PredM. Since this thesis is at an early stage, the three research questions and the identified two research artifacts are presented, which will be addressed sequentially in future work. Within, a methodology for addressing each research question is developed. At the time this work was submitted, the literature review was already being processed and should be completed soon.

References

[1] T. Zonta , C. A. Da Costa , R. da Rosa Righi, M. J. de Lima , E. S. da Trindade, G. P. Li , Predictive maintenance in the Industry 4.0: A systematic literature review , Comput. Ind. Eng . 150 ( 2020 ) 106889 .

[2] P. Klein , R. Bergmann , Generation of Complex Data for AI-based Predictive Maintenance Research with a Physical Factory Model , in: 16th ICINCO Proc., SciTePress , 2019 , pp. 40 - 50 .

[3] O. Serradilla , E. Zugasti , J. Rodriguez , U. Zurutuza, Deep learning models for predictive maintenance: a survey, comparison, challenges and prospects , Appl. Intell. 52 ( 2022 ) 10934 - 10964 .

[4] C. Hegedüs , P. Varga , I. Moldován , The MANTIS Architecture for Proactive Maintenance , in: 5th CoDIT Proc., IEEE , 2018 , pp. 719 - 724 .

[5] L. Malburg , A. Schultheis , R. Bergmann , Modeling and Using Complex IoT Time Series Data in Case-Based Reasoning: From Application Scenarios to Implementations , in: 31st ICCBR Workshop Proc., volume 3438 , CEUR-WS.org, 2023 , pp. 81 - 96 .

[6] F. Nowlan , H. Heap , Reliability-Centered Maintenance , Technical Report AD/A066-579, National Technical Information Service , US Department of Commerce, Springfield, Virginia, 1978 .

[7] C. H. Fajardo-Toro , J. Mula , R. Poler , Adaptive and Hybrid Forecasting Models - A Review , in: 11th IEEM Proc. , Springer, 2019 , pp. 315 - 322 .

[8] K. Benidis , S. S. Rangapuram , V. Flunkert , Y. Wang , D. C. Maddix , A. C. Türkmen , J. Gasthaus , M. Bohlke-Schneider , D. Salinas , L. Stella , F. Aubet , L. Callot , T. Januschowski, Deep Learning for Time Series Forecasting: Tutorial and Literature Survey , ACM Comput. Surv . 55 ( 2023 ) 121 : 1 - 121 : 36 .

[9] A. Aamodt , E. Plaza , Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , AI Commun . 7 ( 1994 ) 39 - 59 .

[10] R. Bergmann , Experience Management: Foundations, Development Methodology , and InternetBased Applications , volume 2432 of LNCS , Springer, 2003 .

[11] D. J. Power , Decision Support Systems: Concepts and Resources for Managers , Quorum Books , 2002 .

[12] F. M. Zanzotto , Viewpoint: Human-in-the-loop Artificial Intelligence , J. Artif . Intell. Res . 64 ( 2019 ) 243 - 252 .

[13] F. Sørmo , J. Cassens , A. Aamodt , Explanation in Case-Based Reasoning - Perspectives and Goals, Artif. Intell. Rev . 24 ( 2005 ) 109 - 143 .

[14] R. Bergmann , W. Wilke, On the Role of Abstraction in Case-Based Reasoning , in: 3rd EWCBR Proc., volume 1168 of LNCS , Springer, 1996 , pp. 28 - 43 .

[15] E. Armengol , Usages of Generalization in Case-Based Reasoning, in: 7th ICCBR Proc., volume 4626 of LNCS , Springer, 2007 , pp. 31 - 45 .

[16] H. Tardieu , Role of Gaia-X in the European Data Space Ecosystem, in: Designing Data Spaces: The Ecosystem Approach to Competitive Advantage, Springer, 2022 , pp. 41 - 59 .

[17] M. D. Jaere , A. Aamodt , P. Skalle , Representing Temporal Knowledge for Case-Based Prediction , in: 6th ECCBR Proc., volume 2416 of LNCS , Springer, 2002 , pp. 174 - 188 .

[18] B. López , Case-Based Reasoning: A Concise Introduction , Synth. Lect. Artif. Intell. Mach. Learn., Morgan & Claypool Publishers, 2013 .

[19] J. M. Corchado , B. Lees , Adaptation of Cases for Case Based Forecasting with Neural Network Support , in: Soft Computing in Case Based Reasoning , Springer, 2001 , pp. 293 - 319 .

[20] J. M. Corchado , B. Lees , A Hybrid Case-based Model for Forecasting, Appl . Artif. Intell. 15 ( 2001 ) 105 - 127 .

[21] Q. Wen , T. Zhou , C. Zhang , W. Chen , Z. Ma , J. Yan , L. Sun, Transformers in Time Series: A Survey, in: 32nd IJCAI Proc., IJACI.org , 2023 , pp. 6778 - 6786 .

[22] R. Kumar , A. Schultheis , L. Malburg , M. Hofmann , R. Bergmann , Considering Inter-Case Dependencies During Similarity-Based Retrieval in Process-Oriented Case-Based Reasoning, in: 35th FLAIRS , 2022 .

[23] J. M. Schoenborn , R. O. Weber , D. W. Aha , J. Cassens , K.-D. Althof , Explainable Case-Based Reasoning: A Survey , in: AAAI-21 Workshop Proc ., 2021 .

[24] A. R. Hevner , A Three Cycle View of Design Science Research , SJIS 19 ( 2007 ) 4 .

[25] L. Malburg , R. Seiger , R. Bergmann , B. Weber , Using Physical Factory Simulation Models for Business Process Management Research , in: BPM 2020 Workshop Proc ., volume 397 of LNBIP , Springer, 2020 , pp. 95 - 107 .

[26] J. Webster , R. T. Watson, Analyzing the Past to Prepare for the Future: Writing a Literature Review , MIS Quarterly 26 ( 2002 ) 13 - 23 .

[27] B. Kitchenham , S. Charters , Guidelines for Performing Systematic Literature Reviews in Software Engineering , Springer, 2007 .

[28] D. Ridley , The Literature Review, SAGE , 2012 .

[29] R. Bergmann , L. Grumbach , L. Malburg , C. Zeyen, ProCAKE: A Process-Oriented Case-Based Reasoning Framework, in: 27th ICCBR Workshop Proc., volume 2567 , CEUR-WS.org, 2019 , pp. 156 - 161 .

[30] A. Schultheis , C. Zeyen , R. Bergmann , An Overview and Comparison of Case-Based Reasoning Frameworks , in: 31st ICCBR Proc., volume 14141 of LNCS , Springer, 2023 , pp. 327 - 343 .

[31] M. M. Richter , Knowledge Containers, in: Readings in Case-Based Reasoning , Morgan Kaufmann Publishers, 2003 .

[32] P. Klein , N. Weingarz , R. Bergmann , Enhancing Siamese Neural Networks Through Expert Knowledge for Predictive Maintenance , volume 1325 of CCIS , Springer, 2020 , pp. 77 - 92 .

[33] A. Schultheis , L. Malburg , J. Grüger , J. Weich , Y. Bertrand , R. Bergmann , E. Serral Asensio , Identifying Missing Sensor Values in IoT Time Series Data: A Weight-Based Extension of Similarity Measures for Smart Manufacturing , in: 32nd ICCBR Proc., volume 14775 of LNCS , Springer, 2024 . Accepted for Publication.

[34] B. Smyth , M. T. Keane, Adaptation-Guided Retrieval: Questioning the Similarity Assumption in Reasoning, Artif . Intell. 102 ( 1998 ) 249 - 293 .

[35] H. Engbers , A. A. Alla , M. Kreutz , M. Freitag , Applicability of Algorithm Evaluation Metrics for Predictive Maintenance in Production Systems , in: 6th IEEE Proc., IEEE , 2021 , pp. 413 - 418 .

[36] K. Bach , P. J. Mork , On the Explanation of Similarity for Developing and Deploying CBR Systems , in: 33rd FLAIRS Proc., FloridaOJ , 2020 , pp. 413 - 416 .

[37] A. Schultheis , M. Hofmann , L. Malburg , R. Bergmann , Explanation of Similarities in ProcessOriented Case-Based Reasoning by Visualization , in: 31st ICCBR Proc., volume 14141 of LNCS , Springer, 2023 , pp. 53 - 68 .

[38] W. Samek , T. Wiegand , K. Müller , Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models , CoRR abs/1708 .08296 ( 2017 ).

[39] A. Ali , T. Schnake , O. Eberle , G. Montavon, K. Müller , L. Wolf, XAI for Transformers: Better Explanations through Conservative Propagation , in: 39th ICML Proc., volume 162 of PMLR , PMLR, 2022 , pp. 435 - 451 .