Anomaly and failure detection methods are crucial in identifying deviations from normal system operational conditions, which allows for actions to be taken in advance, usually preventing more serious damages. Long-lasting deviations indicate failures, while sudden, isolated changes in the data indicate anomalies. However, in many practical applications, changes in the data do not always represent abnormal system states. Such changes may be recognized incorrectly as failures, while being a normal evolution of the system, e.g. referring to characteristics of starting the processing of a new product, i.e. realizing a domain shift. Therefore, distinguishing between failures and such ”healthy” changes in data distribution is critical to ensure the practical robustness of the system. In this paper, we propose a method that not only detects changes in the data distribution and anomalies but also allows us to distinguish between failures and normal domain shifts inherent to a given process. The proposed method consists of a modified Page-Hinkley changepoint detector for identification of the domain shift and possible failures and supervised domain-adaptation-based algorithms for fast, online anomaly detection. These two are coupled with an explainable artificial intelligence (XAI) component that aims at helping the human operator to finally diferentiate between domain shifts and failures. The method is illustrated by an experiment on a data stream from the steel factory.
One of the most common approaches for automated monitoring of industrial processes is
anomaly and failure detection mechanisms. They allow for unsupervised health state monitoring
of an industrial facility and can reduce costs related to maintenance and unplanned stoppage
in production by alerting the operators of upcoming detected problems. However, in many
practical applications, industrial processes frequently adapt to shifts in production lines or
undergo on-demand reconfiguration, resulting in data patterns that may superficially appear as
failures while being a healthy evolution of the system or a consequence of variations in products’
specifications. This may cause anomaly detection or failure detection systems to trigger many
false positive alarms in case of technical changes in the production line, or in case of nontypical
production line configuration. Our research aims at diferentiating between healthy changes in
the data distribution and genuine failures that in such a scenario are indistinguishable from
state-of-the-art approaches [
In this paper, we focus on the dataset from the cold rolling facility of the steel factory. The
cold rolling process aims to reduce the steel thickness to a satisfactory level, making the product
ready to be sold to customers. The dataset was generated with the simulator that reflects the
physical processes of the real facility1. Such data present the state of many sensors in the
production line as a function of time and can be treated as a data stream [
Most of the earlier work has been focused on solving only one of these challenges. However,
in many practical applications, a comprehensive approach is needed that addresses all three
cases simultaneously. This is important, especially in situations where many domain shifts are
present in the data stream. In such a scenario, healthy but diferent from the statistical point
of view data (e.g. new products, rare products, equipment upgrades, etc.) can be mistakenly
interpreted by the automated algorithms as failures, causing a lot of false positive alarms,
which leads to stoppages of production and generating large, unnecessary maintenance costs.
It happens because failures and domain shifts usually are visible as the change in the data
distribution and therefore may be indistinguishable for automated detection methods. This
makes the problem even more challenging in situations where domain adaptation algorithms
are used to quickly update the model between healthy domain shifts [
On the other hand, it is not feasible for human experts to investigate and analyze each of the cases of possible failure/domain shift manually. We argue that developing new techniques to identify potential domain shifts and failures in the data, along with the assistance of a human operator supported by explainable artificial intelligence tools, can improve the online health monitoring of industrial processes.
Therefore, our goal is to propose a method that not only detects domain shifts and failures but also allows to distinguish one from another. Our method is composed of the following elements: the modified Page-Hinkley changepoint detector that identifies domain shifts and possible failures, supervised domain-adaptation-based algorithms for fast, online anomaly detection, and an explainable artificial intelligence (XAI) component that aims at helping the operator diferentiate between domain shifts and failures. The explanations provide insight into how the machine learning model uses particular sensor data between domain shifts to predict anomalies. In a case where there is a visible change in the way the data is used by the model (e.g.similar feature value contributes very diferently to the model prediction across two domain shifts), the expert may undergo additional analysis and order a stoppage of the system in a case of discovered failure or let the system operate if the change was a healthy domain shift. The workflow for our approach is depicted in Fig. 1.
m a e r t S a t a D
XAI
The remainder of the paper is organized as follows. In Section 2, we describe the background of our research. In Section 3 we provide a detailed description of a proposed method, which we later demonstrate on the steel factory data in Section 4. We summarize and discuss future extensions of this work in Section 5.
In this section, we review the relevant literature and discuss previous work related to explainable artificial intelligence, anomaly detection, and data streams, along with their applications in the industry.
Anomaly detection involves identifying patterns in the data that deviate from expected or
normal behavior. Anomalies in data often provide significant, and sometimes critical, actionable
insights across a wide range of application domains, which makes detecting them crucial. For
example, abnormalities in credit card transactions could signal credit card fraud or identity
theft [
Building a robust anomaly detection algorithm may not always meet all the needs of the business. In complex problems, understanding the factors that contribute to anomalies is crucial for gaining insights into the process. This understanding enables the implementation of efective maintenance actions and strategies to prevent future abnormalities. Therefore, employing explainable artificial intelligence solutions can be also very beneficial. XAI methods aim to improve understandability and transparency in the model decision-making process, highlighting which measurements contribute the most to current model decisions. A comprehensive study of interpretable artificial intelligence can be found in [ 12] or [13]. Related examples of widely used XAI methods include the following: LIME [14], SHAP [15] - as methods indicating feature importance for predictions, LUX [16] or counterfactual explanations [17] - which show how to change the input of the model to obtain a desired model output [18]. Nevertheless, most XAI methods are designed for static data and do not directly handle the dynamic nature of industrial data and the data shift. Some recent works for such applications are as follows. The authors in [19] proposed a framework for online anomaly detection in a cold rolling mill that includes the XAI module, helping the operator better understand the origin of the anomaly and take the necessary actions. Ding et al. [20] applied the SHAP module to enable transparent decision–making processes within the diagnostic framework for hot-rolled strip crown.
Generating data continuously at regular time moments and observing changes in data
distribution over time is a common industry practice. Furthermore, because the processing must be
done in real–time, data stream learning [21] is a suitable method for this environment. Let us
recall that a data stream can be defined as an unbounded sequence of instances which arrive at
high speed and require fast processing [
The detailed description of handling streaming data from steel industry facilities can be found in [23]. Changes in the data stream (referring to concept drifts) can significantly impact model performance, potentially resulting in a decrease in production quality. Therefore, monitoring these changes and responding to them are essential aspects of daily operations. To address this challenge, drift detection methods could be applied, such as the Early Drift Detection Method (EDDM) [24], Adaptive Windowing (ADWIN) [25], Page–Hinkley (PH) [26] or others [27].
However, the aforementioned methods do not cover all possible problems that occur in the industry environment. On the one hand, re-configuring production or starting processing a new product might result in incoming samples being marked as anomalies or failures. On the other hand, the data characteristics of new products and potential failures can be similar, making them dificult to distinguish for drift detection methods. In our work, we introduce a method designed to identify changepoints in data streams indicating possible data shifts and failures and provide explanations to assist operators in better diferentiating between them. 3. Method for diferentiating between failures and domain shifts The proposed method consists of three subsequent steps: 1) distribution change discovery, 2) domain adaptation and 3) explanation of the diferences between the source model and the adapted one. First, the detector for identification of the changes in the multivariate data stream is presented. These changes trigger a domain adaptation algorithm that adjusts the anomaly detection model to the changing data distribution. Up to this point, it is still unknown if the distribution change is a healthy data shift or a failure. Finally, an explainable artificial intelligence algorithm is used to provide the feature importance for the classification algorithm before and after the adaptation. These diferences in importance are used as a decision support component that helps the human operator diferentiate between failures and domain shifts. The general workflow of the method is presented in Figure 1. 3.1. Changepoint detector Let us assume that the data instances arrive as the data stream = {x}=0, where x = [ ]=1 is –dimensional attribute vector. The first elements of the stream constitute the reference distribution = {x}=0. The next object in the stream will be approximate distribution = {x+1}. The KL divergence between and is then estimated following the formula presented in [28]: ̂︀( || ) = ∑︁ log (x) 1 =0 (x) + log − 1 , (1) where (x) and (x) are respectively, the Euclidean distances to the ℎ nearest–neighbour of x in ∖ {} and . Further, ̂︀ is added to Page–Hinkley (PH) drift detector. If PH does not trigger an alarm (the threshold for ̂︀ is not exceeded), the reference distribution increases by the sample . Otherwise, the detected drift indicates the changepoint in data stream , which brings about the need for adjustment of the reference distribution. In this case, the reference distribution consists of data stream objects starting from sample (︀ = {x, x+1, . . .})︀ . 3.2. Domain adaptation classifier The subsequent phase is the construction of a domain adaptation classifier : → {0, 1}, where 0 represents the normal instance and 1 refers to the anomaly. As a domain adaptation model, the feature–based classification and contrastive semantic alignment (CCSA) was selected [ 29]. We assume that the data preceding the initial changepoint constitutes a source domain (in the industry scenario analyzed, the source domain data originates from the most prevalent rolled product). Let = {(x, )}=0, where x ∈ and ∈ {0, 1}. The data coming after the changepoint belongs to the target domain . To train the domain adaptation classifier, a small batch of samples is taken. Let the ℎ = , and = {(x, )}+=, and > , ≪ . The prediction is made for the next batch of samples = {(x)}+=2 +. The motivation behind this step is to obtain a classifier making accurate predictions with a limited number of samples from the target domain. After prediction, the target domain is expanded with the predicted batch. This procedure continues until a new changepoint is identified. Taking into account the sudden nature of process changes, the domain adaptation model discards previous targets and initiates adaptation to the new product from the beginning. 3.3. Explanation of the changes of model behavior between changepoints The last step of the proposed method focuses on explaining the decision of the domain adaptation model compared to the source model. It allows us to observe possible abnormal changes in the usage of feature values that may indicate adaptation to failure. We have decided to use the SHAP (SHapley Additive exPlanations) [15] because it allows for monitoring changes in feature importance within model predictions which adapts to diferent domains. We hypothesize that monitoring the evolving impact of diferent features on anomaly prediction over time could help in distinguishing between failure and domain shift.
To accomplish this, the explanations are determined on each training batch of samples . The Shapley value is determined by evaluating the feature value across all possible combinations with other features, weighting and summing the results: Φ =
∑︁ ⊆{ 1,...,}/{} ||!( − | | − 1)! (( ∪ ) − ()), ! (2) 2000
Failure 4000 instParnocdeuct 6000
SHAPvaluesforcurrent_2
trcPoud3 (1, ..., )P∈/ − ( ()), (3) where is the subset of features in the model, and is the total number of features, is the vector of feature values of the instance to be explained.
Such calculated Shapley values show the contribution of the feature values of a particular instance to the prediction of the model with respect to the expected value. This contribution can be positive or negative denoting if the particular value has a larger efect for the positive or negative class respectively, in the case of binary classification. This information confronted with expert knowledge on how the values of diferent machinery sensors contribute to healthy and unhealthy conditions can help to determine if the classification model behaves as if it were a healthy state (also referring to the product shift) or as if it were a failure.
To provide a broader spectrum of such a behavior of the model, we calculate and plot Shapley values over time and present them to the expert for a final decision on the nature of the change. The specialist should pay attention to the explanation of the features that are crucial for the proper operations of the production line. If there is an inconsistent with expert knowledge change in explanations for those key features, then the specialist should look at the raw signals and decide what is the root cause of the change. The example is presented in Figure 2. The expert indicates current_2 as a key feature for anomaly detection in bearing degradation. When the failure begins, the median of SHAP values drops below zero, which should be an alert of unexpected bahavior. There is also a change of characteristics in raw signals for instances where a failure was identified. Adding the knowledge about e.g. previous repairs, the expert should decide if the change corresponds to the new product (domain shift) or failure.
This section presents a case study illustrating the use of our method in an industrial environment. We examined the cold rolling process in a steel plant, where diferentiating between failures and new types of rolled products is crucial because of the variety of diferent products being rolled over time. The following subsections describe the cold rolling dataset and the experimental outcomes.
Entry coil
Tension roll Exit coil Recoiler 4.1. Cold rolling mill dataset The cold rolling process is responsible for reducing the strip’s thickness by passing it through a pair of rolls. Initially, the steel coil is unrolled on an uncoiler. During rolling, the strip moves slowly between the rolling stands. Once the uncoiler grips the strip, the process speeds up to reach the desired thickness (Figure 3). During the rolling process, various products diferent in mechanical properties, thickness, width and reduction are processed. The analyzed dataset Uncoiler
Backup Work roll
roll comes from a simulator of a four–stand rolling mill, which was created using real measurements from a steel company. The raw data samples are collected at regular time intervals, and all the signals are listed in Table 1. For experimental purposes, 10000 samples were acquired containing parameters from 4 types of rolled products. The dataset composition was chosen to reflect the real–world situation, where most of the production consists of one type of product (or related). Each product in the dataset is described by three parameters: ithick/othick/width. The anomalies represent abnormalities in the mechanical degradation of bearings. The expert from this particular rolling infrastructure indicates electric current and mechanical torque as the most important parameters responsible for bearings degradation. The dataset composition is presented below: • Product 1 –3.4/1.61/918.67, the major (source) product which constitutes 80% of the dataset, about 8.5% anomalies corresponding to bearings on all stands. • Product 2 –3.0/1.11/1082.43, constitutes 5% of the dataset, about 10.8% anomalies corresponding to bearings on all stands. • Failure on Product 2 – 3.0/1.11/1082.43, constitutes 5% of the dataset, about 77.4% anomalies simulating bearings failure on stand 2. • Product 3 –2.8/0.82/918.58, constitutes 5% of the dataset, about 6% anomalies corresponding to bearings on all stands. • Product 4 –3.5/1.44/1080.20, constitutes 5% of the dataset, about 12.6% anomalies corresponding to bearings on all stands.
These elements appear in this order in the analyzed stream. Note that products 2,3 and 4 are rather rare events compared to the much longer production of the main product 1. This is similar to research on class imbalanced and concept drifting streams, which are particularly dificult for most learning algorithms, see [30]. 4.2. Classical approaches for anomaly detection For detecting anomalies we chose the state-of-the-art techniques like Isolation Forest (IF), Local Outlier Factor (LOF) and One Class Support Vector Machine (OCSVM), Autoencoder (AE). The models were trained on signals from the source product (Product 1) and evaluated on the rest of the stream (target products). In the Figure 4 the algorithms results on current and torque from stand 2 is presented. The dashed lines denote instances when a new product/failure appears on the production line. Grey lines signify instances where anomalies were detected. It is observable that all tested algorithms tend to classify both products and failures as anomalies. The potential reason is that the measurement characteristic is diferent from the source product for both of those cases. Consequently, it becomes exceedingly challenging for anomaly detection algorithms to discriminate between failures and newly introduced rare products. 4.3. The proposed method In the subsequent section, we present experiments conducted using the proposed method, which leverages transfer learning and explainable artificial intelligence.
Firstly the domain adaptation from the source product to each target is conducted. For domain adaptation, we selected the feature–based classification and contrastive semantic alignment (CCSA) algorithm [29]. The CCSA goal is to create an encoded space where the distances between source and target pairs with the same label are minimized, while the distances between pairs with diferent labels are maximized. This model was chosen because semantic alignment algorithms achieve good results even with a small number of samples in the target domain. Subsequently, the SHAP explainer is applied to each target domain, and the median SHAP values for each parameter are computed. This approach aims to determine if there are diferences in SHAP explanations (i.e. median feature importance) between products and failures. In the Figure 5 can be observed that adaptation for failure leads to clearly diferent median SHAP values for the parameters current_2 and torque_2 which expert has identified as the most critical for failure diagnosis. 1 tcu rPod 0 1 tcu rPod 0 1 tcu rPod 0 1 tcu rPod 0 Figure
4: stand 2.
Anomaly Failure Product instance LOF results for cur ent_2
Anomaly Failure Product instance OCSVM results for cur ent_2
Anomaly Failure Product instance AE results for cur ent_2
Anomaly Failure Product instance reu li Fa reu li Fa reu il Fa 100000 r_oeuq2 t 80000 60000 40000 20000 0 180000 160000 140000 120000 100000 r_oeuq2 t 80000 60000 40000 20000 0 8000 7000 6000 5000 0 1 trcPoud 1 trcPoud 0 1 tcu rPod 0 2000 4000 6000 8000 10000 2000 4000 6000 8000 10000 2000 4000 6000 8000 10000 2000 4000 6000 10000 2000 4000 6000 8000 10000 0 2000 4000 6000
10000 Anomaly Failure Product instance LOF results for torque_2
Anomaly Failure Product instance OCSVM results for cur ent_2
Anomaly Failure Product instance AE results for torque_2
Anomaly Failure Product instance 2 trcPoud 8000 2 tcu rPod 8000 2 trcPoud reu li Fa reu li Fa reu li Fa 3 trcPoud 3 tcu rPod 3 trcPoud 4 trcPoud 4 tcu rPod 4 trcPoud 2000 4000 6000 8000 10000 2000 4000 6000 8000 10000 Anomaly detection alg orithms evalu ation on targ et products on importa nt signals from rolling The next step was to explore whether the explanations of specific parameters changed during the rolling process and in what places it occurre
The experiment started by calc ulating the KL divergence using the formula 1 and applying PH drift detector.
The experimental parameters were defined as follows: 40% of the dataset was designated as the reference distributio n, and the fe atures used for calculation included the current and torque on each stand.
Such defined changepoint dete ctor marked changes in samples 8010, 8550, 9042, and 9515 when the real points were in 8000, 8500, 9000 and 9500.
The outcome of applying the PH drift dete ctor to the estimated KL can be observed in
Upon detecting changepoints, we adapted from source to target products in batches.
Specific ally, the
CCSA model was retrained after each batch of incoming samples from the target domain (demonstrated with batch_size=50 below).
Each time a new changepoint was detecte d, the adaptation process was restarte training, a SHAP explainer was constructed using the training batch of samples. Subsequently, the explanation of specific parameters was presented as a box plot for each training batch to monitor the evolution of SHAP values (Figure 7). Furthermore, the maximum (red line) and minimum (green line) SHAP values were visualised. The plots clearly indicate that for current_2 and torque_2, there is a shift in SHAP value explanations as the model begins to adapt to failure conditions. The median and minimum SHAP values decrease compared to those observed during the adaptation to products. Moreover, it is notable that the SHAP explanations for other rolling stands exhibit reduced variability during failure. To assess whether changes could be detected by an automated method, the Page-Hinkley (PH) drift detector was applied to the medians of the SHAP values. While drifts were identified in stand 2, additional detected changes in torque_3 and torque_4 were observed, which are likely false positives resulting from the sensitivity of the PH detector.
The proposed method is designed to assist experts in detecting potential failures when raw data points are insuficient. In cases of some products, the signal may closely resemble the failure state, making it challenging to definitively classify by operator as healthy or failure. The evolving boxplots significantly change their characteristic when the domain adaptation model begins adapting to failure conditions. The abrupt decrease or increase in the median of SHAP values should alert experts and be a starting point for in–depth analysis. By examining these plots, experts in specific rolling infrastructures could combine our analysis with raw signals and the machinery condition (e.g. recent bearing replacement, last renovation etc.) to determine if a failure is beginning in the rolling mill. Automating the decision–making process to distinguish between failures and new products remains a direction for future research.
SHAP values for cur ent_1
SHAP values for torque_1 SHAP values for cur ent_2
SHAP values for torque_2 SHAP values for cur ent_3
SHAP values for torque_3 SHAP values for cur ent_4
SHAP values for torque_4 t3 c u rod P t3 c u rod P t3 c u rod P t3 c u rod P
In this work, we propose a method that allows for human-guided diferentiation between failures and healthy domain shifts. We presented the feasibility of our approach on a dataset from a t3 c u rod P t3 c u rod P t3 c u rod P t3 c u rod P t4 c u rod P t4 c u rod P t4 c u rod P t4 c u rod P minimum maximum Product
Failure minimum maximum detectedchange Product Failure minimum maximum Product Failure minimum maximum Product Failure cold rolling facility of a steel factory and published the source code along with the dataset used in experiments for full reproducibility. We demonstrated that with state-of-the-art anomaly detection methods, it is not possible to distinguish between healthy domain shift and failure. On the other hand, our approach adds an additional information layer that human operators can use to make a more informed decision about the nature of changes in the data. We consulted the results with an expert from the steel factory, who confirmed the correctness of the results obtained with our method.
Although our primary focus in this work was on the industrial application of the method, it is more generic and can be extended to other cases where the following conditions are met: (1) The data come sequentially and may represent diferent distributions (domain shifts). (2) There are some hidden similarities between the distributions that can be captured by domain adaptation (transfer learning) algorithms. (3) The data is analysed in batches, and in each batch, the majority of samples are considered candidates for the healthy / failure distribution, while the minority is considered anomalous. These conditions can also be found in other areas where the data exhibit dynamic and non-stationary characteristics and adaptability is one of the features of the system. In particular, such a situation is present in various predictive maintenance applications where typical maintenance actions such as replacements of an old component may cause changes in the data that result in the degradation of existing AI models [23]. Another example is network trafic monitoring [ 31] or electricity grid monitoring [32], where many diferent changes in the data appear that may reflect the normal change in electricity or bandwidth demand or a failure or attack. Reaching beyond the industrial cases in patient health monitoring, one can observe the same challenges, especially in online, long-term monitoring of patient health state, smart home installations, or ambient assisted living setups, where changes in habits may be confused with life-risk situations [33].
Future works can include many diferent path directions, as outlined in the following paragraphs. In the current version of the method, the human operator is responsible for the analysis of the explanations. This process can be automated to some extent by the analysis of the dynamics of explanations over time, instead of raw feature values. This technique has been shown to be efective in similar use cases, as described in [34].
The other direction is to focus on concept drift instead of domain shift. Recent works approach concept drift handling with the usage of domain adaptation techniques [35]. However, they mostly address the problem of performance of the model, not touching the explainability aspect. Extending the work presented in our research with concept drift detection and explanation can scale the applicability of the approach to areas where keeping human in the loop is crucial (e.g. medicine, healthcare, etc.).
Finally, one of the limitations of the presented approach, which is also one of the hardest problems to handle in streaming data, is proper changepoint identification in the case of gradual data shifts. Furthermore, the work presented in this aims primarily to distinguish between failures and domain shifts. It does not explicitly explain the root causes of the failures or the domain shifts. In this work, we discuss the preliminary results of our method, and in future works, we aim to address all of the above.
This paper is part of a project that has received funding from the European Union’s Horizon Europe Research and Innovation Programme, under Grant Agreement number 101120406. The paper reflects only the authors’ view and the EC is not responsible for any use that may be made of the information it contains.
Jerzy Stefanowski’s work was supported by the National Science Centre (Poland) grant No. 2023/51/B/ST6/00545.