We apply an anomaly detection method based on Hidden Markov Models and Hellinger distance to a Kairos mobile robot operating in the ICE lab, a research laboratory for Industry 4.0. Two main contributions are proposed: i) a decomposition of the Hellinger distance which allows to identify the causes of anomalous behaviours detected, ii) a graphical user interface that synchronously shows the robot movements in a map and the evolution of the Hellinger distance components, allowing a quick investigation of the causes of the detected anomalies. The tools are applied to a real-world dataset allowing to discover that an anomalous movement of the Kairos robot is caused by a wrong reading of the lidar from a window in the environment.
of data observed online (in a time window) and the most similar distribution in the learnt model of nominal behaviour, the detector successfully detects anomalies in the data.
Besides successfully applying the online anomaly detector to a real-world domain, in this work we propose two other contributions. From a methodological point of view we introduce a decomposition of the Hellinger distance in elements focused on specific variables (i.e., sensors and actuators). This decomposition supports the identification of the causes of the anomalous behaviours. From an application viewpoint we provide a prototypical Python GUI that synchronizes a map view of the robot moving in the environment with a chart view showing the time evolution of the components of the Hellinger distance, for all variables (i.e., sensors or actuators). This GUI allows an informative analysis of the causes of observed anomalies. In the results section we analyze a real anomaly detected by the system and provide interpretation using the proposed tools.
Some works employ deep neural networks for online anomaly detection in robot systems, three
recent examples are [
Other key diferences are present between our approach and state-of-the-art methods for
anomaly detection in autonomous robots. For instance, some works use supervised machine
learning approaches to classify data produced in real-time by a robot. The problem of these
methods is that they need fully labelled data, which are rarely available in the the context of
anomaly detection. Instead our method is completely unsupervised [
Among the approaches in the literature using HMM for anomaly detection, the ones most
similar to our approach are [
The dataset has been collected by sensors installed on a RB Kairos moving in the ICE lab environment. The environment, shown in the left-hand side of Figure 1, contains a reconfigurable production line with a storage system, from which the robot takes the raw materials and puts the ifnal products, and several tools (e.g., manipulators, a subtractive CNC machine, a 3D printer) used to generate and assemble the product. In the specific data acquisition sessions the robot had to move from the staring point to the ending point in the map. Three testing sessions were performed: two of them had nominal behaviours (see the red and blue trajectories in the map of Figure 1) and the third session shown an anomalous behaviour (see the green line in the map which moves close to a window instead of towards the end of the path).
The raw dataset has about 2300 variables and diferent number of time steps for diferent sensors, since each sensor has (potentially) diferent sampling interval. We focused our analysis on robot position (i.e., coordinates X and Y ), robot orientation (O) and distances between the robot and surrounding environment/objects. Distances are perceived by lasers. The robot collects 540 laser signals, with sampling angle of 0.5∘ . We performed data pre-processing with two main goals: to standardize the sampling interval and to reduce the resolution of the lasers. The first operation was performed because the anomaly detection technique requires an uniform sampling interval; the second operation was performed to reduce the dimensionality of the problem, since high dimensionality may generate mathematical issues in model learning and in the computation of the Hellinger distance. In particular, we generated three laser variables, that we called, Laser Left (LL), Laser Center (LC) and Laser Right (LR). LL is the average of laser signals with angles between 0∘ and 89.5∘ , which scan the environment on the left of the robot. LC is the average of laser signals with angles between 90∘ and 179.5∘ , which scan the environment in front of the robot. LR is the average of laser signals with angles between 180∘ and 270∘ , which scan the environment on the right of the robot.
After the pre-processing phase we obtained two datasets, each with six variables (i.e., columns), namely, X, Y, O, LL, LC and LR. The training dataset contains two nominal trajectories of 32 seconds each, for a total of 64 seconds and 340 time steps (i.e., rows), using a time interval of 0.2 sec. The test set contains a trajectory of 45 seconds (230 time steps) which is nominal for 11 seconds and then it diverges from the nominal path moving towards a window in the environment (see the green line in Figure 1).
We first describe the main properties of the anomaly detection technique proposed in [
Given a general -dimensional time series = {1, . . . , } composed of observations, where is a -dimensional vector representing the multivariate (multi-valued) observation at time , the nominal behavior (i.e., training set) of the robot is represented as = {1 , ...,
} and the observed behavior (test set) of the same system along some time period as = {1, . . . ,
}.
According to [
The online procedure computes the Viterbi path of the multivariate time series inside the window. For the state ˆ occurring most frequently in the Viterbi path a multivariate Gaussian distribution ( , Σ) is fit with the data inside the window. Then the Hellinger distance is computed between ( , Σ) and the emission probability of state ˆ in using the formula (, ) = 1 − (Σ 1)1/4(Σ 2)1/4 (︀ Σ1+Σ2 )︀ 1/2 2
{︃ 1
· − 8 ( 1 − 2)
If the distance is larger than a threshold , then a warning is reported. Full details of the
procedure are described in [
The second term of Equation (1) determines the size of the Hellinger distance for each window, namely, if this term is close to 1 then the Hellinger distance is close to 0, indicating a nominal behaviour, and if the term is close to 0 than the Hellinger distance becomes close to 1, reporting an anomalous behaviour. The second term can be split, again, in two factors. The first one, (Σ 1)1/4(Σ 2)1/4 (︀ Σ1+Σ2 )︀ 1/2
2 depends on the diferences in the covariance matrices Σ 1 and Σ 2 of the two distributions, while the second factor, depends on both the means 1 and 2 and the covariance matrices Σ 1 and Σ 2 of the two distributions. Considering variables 1, . . . , , with means 1 = ( 1,1, . . . , 1,) and 2 = ( 2,1, . . . , 2,) for distributions and , respectively, and assuming diagonal covariance matrices for distributions and , respectively, we developed the matrix calculations in the formula of the Hellinger distance and grouped by variable obtaining two Hellinger terms for each variable . Notice that the assumption of diagonal covariance matrices is reasonable in several applications. It only avoids considering covariances between diferent variables, hence the tool would not detect anomalies related to diferent covariances between nominal and observed behaviours. When small amount of data are available, as in our case study, the estimation of covariance parameters is very error prone, hence diagonal covariance matrices are used anyway in these cases that represent the majority of cases in real-world applications. The first term (Eq. (2)) is related to the first factor in the Hellinger formula, and the second term (Eq. (3)) decomposes the exponential in the second factor of the Hellinger formula. For the -th variable the two terms are: 1() = √2 (︁ 1, · 2,)︁ 1/4 ︁( 1, + 2,)︁ 1/2 (2) 2() = ( 1, − 2,)2 4 ( 1, + 2,) According to this decomposition the Hellinger distance can be written as (, ) =
{︃ ∏︁ 1() · − =1...
∑︁ 2() =1... }︃
Components 1() and 2() are very useful to identify the variables more involved in an identified anomalies. In the next section we show a real case study in which this decomposition allowed to identify the causes of an anomalous behaviour of the RB Kairos in the ICE lab.
The Python graphical user interface (GUI) here proposed is displayed in Figure 1. It shows the map of the environment on the left and the evolution of Hellinger components 1() and 2() on the right. The GUI is updated in real-time as new data are loaded from a csv file. At the beginning the marker representing the robot (a black circle) is positioned in the starting point (3) (4) of the map and the nominal trajectories (blue and red lines) are shown. In the chart on the right, the pink vertical line showing the time instant is placed on coordinate 0 of the x-axis. As time goes on, the robot marker is moved in the map and the related green trajectory is plotted. On the right, the green vertical line is moved to the right and the chart of Hellinger components for each variable is generated (in Figure 1 the entire chart is shown because the dataset is analyzed ofline). In the bottom of the GUI some buttons allow to start/stop the run and a slider allows to quickly move forward and backward if the dataset is analyzed ofline. The online version of the GUI only shows the trajectory and the chart until the current time instant and the slider allows only to focus on previously seen parts of the robot trajectory. Other commands allow to switch between diferent HMM models using diferent sets of variables (if they were learnt before).
We computed HMM models of nominal behaviors from the training set defined in Section 3 and using 4 diferent combinations of variables. We computed the accuracy of each model when used in the anomaly detector. The model of greatest interest was that using variables X, Y, O, LL, LC and LR. With parameter tuning we identified a good set of parameters, namely, 9 hidden states in the HMM and a window of 20 steps (i.e., trajectory intervals of about 3 seconds are considered to identify the anomalies). The accuracy of this model is 0.95 (with a precision of 1.00, a recall of 0.88 and a F1-score of 0.94). In the computation of the accuracy we used a ground truth manually defined considering the distance of the trajectory from nominal trajectories.
We tested the anomaly detector on the test set defined in Section 3 containing a known anomalous trajectory. As shown in Figure 1, the detector managed to identify the anomaly (see the red dots on the green trajectory). In particular, if we focus on the point where the anomaly starts (see the circle in the map and the corresponding pink vertical line in the chart) we see that the first Hellinger component that grows is that of variables LL and LC (the left and central lasers, see the red and purple lines). After about 2 seconds, the Hellinger component of variable LR (laser right) grows quickly and reaches a very high level. Only after 2 seconds the Hellinger component of the X position grows (blue line in the chart). This analysis shows that the causes of the anomalous behaviour of the robot, which strangely moves close to the window, are anomalous observations from the laser. For this analysis we only shown components 2() because components 1() were not informative (always close to 1) in this case study.
From a discussion with robot operators it emerged that a probable cause of the anomaly was the fact that the window during the testing trajectory was not covered by a curtain as it was during the nominal trajectories, hence the robot probably received wrong distances from the central laser which observed out of the room, it started moving towards the window to explore an area that it did not explored before, and moving closer to the window also the right laser started to get anomalous readings from the window. Hence the anomalous laser readings can be identified as very probable causes of the anomalous trajectory travelled by the robot.
The tests performed in this work on a RB Kairos robot show that the online anomaly detection methodology investigated is able to identify anomalous behaviours of robot. Furthermore, the decomposition of the Hellinger distance allows to improve the interpretability and to identify the causes of the anomalous behaviours. The GUI supports the analysis both in real-time and ofline. Future developments will focus on comparing the performance and the interpretability of this method with those of other anomaly detectors in the literature. Then we want to integrate the detector and the GUI in the ICE-lab demonstrator to show its capability in real-time applications.
The authors would like to thank Francesco Trotti for collecting the data from the Kairos robot and the ICE Lab of the University of Verona for allowing the use of robots. The research has been partially supported by the projects "Dipartimenti di Eccellenza 2018-2022, funded by the Italian Ministry of Education, Universities and Research (MIUR).