This paper investigates data-driven methods to predict failures of a rotating condenser (RotCo) inside a synchrocyclotron for a proton therapy treatment system [12]. Downtime caused by a failure of the system can lead to signi cant delays in the treatment of the patients, which is why having a reliable predictive maintenance system is essential. The condenser rotates at high speed and rolling bearing elements are responsible for maintaining low friction between the moving components. The aim is to predict failures of the bearing box which contains the shaft and the bearing elements. Several sensors within the cyclotron are constantly measuring multiple relevant signals but, notably, vibration data is not available. We leverage those time-series data to predict a few days in advance whether a failure is likely to happen. To do this, we propose a two-level approach that relies on combining the output of a classi er with an aggregator based on a custom business metric speci cally designed for this problem.
in our situtation. Hence, we must follow a di erent kind of approach: we chose a purely data-driven framework based on machine-learning.
Eight sensors constantly measure the health of the RotCo at a rate of 1Hz. The signals measured and their descriptions are shown in Table 1 below.
From those time-series signals, we want to predict whether the machine is going to fail in the next days ahead. The business need is to detect a failure ve days in advance (seven days being even better) so that if a sign of failure is detected on a Monday, a maintenance can be scheduled in the week-end to limit the downtime. Based on theses business considerations, a speci c metric was designed to evaluate the performance of a given prediction method and is depicted in Fig. 1. It provides a score between zero and one (higher is better) computed from a piecewise-linear function of the number of days between the prediction and the actual failure: { A prediction 7 days ahead gives a perfect score. { Predictions between 7 and 0 days assign a score that decreases linearly to zero (with a slightly higher slope between 7 and 5 days). { Similarly, predictions ranging from 7 to 15 days are assigned a score linearly decreasing from 1 to 0 (with a slightly higher slope between 7 and 10 days). { Finally, predicting a failure more than 15 days in advance leads to a zero score to re ect the unexploited lifetime. 2
There exists multiple ways to tackle a predictive maintenance problem, coming
from di erent research elds and answering di erent kind of questions. A large
number of works can be found in the literature, and many surveys are available:
Jardine et al. [6], Peng et al. [13], Ahmad and Kamaruddin [
1 0.9 0.8 15 10 Viewed at a conceptual level, techniques for maintenance decision making can be split into three categories: 1. Anomaly detection techniques [2] consist in attempting to ag anomalous behavior and isolate or identify faults, and is sometimes referred to as diagnostics in the literature. 2. Techniques based on classi cation [16] try to predict whether a system is going to fail within a given time-window. 3. Techniques based on regression try to estimate the Remaining Useful Life (RUL), i.e. how much lifetime remains before a failure, and is sometimes referred to as prognostic in the literature [5, 9].
RUL estimation usually involves making an assumption of the degradation process via health indicators (directly observed or created via feature engineering) and predict the remaining lifetime based on this assumption. We do not make any assumption on the degradation process and therefore exclude this approach. Then, as our metric for prediction quality clearly involves time, we choose the classi cation approach over anomaly detection, where we will try to predict whether a system is going to fail within a given time horizon. To t the classi cation setting we will label training data points as nominal or critical, depending on whether failure occurs within that time horizon.
A second aspect, orthogonal the the above discussion, is the type of model used.
The literature distinguishes between physics-based models (e.g. [8], [11]),
knowledge based models (expert & fuzzy systems, e.g. [4]), threshold-based signal
processing (e.g. [
Speci c predictive maintenance techniques dedicated to rotating machines and bearing elements also exist but almost always rely on vibration data which, as mentioned before, are not available. For an overview on maintenance techniques for rotary machinery, the reader can refer to [3, 7]. 3 3.1
We collected about a year of data during which 9 replacements occurred. From those 9 replacements, 8 were real failures and 1 was a preventive replacement. Due to the high number of data points (for one year of data: about 30 millions points per signal), we decrease their number by computing averages over 30-second intervals, leading to about 500,000 data points per signal which is more tractable. All experiments will be run using those. 3.2
After cleaning the data and removing physically impossible values (due to sensor errors), we create some new features in the time and frequency domain. Those features are computed on adjacent non-overlapping 2-hour long time windows, separately for each signal.
The time domain features used for this research are computed as follows: for each signal, the following are computed over each time window (240 samples): mean, standard deviation, skewness, kurtosis and sum of peaks, which is the number of peaks occurring in a time window. A peak is de ned as a local maximum (by comparison with its two neighbouring values) that is greater than median(X) + 2mad(X), where median(X) and mad(X) are respectively the median and the median absolute deviation of the window. The frequency domain features are computed via Fourier transform of the time window values (which contains 240 samples). Those amplitudes are split into ve frequency bins (i.e. 48 samples). Then, in each of those ve bins, we compute the average of the ve largest amplitudes, and use those as features. This procedure is used as a tradeo between using the average and the peak amplitude inside of each bin.
Independently of how those features are designed, we investigate two di erent ways of handling each time-series: a simple approach where each data point contains all features computed over a single two-hour time window for each signal, and a history-enriched approach where we gather for each signal the twelve most recent values of each feature (i.e. going 24 hours backwards in time), with the objective of including enough details about the history without adding too many variables (to avoid risking over tting and/or excessive computational burden). The simple approach thus contains 8 10 = 80 features (8 signals and 10 features, being the 5 statistical and 5 frequency features) while the history-enriched case contains 8 10 12 = 960 features (8 signals, 10 features and 12 time windows). 3.3
We chose a two-level approach, depicted in Fig. 2, that consists of a support vector machine (SVM) binary classi er at the lower level, trained to distinguish between nominal and critical time windows, combined with a higher-level aggregator. A Radial Basis Function (RBF) kernel is chosen for its general-purpose and good accuracy. A one-level approach based on a multi-classi er SVM, where each class corresponds to a certain range of days before failure, was also a possibility but would not have been fully compatible with the considered business metric.
- Preprocessing - Feature extraction
SEQUENTIAL INPUT
Data is rst preprocessed and the features are extracted (see section 3.2 above). The processed data is then sent as input to the SVM (step 1 in Fig. 2) that classi es whether the data is critical or nominal.
The result of the SVM is then fed into the aggregator whose responsibility is to decide whether to trigger an alarm or not. The output of the SVM fed into the aggregator is not just its binary class output, but the distance to the hyperplane de ned by the decision function
n f (x) = X iK(xi; x) + b
i=1 where K(xi; x) = exp( kxi xk2) is the RBF kernel used in the SVM. The aggregator computes a rolling average of the SVM decision function over 8 hours, and raises an alarm if this average exceeds a prede ned threshold. Based on the distance interpretation of the decision function (margin) we selected a zero value for the threshold.
For training the SVM, we need to de ne a labelling. As explained in the introduction, we label each data point as nominal or critical based on whether a failure occurred during some xed time horizon after the considered time window. As we do not have a precise idea of when the machine actually enters a critical state (which may not even correspond to an actual physical condition), we choose the length of that horizon somewhat arbitrarily to be equal to ve days. All data points between the failure date and 5 days before the failure date will be marked as critical and the remaining points as nominal. Other lengths for the labelling horizon have been tested in a preliminary analysis but performed slightly worse.
The general architecture of the training process is depicted in Fig. 3. After data is processed and labelled, it is split in 9 folds, where each fold is a run-to-failure (or a run-to-preventive replacement in one case). Each fold is thus once used as a test set. The SVM is trained on all other folds (STEP 1) and is used to predict on the remaining fold. The output of the classi cation of this fold is fed to the aggregator that triggers an alarm if it exceeds the zero threshold. As soon as the alarm is triggered, we compute its di erence in time with the actual failure, from which we compute the business metric according to Fig.1. The business score is then averaged on the 9 test folds.
To reduce the dimension of the problem, we use a backward feature selection process with a wrapper approach. We start with all features included in the model and recursively eliminate among all remaining signals or aggregation types the one that increases the score the most (which indicates some level of over tting for the model with all features). We do this until the score starts to decrease. A cross-validated grid search is done concurrently with the feature selection. The Raw data - Preprocessing - Feature extraction - Run-to-failure split(folds) Processed datL5oaatdhbaeeyrlswdbaiseteafoNfroOerMfcalIaiNlsusArieLficCaRtIiToInCA:L IILTTEPAUNN
U Q
SE Labelled data
Fo…ld 1 Fold n
STEP 1
SVM classifier
Alarm signal
Failure date –
Alarm date Business metric computation
Business metric computation
Average HYPERPARAMETERS: - Feature selection - Grid search (SVM)
Best model principle is the following: We start with all features and recursively eliminate one signal or one aggregation type, i.e. the candidates for removal are Speed, Torque, Current, Voltage, Bearing Temperature, Motor Temperature, Pyrometer temperature, Vacuum, mean, std, skewness, kurtosis, sum of peaks and frequency bins 1,2,3,4,5. This gives a total of 18 candidates. Once we remove a candidate, we remove all the features associated with it. When a signal is eliminated, we remove all its aggregation and history, and when deleting an aggregation type, we remove it in all signals and history. Hence, we remove multiple features per selection step. The candidate removed is selected so as to maximize the business metrics. The backward selection is stopped whenever the removal of a candidate decreases the score by more than one percent.
This process is computationally intensive. Indeed, the rst step of the feature selection requires 18 evaluations on di erent sets of features and, for each set of features, a grid search on the parameters of the SVM with a 9-fold crossvalidation is performed. Hyperparameters of the SVM are the penalty of misclassifying data points, C and the parameter of the RBF kernel . The grid chosen is a logarithmic grid with the following values: C = [1; 10; 100; 103], and = [10 5; 10 4; 10 3; 10 2]. Thus, the rst step of the selection alone already requires 18 feature sets 16 parameters 9 folds = 2; 592 ts. For each set of features, we thus have 144 scores (from the grid search). For each hyperparameter set, we take the average value across all folds and the score of the feature set is chosen as the maximum value across the hyperparameters.
4.1
The results of the feature selection (with a cross-validated grid search) for the history enriched case are shown in gure Fig. 4 (Top). Each point is an average across 9-folds of the best SVM's hyperparameters. As can be seen, removing a candidate (there are several features per candidate) increases the business score up to a certain value around 0.77 and nally starts to decrease after removing 11 candidates. The fact that the score increases is due to over tting when too many features are taken into account. This also means that not all features are relevant. For example, the voltage is unnecessary and even detrimental to the classi er, as removing this feature already increase the score by 0.13 in comparison with all features taken into account. The remaining features after the selection process are the following: Current, Bearing Temperature and Pyrometer Temperature with the following aggregations: peaks, frequency bins 1 and 5, and standard deviation. This is a count of 144 (3 signals 4 aggregations 12 windows) features out of the initial 960. The grid search of the hyperparameters of the SVM for the nal feature selection is shown in Fig. 4(Bottom).
1 Evolution of business score from feature selection
Voltage
Torque mean
Vacuum bin 2 bin 4 bin 3 skew kurtosis Motor Temperature
Speed std 0.8 e r cso0.6 s s e isn0.4 u B 0.2 0 0 all 10 2 10 3 a m m ag 10 510 1
Candidate6s removed 2 4 8 10 12 Gridsearch results 0.76 0.65 0.54 0.44 0.33 0.22 0.11 100 101 102 103
104
C umn # days shows how many days in advance an alarm is triggered before the system fails. Column Business shows the business score associated with it (as in Fig. 1). Precision and recall scores (based on the 5-days labelling) are also displayed. As we can see, the average business score for the history enriched approach is 0.2 above the simple approach. Adding history (i.e. data from past time windows) is therefore clearly bene cial.
The cross-validated result on each fold for the history enriched case is shown in Fig. 5. For each fold, we plot the decision function from the SVM which gives the distance to the separating hyperplane. Whenever a data point is below zero, it is considered a nominal value and whenever it is strictly above zero, it is considered a critical point. The output of the aggregator is shown as the red solid line on the graph. Whenever this aggregate exceeds 0, we send an alarm. As can be seen in the graph, an alarm is sent for all the run-to-failure occurrences except the fourth graph which represents a preventive maintenance. Looking at the results, we observe that our approach performs quite well. Indeed, in 8 out of 9 cases, the algorithm detects a failure between 2.5 and 8 days in advance (while in the remaining case it is detected 19 days in advance, which is slightly too early for our business metric). The algorithm is therefore capable of distinguishing abnormal behaviour and sending a preventive alarm several days ahead of failures. 5
We have shown that it is possible to mostly predict the failure of the rotating condenser in a synchrocyclotron several days in advance, using a purely datadriven approach based on the sampled signals that were provided to us. Indeed, we observed a relatively low false alarm rate, as no false alarm was raised in 8 cases out of 9, and the only case for which the alarm was triggered too early was at roughly 70% of the machine's life, which remains acceptable. We achieved this goal thanks to an original two-level approach based on the combination of a conventional binary classi er (SVM) and an aggregator, optimized with respect to a custom business metric.
Comparing the performance of our approach with a one-step approach based on a multi-classi er SVM is left for further research. The use of transfer learning techniques to apply the learned model to other but similar machines could also be investigated.
3/10 5/01 8/15 9/15 10/01 10/15 10/22 10/25 10/28 11/01 11/04 11/07 11/10 Fig. 5: Results per fold: Green points are point classi ed as nominal, Orange points are critical points, red solid line is the rolling average of the SVM output for eight hours, vertical dashed red line mark 5 days before failure and the vertical dashed orange line mark the alarm sent by the decision maker. On the x-axis we have the time and on the y-axis, the decision function value of the SVM.