Building robust systems that are able to withstand various kinds of disturbances is an ongoing subject of research in the broader field of intelligent systems. Hereby, physical disturbances, therefore, disturbances that afect a system's hardware, are rarely discussed. We present an XCSF-based approach for identifying physical disturbances. We utilize a concept for limiting XCSF's necessary supervision as well as various system metrics in form of triggers in order to reduce the amount of interaction with human experts. We evaluate our approach using a simple proof of concept, showing that an XCSF with our extensions is able to perform acceptable compared to a naive XCSF. Besides, we discuss possible future research direction for our approach.
The increasing complexity in Information and Communication Technology has led to the
emergence of various research initiatives concerned with building intelligent systems that
incorporate several so-called self-x capabilities, such as Organic Computing (OC) ([
As suggested in [
As an online learner, the XCS Classifier System (XCS) has gained notable attention in previously
mentioned research initiatives. Originally designed as a reinforcement learning algorithm, there
exist various derivatives for many machine learning settings, such as supervised classification
tasks ([
In this paper, we present an XCSF-based approach for learning to identify broken components
in a intelligent system. More concrete, our solution focuses on limiting the amount of human
supervision while maintaining an acceptable identification quality. In order to achieve this,
we adapt the algorithmic internals of XCSF in order to reduce the number of supervision tasks
according to [
The remainder of this paper is structured as follows. At first, we motivate our problem and introduce various relevant research works. Afterwards, our adapted XCSF as well as the Triggers implemented so far are presented. Our methodology is then evaluated using a real-world scenarios, before concluding with an outlook on possible future work.
In the following, we motivate our problem statement using the architectural principles used in OC. However, the concepts presented in this paper can be applied to any other kind of intelligent system as well.
We assume an OC system consisting of an System under Observation and Control (SuOC) as
well as an Multi Level Observer Controller-Instance (MLOC). Intuitively, the SuOC represents
some sort of machinery or complex technical system that is observed and controlled by the
MLOC in order to work in a desired way. Typical examples for a SuOC include a smart factory
lfoor, as used for evaluatory purposes later on, or a smart home environment. Each system
is associated with some sort of utility measure that can be used as a metric to assess the
systems performance. Correspondingly, there may exist several variants of disturbances that
afect . A system is called robust if it manages to adapt to upcoming disturbances such that
remains over some acceptable performance threshold (or, at least, is able to return to an
acceptable shortly after a disturbance). A broader overview on this topic, as well as on the
relations between disturbances and acceptable performance thresholds can be found in [
consists of various components ∈ ||. Hereby, a component acts as an abstraction for any kind of actuator or sensor that is used within the system context. As some sort of machinery or other physical device, each component situated in a real-world scenario will sufer from degradation or other physical disturbances from time to time, hence, disturbances that are not software-sided but situated within the system’s components. We assume that a physical disturbance is not directly observable in terms of beeing noticed by . However, we assume that an occurance of causes either permanent or at least temporary (if it is repaired) damage to the afected components in such ways that is no longer able to perform on a level it was beforehand. Therefore, at least if no redundancy exists to compensate the broken component, afects ’s utility . Furthermore, depending on the type of a component, additional information such as sensor measurements or some sort of local utility measure might exist. Again, these measurements could be afected by a breakdown and therefore might difer to those gained from an undisturbed component.
Finally, we assume that some sort of human expert - e.g. an repair worker or some sort of engineering staf - exists. The human expert has the corresponding expert knowledge to assess if a component is broken or not, hence, to deliver the ground truth to , if necessary.
The overall problem can be summarized as follows: We intend to continuously assess the overall state of in order to identify disturbances, or, in other words, to identify broken components. Hereby, we need to call upon a human expert in order to gain some ground truth on a component’s state from time to time. As we want to reduce the overall costs, we seek to limit the amount of supervision as far as possible. Iteratively, should be able to build a solid knowledge base for deciding if a component is broken, based on actual measurements from within the component or metrics generally available to .
A conceptual term closely related to this work is self-healing, altough it mostly focuses on
software-sided disturbances (e.g. [10]). Few work exists on healing actual hardware-related
failures (e.g. [11], [12]). However, self-healing difers from our work such that we solely focus
on identifying disturbances, not healing (or repairing) them. Another relevant concept in OC
(and other initiative for building intelligent systems as well) is robustness. Robustness describes
the ability of a system to withstand or compensate disturbances while preserving a certain
degree of functionality. An approach to quantify robustness was published in [
As already mentioned in the introduction, our approach uses an adapted XCSF for learning under
limited supervision, as firstly introduced in [
Depending on the application scenario, two types of data sources that can be used to learn from may exist: Actual measurements gathered from ’s components as well as data gathered from the triggers provides. Intuitively, it cannot be determined in advance how measurements may look like. For the scope of this work, we assume that the measurements gathered from the components are real-valued, however, it is also conceivable that more complex data sources exist (e.g. cameras return images). Trigger data, on the other hand, is represented binarily: Either a trigger is active or it is not.
XCSF is an evolutionary online-learner for function approximation. On each discrete timestep,
XCSF is confronted with a situation description (also called sigma in XCS-terminology) ,
calculates an approximation , and is presented with the actual correct prediction . The XCSF
̂︀
implementation used throughout this work is based on the Algorithmic Description by [14]
with XCSF-specific enhancements described in [
XCSF consists of a population [ ], representing XCSF’s knowledge base in form of classifiers .
A classifier contains a condition representing the parts of the problem space the classifier covers,
as well as various parameters. The original XCS described in [14] uses tenary conditions,
that is binary conditions extended with a do not care-operator #. Later implementations use
diferent variants of real-valued conditions (cf. [ 16], [15]. As our problem space consists of both
real-valued data (measurements) as well as binary data (triggers), the classifiers used in our
implementation consist of a real-valued part, as well as a tenary part. Each classifier ∈ [ ]
is associated with a linear approximation ℎ () estimating the prediction of the classifier for a
given input . The weights of ℎ (· ) are regarded as parameters of the classifier. Furthermore, a
classifier contains a prediction error , estimating the error in ℎ and a fitness value . As the
functionality of the linear approximation is of subordinate relevance for this work, we refer
to [
Upon each discrete timestep , XCSF is confronted with an input . XCSF’s [ ] is scanned for classifiers matching . The hyperrectangular part of classifier matches if for each ∈ , < < with beeing the lower boundary and beeing the upper boundary, respectively. The tenary part matches if = or = # with # beeing the previously mentioned do not care-operator.
All classifiers matching are added to the match set [ ]. If no matching classifiers could
be found, a covering mechanism is invoked. XCSF’s original covering mechanism creates a
matching condition by generating boundaries for each in with = − [0, 0) and
= + [0, ). Intuitively, [0, 0) returns a random number between 0 and the maximum
spread 0. The classifiers parameters (e.g. the weights of ℎ(), , ) are set to predefined
values. For now, we do not replace the internal covering mechanism by an external covering as
suggested in [
The covered classifier is added to both [ ] as well as [ ]. Afterwards, ℎ () of each
in [ ] is calculated. The results are averaged weighted with corresponding classifiers fitness
values and returned as XCSF’s prediction . Afterwards, XCSF needs to decide if a supervision
is necessary as explained in [
̂︀ As the update cycle is not relevant for the understanding of this work, we again refer to [14] for a more detailed description.
On some timesteps, XCSF invokes a genetic algorithm (GA) after executing the update
procedure. The GA selects two classifiers from [ ] and generates two ofspring classifiers. With a
certain probability, a crossover operator is applied to the boundaries of the ofspring’s
condition. Also, again with a certain probability, a mutatation operator is used on the boundaries.
Altogether with a subsumption procedure as well as a deletion scheme, the GA is responsible for
some evolutionary pressures that allow XCSF to evolve populations of accurate classifiers ([ 17]).
At this point, we refer to the corresponding literature ([14], [
As another approach for reducing the number of supervision calls, [
As mentioned earlier, the idea of the triggers is to reflect already existing metrics of the surrounding CM as well as to simplify potentially complex measurements. We identified and implemented four triggers of diferent complexity within the scope of this work, which will be explained in the following.
The System-level utility trigger is based on the violation of the acceptance space
boundary of the system according to ([
0, else
Hereby, is the current discrete timestep, Θ the acceptance space boundary and the utility of at timestep .
The Component-level utility trigger is similar to . The functionality follows the system level utility trigger so far. However, for this trigger, we assume that is able to assess each component ’s individual utility as well as to provide an acceptance space boundary Θ.
(, ) = {︃1, if < Θ
0, else
The applicability of the component-level utility trigger might depend on the underlying SuOC: Not every scenario would feature component-wise utility measures.
The external trigger formalizes the idea of a user filing some sort of maintenance/problem report after encountering errorous hardware in his environment: This could be, for example, a blue-collar worker utilizing some sort of machinery during his shift. Again, will not necessarily be present in each conceivable scenario. Furthermore, we could suspect that the external trigger may be associated with a certain degree of uncertainty: The reporting user might just suspects a defect, while actually being not able to use the corresponding machinery in a correct way.
The anomaly-based trigger is by far the most complex trigger. As mentioned before, we suspect that a component afected by a breakdown would also feature diferent measurements or metrics from its internal sensors. The idea of the anomaly-based trigger is to make use of a suspected measurable diference between data gathered during normal component function and data gathered during a breakdown. In order to do so, each time the supervision returns an that ̂︀ indicates a breakdown, the first percent of the corresponding component’s measurements are added to a so-called normal data pool. Afterwards, all measurements in the normal data pool are used to train a One-Class Support Vector Machine (SVM) ([18]). The idea of a One-Class SVM is to learn a decision boundary around already known data. Future data points which do not lie within these boundaries are considered to be outliers or anomalies. Therefore, is activated when the trained SVM considers an input as an outlier. We use the SGDOneClassSVM Implementation from scikit-learn ([19]), as it allows to warm-start the SVM (therefore, to learn in an iterative, online manner).
The overall scenario is structured as follows. Our SuOC consists of 5 components in form of
identical production machines producing some identical parts (e.g. for later processing in another
area of the factory). The number of produced parts per time can be considered as the utility
of the overall system. Accordingly, if a machine fails, the overall utility of the system drops
below the acceptance threshold. Besides, we assume that the MLOC instance encapsulating
the SuOC is able to assess the individual utility measures of each machine (therefore, the
number of parts per time for each machine seperately). Furthermore, each machine delivers
various internal measurements that can be used to assess the machines state. Additionally,
no configuration changes (e.g. a changed system utility boundary due to a higher demand of
parts) or other disturbances except actual machine breakdowns will take place, resulting in a
quite simple evaluation scenario. We used the Azure AI Predictive Maintenance Dataset 1 for
simulating the data measurements from the individual machines. The CSV-files in the dataset
were preprocessed such that a single CSV-file exists for each machine (containing measurements
from installation until breakdown in chronologial order) in the dataset. Besides, incomplete
traces (that is, machines without breakdowns and machines that were repaired although no
breakdown happened) were removed. Furthermore, the error column was removed, therefore,
the only measurements available to XCSF are volt, rotate, pressure and vibration. This results in
a total of 672 machines. The 5 components of the SuOC are equipped with 5 uniformly chosen
machine CSV-files. If a component fails (that is: the CSV reaches its last row), it is replaced by
another CSV. In an iterative manner, XCSF is confronted with the measurements for a machine
as well as the corresponding calculated metrics. Furthermore, we assume that our supervision
does not make mistakes, that is: If XCSF asks for supervision, the received ground truth is
calculated based on the current system state. We evalute three diferent scenarios: 1) A naive
XCSF without the limited supervision improvements, using only the Azure AI-measurements
as a data basis (XCSF-M), 2) A limited XCSF with Caching using only the measurements
(XCSFC-M) and 3) A limited XCSF with Caching using the measurements as well as the trigger data
(XCSF-C-MT). We conducted 30 repetitions with diferent fixed random seeds for each scenario.
The parametrisation of the XCSF itself as well as the limited supervision mostly follows [
The average system error as well as the supervision calls (for XCSF-C-M and XCSF-C-MT) are tested on significance. The shapiro-wilk-test was used in order to assess if we can suspect the data to be normal distributed. If so, a paired t-test was used for comparison, otherwise, a twosided Wilcoxon rank-sum test was used. For all tests, an alpha-level of 0.05 was used. XCSF-M performed significantly better than both XCSF-C-M (Wilcoxon rank-sum test, p-value: 0.99) as well as XCSF-C-MT (Wilcoxon rank-sum test, p-value: 0.99). Also, XCSF-C-M performed significantly better dann XCSF-C-MT (Wilcoxon rank-sum test, p-value: 0.99). XCSF-C-M needed significantly more supervision calls (Wilcoxon rank-sum test, p-value: 0.99). The average system error as well as the needed supervision calls, both averaged over 30 runs are shown in Table 1. Figure 1 shows the average system error for all experiments over the course of the evaluation.
0.023 ± 0.002 0.070 ± 0.018 0.108 ± 0.023
In this work, we proposed a novel approach on how systems from the broader domain of intelligent systems can identify hardware-related disturbances in their system context. We introduced several triggers derived from measurements and existing system metrics and integrated them in a XCSF-variant that is able to learn under limited supervision. We evaluated our approach using a simple proof of concept. The results indicate that further research is necessary in order to evaluate if the proposed methodology is applicable in more complex scenarios.
We currently see three interesting aspects for further research. First of all, a broader evaluation with diferent application scenarios appears useful. Based on the evaluation provided in this work, a quite obvious application scenario would be a more sophisticated smart factory environment, involving diferent kinds of production machinery. In such settings, the metrics used for calculating the triggers might difer, depending on the influence various machines might have on each other. Thereby, the corresponding XCSF-instance additionally needs to learn how the overall system context influences the trigger decisions in order to assess if a trigger can be used as a reliable source for properly identifying a component’s physical state. Besides, any application scenario that involves intelligent systems consisting of sensors and actuators appears feasible. Exemplary scenarios involving more simple components than production machinery are sensor networks ([12]) as well as smart home/environment appliances ([20]). The actual behaviour of disturbances (e.g. sudden breakdown in contrast to slowly degrading components) can difer depending on the application scenario or the type of component. The triggers and measurements used right now only focus on a component’s current state. However, it is conceivable to introduce a few more triggers that can be used to identify several disturbance characteristics in order to enlarge the potentially usable data presented to XCSF.
Another notable idea would be an investigation on how both the measurements as well as the triggers afect the performance of the individual learners. Depending on the scenario, there could be triggers that are not necessary for identifying failures. Intuitively, one would suspect that XCSF would evolve classifiers that feature a do not care-operator for the corresponding trigger. Finally, it must be investigated how various data-related aspects can influence the applicability of the whole approach. It is conceivable that more realistic application scenarios may feature more durable components compared to the evaluation provided in this work, resulting in an even more imbalanced dataset, which might afect XCSF’s disturbance identification capabilities. A brief discussion on imbalanced data in XCS-research can be found in [21]. Also, an application scenario might feature some sort of concept drift or shift ([22]), e.g. through a slightly changed hardware configuration. Due to the limitation of supervision received by XCSF, it is conceivable that such a drift will afect the prediction quality without noticing, as (at least after building a stable classifier population) no ground truth is received anymore. A simple approach to compensate this aspect would be to introduce some sort of durability parameter for classifiers: From time to time, XCSF could call for supervision for classifiers with a specific age in order to validate if their prediction is still correct. [10] J. Schmitt, M. Roth, R. Kiefhaber, F. Kluge, T. Ungerer, Using an automated planner to control an organic middleware, in: 2011 IEEE Fifth International Conference on Self-Adaptive and Self-Organizing Systems, IEEE, 2011, pp. 71–78. [11] E. Maehle, W. Brockmann, K.-E. Grosspietsch, A. El Sayed Auf, B. Jakimovski, S. Krannich, M. Litza, R. Maas, A. Al-Homsy, Application of the organic robot control architecture orca to the six-legged walking robot oscar, in: Organic Computing—A Paradigm Shift for Complex Systems, Springer, 2011, pp. 517–530. [12] M. Jänicke, B. Sick, P. Lukowicz, D. Bannach, Self-adapting multi-sensor systems: A concept for selfimprovement and self-healing techniques, in: 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems Workshops, IEEE, 2014, pp. 128–136. [13] T. P. Carvalho, F. A. Soares, R. Vita, R. d. P. Francisco, J. P. Basto, S. G. Alcalá, A systematic literature review of machine learning methods applied to predictive maintenance, Computers & Industrial Engineering 137 (2019). URL: https://www.sciencedirect.com/science/article/pii/S0360835219304838. doi:https://doi.org/ 10.1016/j.cie.2019.106024. [14] M. V. Butz, S. W. Wilson, An algorithmic description of XCS, Soft Computing 6 (2002) 144–153. [15] H. H. Dam, H. A. Abbass, C. Lokan, Be real! XCS with continuous-valued inputs, in: Proceedings of the 7th Annual Workshop on Genetic and Evolutionary Computation, GECCO ’05, Association for Computing Machinery, New York, NY, USA, 2005, p. 85–87. URL: https://doi.org/10.1145/1102256.1102274. doi:10.1145/ 1102256.1102274. [16] S. W. Wilson, Get real! XCS with continuous-valued inputs, in: International Workshop on Learning Classifier
Systems, Springer, 1999, pp. 209–219. [17] M. Butz, T. Kovacs, P. Lanzi, S. Wilson, Toward a theory of generalization and learning in XCS, IEEE
Transactions on Evolutionary Computation 8 (2004) 28–46. doi:10.1109/TEVC.2003.818194. [18] B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, R. C. Williamson, Estimating the support of a highdimensional distribution, Neural computation 13 (2001) 1443–1471. [19] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Research 12 (2011) 2825–2830. [20] F. Allerding, H. Schmeck, Organic smart home: Architecture for energy management in intelligent buildings, in: Proceedings of the 2011 Workshop on Organic Computing, OC ’11, Association for Computing Machinery, New York, NY, USA, 2011, p. 67–76. URL: https://doi.org/10.1145/1998642.1998654. doi:10.1145/1998642. 1998654. [21] M. Nakata, W. Browne, T. Hamagami, K. Takadama, Theoretical XCS parameter settings of learning accurate classifiers, in: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’17, 2017, pp. 473–480. [22] S. Wang, S. Schlobach, M. Klein, What is concept drift and how to measure it?, in: International Conference on Knowledge Engineering and Knowledge Management, Springer, 2010, pp. 241–256.