=Paper=
{{Paper
|id=Vol-3665/short2
|storemode=property
|title=Methods for Predicting Failures in a Smart Home
|pdfUrl=https://ceur-ws.org/Vol-3665/short2.pdf
|volume=Vol-3665
|authors=Viktoriia Zhebka,Pavlo Skladannyi,Yurii Bazak,Andrii Bondarchuk,Kamila Storchak
|dblpUrl=https://dblp.org/rec/conf/decat/ZhebkaSBBS24
}}
==Methods for Predicting Failures in a Smart Home==
https://ceur-ws.org/Vol-3665/short2.pdf
Methods for Predicting Failures in a Smart Home
Viktoriia Zhebka1, Pavlo Skladannyi2, Yurii Bazak1, Andrii Bondarchuk1,
and Kamila Storchak1
1 State University of Information and Communication Technologies, 7 Solomenskaya str., Kyiv, 03110, Ukraine
2 Borys Grinchenko Kyiv University, 18/2 Bulvarno-Kudriavska str., Kyiv, 04053, Ukraine
Abstract
Methods for predicting possible failures in smart home systems and analyzing the data
required for this have been considered in the study. A study of machine learning methods
has been carried out: their features, advantages, and disadvantages have been identified,
the metrics of each method have been studied, and the effectiveness of methods for
predicting failures in a smart home has been established. It has been found that the Long
Short-Term Memory (LSTM) model is distinguished by its ability to work with data
sequences and store information for a long time. The characteristics of the LSTM method
and its algorithm have been studied in detail. The study emphasizes the importance of
collecting and processing various data, such as sensor data, energy consumption, and
information about devices and users. The results of the study can be useful for the further
development of smart home control systems to improve their reliability and efficiency.
Keywords 1
Long short-term memory, LSTM, Machine learning, data processing, forecasting, smart
home, failure, information technology.
1. Introduction predicting possible problems and taking
measures to prevent them before they occur.
Smart houses are becoming increasingly Today, smart home failure prediction is
common thanks to the development of the mostly based on reactive data analysis. This
Internet of Things (IoT) and smart technologies means that systems detect anomalous
[1]. They provide automation and ease of situations or failures after they occur, which can
control of various systems such as lighting, make it difficult to avoid potential problems.
heating, security, energy efficiency, and many However, using machine learning methods,
others [2, 3]. such as classification, clustering, and prediction
However, as the complexity of these systems algorithms, it is possible to develop systems that
grows, the likelihood of failures or problems can predict failures in a smart home in advance.
increases. Network instability, software errors, Such systems use data analysis from sensors,
and faulty devices can all lead to unpredictable IoT devices, control systems, energy
situations that affect the usability and security consumption, and other data to identify
of a smart home [4]. patterns and anomalies that may precede
Therefore, predicting failures in a smart disruptions [5, 6]. Based on this information,
home is a relevant issue, and it is appropriate to machine learning systems can build predictive
conduct such a prediction using machine models that respond to certain signals or
learning methods. The use of machine learning changes in normal operation, warning of
algorithms allows for analyzing large amounts potential problems or taking steps to prevent
of data; and identifying deviations and patterns them [7, 8].
that precede failures. This approach allows This area of research is still evolving, but it
promises to improve smart home control
DECaT’2024: Digital Economy Concepts and Technologies, April 4, 2024, Kyiv, Ukraine
EMAIL: viktoria_zhebka@ukr.net (V. Zhebka); p.skladannyi@kubg.edu.ua (P. Skladannyi); jura.bazak@gmail.com (Y. Bazak);
dekan.it@ukr.net (A. Bondarchuk); kpstorchak@ukr.net (K. Storchak)
ORCID: 0000-0003-4051-1190 (V. Zhebka); 0000-0002-7775-6039 (P. Skladannyi); 0009-0000-6098-2809 (Y. Bazak); 0000-0001-5124-
5102 (A. Bondarchuk); 0000-0001-9295-4685 (K. Storchak)
©️ 2024 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
70
systems by enabling them to predict and x − min
prevent possible failures in advance, providing xn = (1)
max − min .
greater reliability and security for users.
x−
xn = (2)
2. Research Results
2 x 2 x
xsin = sin , xcos = cos (3)
Machine learning techniques can help detect N N
and avoid some disruptions before they occur where х is the feature value, min is the
or restore system operations faster after a minimum value of the feature, max is the
failure. They can provide early detection of maximum value of the feature, µ is the average
anomalies in performance, which will prevent value of the feature, σ is the standard deviation
problems from occurring or respond quickly to of a feature, and N is the total number of
them, which in turn will help reduce the impact different values of the features.
of these failures on the smart home. Some of the algorithms may have problems
Machine learning algorithms are compared with dimensionality and run much slower than
on four different data representations: original, other algorithms. The problem can be solved
balanced, normalized, and standardized. The by reducing the number of dimensions using
original data is unchanged from the selected principal component analysis. The method is
data except for the removal of timestamp only applicable to specific algorithms and
values. Balancing is performed by under- specific data representations, depending on
sampling the faultless data in the training data. the speed of learning and prediction. In
Failure-free data inputs are randomly selected addition, some of the algorithms work only
and removed from the dataset, resulting in the with a certain input format.
same number of failures and no failures. Many machine learning algorithms can be
Insufficient sampling can erase important used to predict device failures. They can differ
information from the data, leading to poor in many properties and features [9]. They can
algorithm performance. The main advantage of be supervised or unsupervised, they can solve
data balancing is that it reduces the resources classification, regression, or clustering
and time required to train algorithms. Data problems or they can belong to different
normalization refers to the scaling of feature families such as deep learning, tree-based,
values in the range from zero to one. Scaling is probabilistic, or linear. The total number of
performed using (1) separately for each feature algorithms considered for comparison was
by finding its minimum and maximum values. limited due to their lengthy setup and training.
The data is also standardized by considering The algorithms were selected based on several
each feature separately. The feature values are criteria:
subtracted from the mean and then divided by 1. Supervised learning: based on the
the standard deviation, as shown in (2). These assumption that the data to be used have
results in values centered around zero with unit been chosen.
dispersion. An additional pre-processing step 2. Practical use: some of the algorithms are
that is performed before normalizing and used more than others for predictive
standardizing the data is the conversion of the maintenance.
time features of the day of the week and the 3. Diversity: algorithms were chosen to
hour. To indicate that the difference between represent different families, tasks, and
the hours 23 and 0 is the same as the difference functions, such as online learning or
between 22 and 23, the values are converted to prediction over time.
cyclic representations using the Fourier Based on these criteria, ten algorithms have
transform [6]. The transformation calculates been selected, nine of which are implemented
the sine and cosine values for each feature, as as classification algorithms and one as a time
shown in equation (3). Thus, each feature is series regression algorithm (Tables 1 and 2)
replaced by the corresponding sine and cosine [10, 11]. There are representatives of different
feature. The calculation depends on the total types. All algorithms support online learning
number of different feature values N, which is either implicitly or through certain variations.
24 for hours and 7 for days of the week.
71
Table 1
List of machine learning methods and their brief description
English name Description
k-Nearest Neighbor A classification method that determines the class of an object by analyzing its nearest neighbors in the
feature space [12].
Decision Tree A classification or regression algorithm that uses a tree structure to make decisions based on feature
separations [13].
Random Forest Decision tree ensemble, where multiple trees are combined to avoid overlearning and improve accuracy [14].
Extreme Gradient Boosting An ensemble of models that use gradient lift to improve the accuracy of each subsequent model [15].
Naive Bayes A probabilistic method that uses Bayes’ theorem for classification based on the probabilities of features
entering a class [16].
Support Vector Machine An algorithm that determines the optimal boundary of separation between classes using support vectors
[17].
Logistic Regression A classification method that uses a logistic function to determine the probability of an object belonging to a
certain class [12].
Stochastic Gradient Descent An optimization algorithm that uses a gradient to find the minimum of a loss function with a randomly
selected subset of data [16].
Multi-Layer Perceptron A neural network with one or more hidden layers is used for classification and regression based on
weighting coefficients [12].
LSTM A type of recurrent neural network designed to store and use information over a long period to predict
failures or events [16, 17].
Table 2
Comparative characteristics of machine learning methods
Method The principle of operation Application area Advantages Disadvantages
k-Nearest Determining the class of an Detecting anomalies, Easy to implement, no Sensitivity to emissions,
Neighbor object through its nearest predicting failures training required high computational
neighbors costs
Decision Tree Decision-making based on Anomaly detection, failure Ease of interpretation, Tendency to overlearn,
sequential divisions by features classification accommodating conditions instability
Random Forest Tree ensemble to avoid Detecting anomalies, High accuracy, and A large number of
overtraining predicting failures consistency of solutions hyperparameters,
training time
Extreme Gradient Using gradient lift to improve Failure prediction, High accuracy, less A large number of
Boosting accuracy anomaly detection tendency to overlearn hyperparameters,
complexity of
interpretation
Naive Bayes Using Bayes’ theorem for Filtering anomalies, Efficiency for small data, The predictions are not
probabilistic classification detecting failure patterns simplicity of the model flexible enough
Support Vector Determining the optimal Classification of Efficiency in large spaces, Requirements for data
Machine boundary of separation between anomalies, forecasting flexibility preparation, high
classes failures complexity of
customization
Logistic Determining the probability of an Failure classification, Interpretability, ease of Requires a linear
Regression object belonging to a certain class anomaly detection implementation separation surface
Stochastic Using a gradient to optimize the Model training, failure Fast learning, efficient for Requirements for
Gradient Descent loss function analysis big data hyperparameters,
tendency to stutter
Multi-Layer A neural network with one or Pattern recognition, time Ability to solve complex Requires a lot of data for
Perceptron more hidden layers series forecasting problems training, training time
LSTM Recurrent neural network for Time sequence analysis, Ability to recognize High computational costs,
long-term memorization failure prediction dependencies over time the complexity of setup
The main metrics for evaluating different Accuracy represents the percentage of
machine learning algorithms in prediction or correctly classified cases in the total number of
classification tasks allow for an objective cases, which gives a general idea of the
comparison of the effectiveness of these algorithm’s accuracy [11]:
algorithms.
72
Т = (TP + TN) / (FP + FN + TP + TN), (4) method are usually used, since the ROC-AUC
formula itself is an integral.
where TP is true positive (correctly categorized
The completeness (R) and frequency (S) are
positive ones), TN is true negative (correctly
determined using a confusion matrix for binary
categorized negative ones), FP is false positive
classification.
(incorrectly categorized positive ones), and FN is
Indicator S is calculated using formula (6),
false negative (incorrectly categorized negative
and the frequency is calculated using formula (9).
ones).
The precision determines the percentage of S=
FP
(9)
correctly identified positive classes among all FP + TN .
identified positive classes, which is useful for The Confusion Matrix provides detailed
working with uneven classes. information about the real and predicted classes,
P=
TP
(5) which helps to estimate the level of correctness
FP + TP . and errors for each class, which is important
Recall displays the percentage of correctly when analyzing the model.
identified positive classes among all actual The Confusion Matrix helps to evaluate the
positive classes, which is important for performance of a classification model by
identifying important cases that have been visualizing real and predicted values. It is the
missed. basis for calculating various metrics, such as
TP accuracy, sensitivity, specificity, F1 score, and
R= (6) others.
(TP + FN )
. These metrics are crucial for evaluating the
F1-average is a score that uses the harmonic effectiveness of algorithms and choosing the one
mean between accuracy and completeness to that is most suitable for a particular task,
understand how well the model solves the depending on the requirements and needs [18,
classification task. 19].
𝑃𝑅 Table 3 shows the performance of different
𝐹1 = 2 . (7) machine learning methods by the main metrics
𝑃+𝑅
considered.
ROC-AUC measures the area under the ROC
Table 3 and Fig. 1 show the performance of
curve and evaluates the model’s performance
different machine learning methods in terms of
depending on different classification thresholds,
the main metrics such as precision, accuracy,
helping to determine its ability to make correct
classification accuracy, completeness, F1-mean,
predictions.
1
and ROC-AUC. The score of “High,” “Medium”, or
ROC − AUC = R( S ) d(S) “Very High” in the “Performance” column is a
(8)
0 . generalized characterization of the methods’
performance based on these metrics.
To approximate this area, numerical methods
such as the trapezoidal method or Simpson’s
Table 3
Effectiveness of different machine learning methods by key metrics
Classification F1- ROC-
Method Accuracy Completeness Effectiveness
accuracy average AUC
k-Nearest Neighbor 0.85 0.81 0.89 0.85 0.92 High
Decision Tree 0.78 0.82 0.75 0.76 0.85 High
Random Forest 0.81 0.85 0.79 0.80 0.88 High
Extreme Gradient Boosting 0.87 0.88 0.86 0.87 0.94 High
Naive Bayes 0.75 0.79 0.72 0.73 0.82 Average
Support Vector Machine 0.82 0.84 0.80 0.81 0.89 High
Logistic Regression 0.79 0.83 0.77 0.78 0.86 Average
Stochastic Gradient Descent 0.80 0.82 0.79 0.80 0.87 Average
Multi-Layer Perceptron 0.84 0.86 0.82 0.83 0.91 High
LSTM 0.88 0.90 0.87 0.88 0.95 Very high
73
Figure 1: Comparison of machine learning methods
The study has found that the LSTM model is 5. New memory state Ct
distinguished by its ability to work with data 6. New exit ht.
sequences and store information for a long The model itself consists of the following
time. This makes LSTM effective for analyzing formulas:
time series, such as sensor data in a smart 1. Forgetting the previous memory state:
home, where information is usually sequential
in time. . (10)
The LSTM model is capable of storing 2. Defining what will be updated in memory:
information for a long time, allowing it to
effectively understand and analyze a sequence . (11)
of real-time sensor data. By using mechanisms
that ensure that some information is forgotten (12)
.
and others are retained, the LSTM can take into
account long-term dependencies and the 3. Update the memory status:
importance of individual events in time series. C = f *C + it * C t (13)
LSTM can adapt to and learn from different
t t t- 1
.
amounts of data, including large amounts of 4. Update the output value:
data from smart home sensors, which allows for
more accurate predictions of failures. The LSTM . (14)
model can adapt to changing conditions and
detect changes in time series, which allows for h = o * tg (C ).
t t t (15)
predicting failures and anomalies in real-time.
where ft is the forget gate, which decides that
LSTM can process a variety of data types (text,
the previous state should be forgotten, it is the
numbers, sequences, etc.), making it versatile
input gate, which determines how much of the
for use in various forecasting and analysis
new input will be added to the memory state,
scenarios.
𝐶̃𝑡 is a new candidate for the memory state, Ct
That is why it is not surprising that this
is memory status, ot is output gate, determines
algorithm showed the best results for
which output will be next, xt is input on a time
predicting failures in a smart home.
step t, ht−1 is preliminary output on the time
The LSTM model has the following elements
step t−1, W and а b is weights and
at each time step t:
displacements to be taught during the training
1. Input data xt
process.
2. Previous output ht−1
These equations allow the LSTM to
3. Previous memory state Ct−1
determine what information to forget, what to
4. Gates:
keep, and how to use it to generate output
• Forget gate ft
values.
• Input gate it
• Output gate ot.
74
The step-by-step algorithm for training an This is a general description of the LSTM
LSTM model includes the following steps: learning algorithm for predicting failures in a
1. Data preparation: smart home.
• Input: receive a set of data containing time The approach presented in this study takes
series or sequences. advantage of LSTM to predict time series. The
• Data preparation: normalization, and LSTM is implemented using Keras (a high-level
conversion of data format to meet model neural network interface that simplifies the
requirements. process of creating and training artificial neural
2. Building an LSTM architecture: networks; it is a machine learning library that
• Creating a model: using machine learning runs on the Tensorflow, Theano, and Microsoft
libraries to build an LSTM network. Cognitive Toolkit frameworks) as a sequential
• Defining parameters: number of layers, model with two LSTM layers and a dense output
number of neurons in each layer, layer. It receives a sequence of data inputs
activation functions, etc. (normalized feature values without rejections)
3. Data separation: and outputs a sequence of failure values.
• Training and testing sets: splitting data It was decided to use a single data input, as
into training and testing sets to evaluate this significantly reduces the training time and is
model performance. sufficient for the algorithm to recognize failure
4. Model training: patterns. The length of the source sequence
• Model training: fitting LSTM to training determines the duration of the runtime.
data using backpropagation. Prediction performance is tested on three
• Model evaluation: assessing whether the different input sequences: 1 (1 second), 300 (5
model’s predictions match the actual data minutes), and 1800 (30 minutes). As a result,
on the test set. three different LSTM models were built. For
5. Evaluation and improvement of the model: training, we used a dataset with 70% failures,
• Analyzing the results: reviewing the creating a split into training and test sets in the
forecast results and assessing their proportion of 25–75% without mixing. In
accuracy. addition, the data was prepared by creating
• Improving the model: use optimization output sequences for each data record, which
techniques, change hyperparameters, or were then used for training.
modify the architecture to improve Based on the conducted research, a data
results. prediction platform has been developed. Once
6. Testing and forecasting: the system has been successfully integrated
• Failure prediction: applying a trained into the smart home, the implementation
model to new data to predict failures in a process takes place, when the system becomes
smart home. an active part of the home environment.
7. Evaluation of the results: However, this is only the beginning: further
• Performance evaluation: comparing support, optimization, and continuous
forecasts with real data to determine the improvement of the system play a key role in
accuracy and efficiency of the model. ensuring its long-term and efficient operation
8. Maintaining and improving the model: in a smart home, adapting to changing needs
• Continuous learning: collecting new data and conditions (Fig. 2).
and improving the model based on it to
keep forecasts up-to-date.
75
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Energy efficiency Automation Security Comfort Resource Responding to
management forecasting and changes
optimization
System without machine learning Machine learning system
Figure 2: Smart home system performance with and without machine learning methods
3. Discussion
Machine learning helps to avoid certain
problems by analyzing previous data and
recognizing patterns, but it cannot predict
absolutely all possible scenarios, especially if
they arise from certain unpredictable factors
or third-party interventions.
A smart home system that uses machine
learning methods proves to be better than a
system without this technology (as the study
results show, the performance of a smart home
using failure prediction methods gives an
average of 22% better result compared to a
similar system without prediction—Fig. 3).
Machine learning allows the system to adapt to
changes in the environment and user
requirements, respond more quickly to new
conditions, and optimize resource use. This
helps to improve the system’s efficiency in
managing energy, comfort, safety, and user
satisfaction [20].
Machine learning allows the system to
predict and avoid failures, which ensures Figure 3: The process of integrating an
greater reliability and durability of the system. information system into a smart home
This approach also allows for increased
automation, helping the system perform 4. Conclusions
routine tasks without user intervention [21,
22]. Overall, a machine learning system The study results have shown a wide range of
remains the preferred choice due to its ability modern technologies, sensors, and control
to predict, optimize, and adapt to changes, systems used in smart homes. The overview
enabling it to provide more efficient and has shown that existing technologies have the
convenient smart home management [23, 24]. potential to improve convenience, security,
and energy efficiency.
The analysis of available machine learning
methods indicates their potential in predicting
76
and managing risks in smart homes. The communications, Science and
considered models have shown high accuracy Technology (2023) 522–526. doi:
in predicting failures. 10.1109/PICST57299.2022.10238518.
The following areas can be considered for [7] Smart Home Technology: How AI Creates
further development of this work: a Space that is Comfortable for life. URL:
• Improving machine learning methods to https://www.everest.ua/tehnologiya-
increase the accuracy of failure prediction. rozumnogo-budynku-yak-ai-stvoryuye-
• In-depth study of the impact of the prostirkomfortnyj-dlya-zhyttya/
introduction of machine learning systems [8] T. Arsan, Smart Systems: From Design to
on the functioning of a smart home. Implementation of Embedded SMART
Based on the obtained data, it is possible to Systems (2016). doi: 10.1109/HONET.
build a methodology for predicting failures in a 2016.7753420.
smart home, which will be the direction of the [9] V. Zhebka et al., Optimization of Machine
authors’ next research. Learning Method to Improve the
Management Efficiency of
References Heterogeneous Telecommunication
Network, Cybersecurity Providing in
Information and Telecommunication
[1] Z. Hu, et al., Bandwidth Research of
Systems Vol. 3288 (2022) 149–155.
Wireless IoT Switches, in: IEEE 15th
[10] A. Boiko, Modeling of the Automated
International Conference on Advanced
System of Operational Control of the
Trends in Radioelectronics,
Parameters of the “Smart Home” in the
Telecommunications and Computer
Proteus Environment, Technologies and
Engineering (2020). doi:
Design 2(35) (2020).
10.1109/tcset49122.2020.2354922
[11] V. Zhebka, Y. Bazak, K. Storchak,
[2] I. Kuzminykh, et al., Investigation of the
Features of Predicting Failures in a
IoT Device Lifetime with Secure Data
Smart Home based on Machine Learning
Transmission, Internet of Things, Smart
Methods, Telecommunications and
Spaces, and Next Generation Networks
Information Technologies 4(81) (2023)
and Systems, vol. 11660 (2019) 16–27.
4–12.
doi: 10.1007/978-3-030-30859-9_2
[12] I. Ruban, V. Martovytskyi, S. Partyka,
[3] B. Zhurakovskyi, et al., Secured Remote
Classification of Methods for Detecting
Update Protocol in IoT Data Exchange
Anomalies in Information Systems,
System, in: Workshop on Cybersecurity
Systems of Armament and Military
Providing in Information and
Equipment 3(47) (2016) 47.
Telecommunication Systems, vol. 3421
[13] V. Kharchenko, Fundamentals of
(2023) 67–76
Machine Learning, Sumy State
[4] O. Shevchenko, et al., Methods of the
University (2023).
Objects Identification and Recognition
[14] X. Sun, et al., System-Level Hardware
Research in the Networks with the IoT
Failure Prediction using Deep Learning,
Concept Support, in: Cybersecurity
56th Annual Design Automation
Providing in Information and
Conference 20 (2019) 1–6. doi:
Telecommunication Systems, vol. 2923
10.1145/3316781.3317918.
(2021) 277–282.
[15] V. Malinov, et al., Biomining as an
[5] M. Moshenchenko, et. al., Optimization
Effective Mechanism for Utilizing the
Algorithms of Smart City Wireless
Bioenergy Potential of Processing
Sensor Network Control, in:
Enterprises in the Agricultural Sector, in:
Cybersecurity Providing in Information
Cybersecurity Providing in Information
and Telecommunication Systems II, vol.
and Telecommunication Systems Vol.
3188 (2021) 32–42.
3421 (2023) 223–230.
[6] V. Sokolov, et al., Method for Increasing
[16] Y. Bazak, Comparison of Machine
the Various Sources Data Consistency for
Learning Methods for Predicting
IoT Sensors, in: IEEE 9th International
Failures in a Smart Home, Abstracts of
Conference on Problems of Info-
the Report at the Scientific and Practical
77
Conference “Problems of Computer
Engineering,” Collection of abstracts
(2023) 125–127.
[17] K. Kononova, Machine Learning:
Methods and Models: a Textbook for
Bachelors, Masters and Doctors of
Philosophy in Specialty 051
“Economics,” V. N. Karazin Kharkiv
National University (2020).
[18] Y. Bondarenko, Manual for the Study of
the Discipline “Statistical Analysis of
Data,” Lira (2018).
[19] K. Hureeva, O. Kudin, A. Lisnyak, A
Review of Machine Learning Methods in
the Task of forecasting Financial Time
Series, Computer Science and Applied
Mathematics (2) (2018) 18–28.
[20] I. Puleko, A. Yefimenko, Architecture and
Technologies of the Internet of Things: a
Textbookб State University “Zhytomyr
Polytechnic” (2022).
[21] B. Zhurakovskyi, et al., Coding for
Information Systems Security and
Viability, in: Information Technologies
and Security Vol. 2859 (2021) 71–84.
[22] M. Moshenchenko, et al., Optimization
Algorithms of Smart City Wireless
Sensor Network Control, in:
Cybersecurity Providing in Information
and Telecommunication Systems Vol.
3188 (2021) 32–42.
[23] B. Chen, et al., Smart Factory of Industry
4.0: Key Technologies, Application Case,
and Challenges, IEEE Access 6 (2018)
6505–6519. doi: 10.1109/ACCESS.2017.
2783682.
[24] J. Mineraud, et al., A Gap Analysis of
Internet-of-Things Platforms, Computer
Communications 89–90 (2016) 5–16.
doi: 10.1016/j.comcom.2016.03.015.
78