Methods for Predicting Failures in a Smart Home Viktoriia Zhebka1, Pavlo Skladannyi2, Yurii Bazak1, Andrii Bondarchuk1, and Kamila Storchak1 1 State University of Information and Communication Technologies, 7 Solomenskaya str., Kyiv, 03110, Ukraine 2 Borys Grinchenko Kyiv University, 18/2 Bulvarno-Kudriavska str., Kyiv, 04053, Ukraine Abstract Methods for predicting possible failures in smart home systems and analyzing the data required for this have been considered in the study. A study of machine learning methods has been carried out: their features, advantages, and disadvantages have been identified, the metrics of each method have been studied, and the effectiveness of methods for predicting failures in a smart home has been established. It has been found that the Long Short-Term Memory (LSTM) model is distinguished by its ability to work with data sequences and store information for a long time. The characteristics of the LSTM method and its algorithm have been studied in detail. The study emphasizes the importance of collecting and processing various data, such as sensor data, energy consumption, and information about devices and users. The results of the study can be useful for the further development of smart home control systems to improve their reliability and efficiency. Keywords 1 Long short-term memory, LSTM, Machine learning, data processing, forecasting, smart home, failure, information technology. 1. Introduction predicting possible problems and taking measures to prevent them before they occur. Smart houses are becoming increasingly Today, smart home failure prediction is common thanks to the development of the mostly based on reactive data analysis. This Internet of Things (IoT) and smart technologies means that systems detect anomalous [1]. They provide automation and ease of situations or failures after they occur, which can control of various systems such as lighting, make it difficult to avoid potential problems. heating, security, energy efficiency, and many However, using machine learning methods, others [2, 3]. such as classification, clustering, and prediction However, as the complexity of these systems algorithms, it is possible to develop systems that grows, the likelihood of failures or problems can predict failures in a smart home in advance. increases. Network instability, software errors, Such systems use data analysis from sensors, and faulty devices can all lead to unpredictable IoT devices, control systems, energy situations that affect the usability and security consumption, and other data to identify of a smart home [4]. patterns and anomalies that may precede Therefore, predicting failures in a smart disruptions [5, 6]. Based on this information, home is a relevant issue, and it is appropriate to machine learning systems can build predictive conduct such a prediction using machine models that respond to certain signals or learning methods. The use of machine learning changes in normal operation, warning of algorithms allows for analyzing large amounts potential problems or taking steps to prevent of data; and identifying deviations and patterns them [7, 8]. that precede failures. This approach allows This area of research is still evolving, but it promises to improve smart home control DECaT’2024: Digital Economy Concepts and Technologies, April 4, 2024, Kyiv, Ukraine EMAIL: viktoria_zhebka@ukr.net (V. Zhebka); p.skladannyi@kubg.edu.ua (P. Skladannyi); jura.bazak@gmail.com (Y. Bazak); dekan.it@ukr.net (A. Bondarchuk); kpstorchak@ukr.net (K. Storchak) ORCID: 0000-0003-4051-1190 (V. Zhebka); 0000-0002-7775-6039 (P. Skladannyi); 0009-0000-6098-2809 (Y. Bazak); 0000-0001-5124- 5102 (A. Bondarchuk); 0000-0001-9295-4685 (K. Storchak) ©️ 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings 70 systems by enabling them to predict and x − min prevent possible failures in advance, providing xn = (1) max − min . greater reliability and security for users. x− xn = (2) 2. Research Results   2 x   2 x  xsin = sin  , xcos = cos   (3) Machine learning techniques can help detect  N   N  and avoid some disruptions before they occur where х is the feature value, min is the or restore system operations faster after a minimum value of the feature, max is the failure. They can provide early detection of maximum value of the feature, µ is the average anomalies in performance, which will prevent value of the feature, σ is the standard deviation problems from occurring or respond quickly to of a feature, and N is the total number of them, which in turn will help reduce the impact different values of the features. of these failures on the smart home. Some of the algorithms may have problems Machine learning algorithms are compared with dimensionality and run much slower than on four different data representations: original, other algorithms. The problem can be solved balanced, normalized, and standardized. The by reducing the number of dimensions using original data is unchanged from the selected principal component analysis. The method is data except for the removal of timestamp only applicable to specific algorithms and values. Balancing is performed by under- specific data representations, depending on sampling the faultless data in the training data. the speed of learning and prediction. In Failure-free data inputs are randomly selected addition, some of the algorithms work only and removed from the dataset, resulting in the with a certain input format. same number of failures and no failures. Many machine learning algorithms can be Insufficient sampling can erase important used to predict device failures. They can differ information from the data, leading to poor in many properties and features [9]. They can algorithm performance. The main advantage of be supervised or unsupervised, they can solve data balancing is that it reduces the resources classification, regression, or clustering and time required to train algorithms. Data problems or they can belong to different normalization refers to the scaling of feature families such as deep learning, tree-based, values in the range from zero to one. Scaling is probabilistic, or linear. The total number of performed using (1) separately for each feature algorithms considered for comparison was by finding its minimum and maximum values. limited due to their lengthy setup and training. The data is also standardized by considering The algorithms were selected based on several each feature separately. The feature values are criteria: subtracted from the mean and then divided by 1. Supervised learning: based on the the standard deviation, as shown in (2). These assumption that the data to be used have results in values centered around zero with unit been chosen. dispersion. An additional pre-processing step 2. Practical use: some of the algorithms are that is performed before normalizing and used more than others for predictive standardizing the data is the conversion of the maintenance. time features of the day of the week and the 3. Diversity: algorithms were chosen to hour. To indicate that the difference between represent different families, tasks, and the hours 23 and 0 is the same as the difference functions, such as online learning or between 22 and 23, the values are converted to prediction over time. cyclic representations using the Fourier Based on these criteria, ten algorithms have transform [6]. The transformation calculates been selected, nine of which are implemented the sine and cosine values for each feature, as as classification algorithms and one as a time shown in equation (3). Thus, each feature is series regression algorithm (Tables 1 and 2) replaced by the corresponding sine and cosine [10, 11]. There are representatives of different feature. The calculation depends on the total types. All algorithms support online learning number of different feature values N, which is either implicitly or through certain variations. 24 for hours and 7 for days of the week. 71 Table 1 List of machine learning methods and their brief description English name Description k-Nearest Neighbor A classification method that determines the class of an object by analyzing its nearest neighbors in the feature space [12]. Decision Tree A classification or regression algorithm that uses a tree structure to make decisions based on feature separations [13]. Random Forest Decision tree ensemble, where multiple trees are combined to avoid overlearning and improve accuracy [14]. Extreme Gradient Boosting An ensemble of models that use gradient lift to improve the accuracy of each subsequent model [15]. Naive Bayes A probabilistic method that uses Bayes’ theorem for classification based on the probabilities of features entering a class [16]. Support Vector Machine An algorithm that determines the optimal boundary of separation between classes using support vectors [17]. Logistic Regression A classification method that uses a logistic function to determine the probability of an object belonging to a certain class [12]. Stochastic Gradient Descent An optimization algorithm that uses a gradient to find the minimum of a loss function with a randomly selected subset of data [16]. Multi-Layer Perceptron A neural network with one or more hidden layers is used for classification and regression based on weighting coefficients [12]. LSTM A type of recurrent neural network designed to store and use information over a long period to predict failures or events [16, 17]. Table 2 Comparative characteristics of machine learning methods Method The principle of operation Application area Advantages Disadvantages k-Nearest Determining the class of an Detecting anomalies, Easy to implement, no Sensitivity to emissions, Neighbor object through its nearest predicting failures training required high computational neighbors costs Decision Tree Decision-making based on Anomaly detection, failure Ease of interpretation, Tendency to overlearn, sequential divisions by features classification accommodating conditions instability Random Forest Tree ensemble to avoid Detecting anomalies, High accuracy, and A large number of overtraining predicting failures consistency of solutions hyperparameters, training time Extreme Gradient Using gradient lift to improve Failure prediction, High accuracy, less A large number of Boosting accuracy anomaly detection tendency to overlearn hyperparameters, complexity of interpretation Naive Bayes Using Bayes’ theorem for Filtering anomalies, Efficiency for small data, The predictions are not probabilistic classification detecting failure patterns simplicity of the model flexible enough Support Vector Determining the optimal Classification of Efficiency in large spaces, Requirements for data Machine boundary of separation between anomalies, forecasting flexibility preparation, high classes failures complexity of customization Logistic Determining the probability of an Failure classification, Interpretability, ease of Requires a linear Regression object belonging to a certain class anomaly detection implementation separation surface Stochastic Using a gradient to optimize the Model training, failure Fast learning, efficient for Requirements for Gradient Descent loss function analysis big data hyperparameters, tendency to stutter Multi-Layer A neural network with one or Pattern recognition, time Ability to solve complex Requires a lot of data for Perceptron more hidden layers series forecasting problems training, training time LSTM Recurrent neural network for Time sequence analysis, Ability to recognize High computational costs, long-term memorization failure prediction dependencies over time the complexity of setup The main metrics for evaluating different Accuracy represents the percentage of machine learning algorithms in prediction or correctly classified cases in the total number of classification tasks allow for an objective cases, which gives a general idea of the comparison of the effectiveness of these algorithm’s accuracy [11]: algorithms. 72 Т = (TP + TN) / (FP + FN + TP + TN), (4) method are usually used, since the ROC-AUC formula itself is an integral. where TP is true positive (correctly categorized The completeness (R) and frequency (S) are positive ones), TN is true negative (correctly determined using a confusion matrix for binary categorized negative ones), FP is false positive classification. (incorrectly categorized positive ones), and FN is Indicator S is calculated using formula (6), false negative (incorrectly categorized negative and the frequency is calculated using formula (9). ones). The precision determines the percentage of S= FP (9) correctly identified positive classes among all FP + TN . identified positive classes, which is useful for The Confusion Matrix provides detailed working with uneven classes. information about the real and predicted classes, P= TP (5) which helps to estimate the level of correctness FP + TP . and errors for each class, which is important Recall displays the percentage of correctly when analyzing the model. identified positive classes among all actual The Confusion Matrix helps to evaluate the positive classes, which is important for performance of a classification model by identifying important cases that have been visualizing real and predicted values. It is the missed. basis for calculating various metrics, such as TP accuracy, sensitivity, specificity, F1 score, and R= (6) others. (TP + FN ) . These metrics are crucial for evaluating the F1-average is a score that uses the harmonic effectiveness of algorithms and choosing the one mean between accuracy and completeness to that is most suitable for a particular task, understand how well the model solves the depending on the requirements and needs [18, classification task. 19]. 𝑃𝑅 Table 3 shows the performance of different 𝐹1 = 2 . (7) machine learning methods by the main metrics 𝑃+𝑅 considered. ROC-AUC measures the area under the ROC Table 3 and Fig. 1 show the performance of curve and evaluates the model’s performance different machine learning methods in terms of depending on different classification thresholds, the main metrics such as precision, accuracy, helping to determine its ability to make correct classification accuracy, completeness, F1-mean, predictions. 1 and ROC-AUC. The score of “High,” “Medium”, or ROC − AUC =  R( S ) d(S) “Very High” in the “Performance” column is a (8) 0 . generalized characterization of the methods’ performance based on these metrics. To approximate this area, numerical methods such as the trapezoidal method or Simpson’s Table 3 Effectiveness of different machine learning methods by key metrics Classification F1- ROC- Method Accuracy Completeness Effectiveness accuracy average AUC k-Nearest Neighbor 0.85 0.81 0.89 0.85 0.92 High Decision Tree 0.78 0.82 0.75 0.76 0.85 High Random Forest 0.81 0.85 0.79 0.80 0.88 High Extreme Gradient Boosting 0.87 0.88 0.86 0.87 0.94 High Naive Bayes 0.75 0.79 0.72 0.73 0.82 Average Support Vector Machine 0.82 0.84 0.80 0.81 0.89 High Logistic Regression 0.79 0.83 0.77 0.78 0.86 Average Stochastic Gradient Descent 0.80 0.82 0.79 0.80 0.87 Average Multi-Layer Perceptron 0.84 0.86 0.82 0.83 0.91 High LSTM 0.88 0.90 0.87 0.88 0.95 Very high 73 Figure 1: Comparison of machine learning methods The study has found that the LSTM model is 5. New memory state Ct distinguished by its ability to work with data 6. New exit ht. sequences and store information for a long The model itself consists of the following time. This makes LSTM effective for analyzing formulas: time series, such as sensor data in a smart 1. Forgetting the previous memory state: home, where information is usually sequential in time. . (10) The LSTM model is capable of storing 2. Defining what will be updated in memory: information for a long time, allowing it to effectively understand and analyze a sequence . (11) of real-time sensor data. By using mechanisms that ensure that some information is forgotten (12) . and others are retained, the LSTM can take into account long-term dependencies and the 3. Update the memory status: importance of individual events in time series. C = f *C + it * C t (13) LSTM can adapt to and learn from different t t t- 1 . amounts of data, including large amounts of 4. Update the output value: data from smart home sensors, which allows for more accurate predictions of failures. The LSTM . (14) model can adapt to changing conditions and detect changes in time series, which allows for h = o * tg (C ). t t t (15) predicting failures and anomalies in real-time. where ft is the forget gate, which decides that LSTM can process a variety of data types (text, the previous state should be forgotten, it is the numbers, sequences, etc.), making it versatile input gate, which determines how much of the for use in various forecasting and analysis new input will be added to the memory state, scenarios. 𝐶̃𝑡 is a new candidate for the memory state, Ct That is why it is not surprising that this is memory status, ot is output gate, determines algorithm showed the best results for which output will be next, xt is input on a time predicting failures in a smart home. step t, ht−1 is preliminary output on the time The LSTM model has the following elements step t−1, W and а b is weights and at each time step t: displacements to be taught during the training 1. Input data xt process. 2. Previous output ht−1 These equations allow the LSTM to 3. Previous memory state Ct−1 determine what information to forget, what to 4. Gates: keep, and how to use it to generate output • Forget gate ft values. • Input gate it • Output gate ot. 74 The step-by-step algorithm for training an This is a general description of the LSTM LSTM model includes the following steps: learning algorithm for predicting failures in a 1. Data preparation: smart home. • Input: receive a set of data containing time The approach presented in this study takes series or sequences. advantage of LSTM to predict time series. The • Data preparation: normalization, and LSTM is implemented using Keras (a high-level conversion of data format to meet model neural network interface that simplifies the requirements. process of creating and training artificial neural 2. Building an LSTM architecture: networks; it is a machine learning library that • Creating a model: using machine learning runs on the Tensorflow, Theano, and Microsoft libraries to build an LSTM network. Cognitive Toolkit frameworks) as a sequential • Defining parameters: number of layers, model with two LSTM layers and a dense output number of neurons in each layer, layer. It receives a sequence of data inputs activation functions, etc. (normalized feature values without rejections) 3. Data separation: and outputs a sequence of failure values. • Training and testing sets: splitting data It was decided to use a single data input, as into training and testing sets to evaluate this significantly reduces the training time and is model performance. sufficient for the algorithm to recognize failure 4. Model training: patterns. The length of the source sequence • Model training: fitting LSTM to training determines the duration of the runtime. data using backpropagation. Prediction performance is tested on three • Model evaluation: assessing whether the different input sequences: 1 (1 second), 300 (5 model’s predictions match the actual data minutes), and 1800 (30 minutes). As a result, on the test set. three different LSTM models were built. For 5. Evaluation and improvement of the model: training, we used a dataset with 70% failures, • Analyzing the results: reviewing the creating a split into training and test sets in the forecast results and assessing their proportion of 25–75% without mixing. In accuracy. addition, the data was prepared by creating • Improving the model: use optimization output sequences for each data record, which techniques, change hyperparameters, or were then used for training. modify the architecture to improve Based on the conducted research, a data results. prediction platform has been developed. Once 6. Testing and forecasting: the system has been successfully integrated • Failure prediction: applying a trained into the smart home, the implementation model to new data to predict failures in a process takes place, when the system becomes smart home. an active part of the home environment. 7. Evaluation of the results: However, this is only the beginning: further • Performance evaluation: comparing support, optimization, and continuous forecasts with real data to determine the improvement of the system play a key role in accuracy and efficiency of the model. ensuring its long-term and efficient operation 8. Maintaining and improving the model: in a smart home, adapting to changing needs • Continuous learning: collecting new data and conditions (Fig. 2). and improving the model based on it to keep forecasts up-to-date. 75 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Energy efficiency Automation Security Comfort Resource Responding to management forecasting and changes optimization System without machine learning Machine learning system Figure 2: Smart home system performance with and without machine learning methods 3. Discussion Machine learning helps to avoid certain problems by analyzing previous data and recognizing patterns, but it cannot predict absolutely all possible scenarios, especially if they arise from certain unpredictable factors or third-party interventions. A smart home system that uses machine learning methods proves to be better than a system without this technology (as the study results show, the performance of a smart home using failure prediction methods gives an average of 22% better result compared to a similar system without prediction—Fig. 3). Machine learning allows the system to adapt to changes in the environment and user requirements, respond more quickly to new conditions, and optimize resource use. This helps to improve the system’s efficiency in managing energy, comfort, safety, and user satisfaction [20]. Machine learning allows the system to predict and avoid failures, which ensures Figure 3: The process of integrating an greater reliability and durability of the system. information system into a smart home This approach also allows for increased automation, helping the system perform 4. Conclusions routine tasks without user intervention [21, 22]. Overall, a machine learning system The study results have shown a wide range of remains the preferred choice due to its ability modern technologies, sensors, and control to predict, optimize, and adapt to changes, systems used in smart homes. The overview enabling it to provide more efficient and has shown that existing technologies have the convenient smart home management [23, 24]. potential to improve convenience, security, and energy efficiency. The analysis of available machine learning methods indicates their potential in predicting 76 and managing risks in smart homes. The communications, Science and considered models have shown high accuracy Technology (2023) 522–526. doi: in predicting failures. 10.1109/PICST57299.2022.10238518. The following areas can be considered for [7] Smart Home Technology: How AI Creates further development of this work: a Space that is Comfortable for life. URL: • Improving machine learning methods to https://www.everest.ua/tehnologiya- increase the accuracy of failure prediction. rozumnogo-budynku-yak-ai-stvoryuye- • In-depth study of the impact of the prostirkomfortnyj-dlya-zhyttya/ introduction of machine learning systems [8] T. Arsan, Smart Systems: From Design to on the functioning of a smart home. Implementation of Embedded SMART Based on the obtained data, it is possible to Systems (2016). doi: 10.1109/HONET. build a methodology for predicting failures in a 2016.7753420. smart home, which will be the direction of the [9] V. Zhebka et al., Optimization of Machine authors’ next research. Learning Method to Improve the Management Efficiency of References Heterogeneous Telecommunication Network, Cybersecurity Providing in Information and Telecommunication [1] Z. Hu, et al., Bandwidth Research of Systems Vol. 3288 (2022) 149–155. Wireless IoT Switches, in: IEEE 15th [10] A. Boiko, Modeling of the Automated International Conference on Advanced System of Operational Control of the Trends in Radioelectronics, Parameters of the “Smart Home” in the Telecommunications and Computer Proteus Environment, Technologies and Engineering (2020). doi: Design 2(35) (2020). 10.1109/tcset49122.2020.2354922 [11] V. Zhebka, Y. Bazak, K. Storchak, [2] I. Kuzminykh, et al., Investigation of the Features of Predicting Failures in a IoT Device Lifetime with Secure Data Smart Home based on Machine Learning Transmission, Internet of Things, Smart Methods, Telecommunications and Spaces, and Next Generation Networks Information Technologies 4(81) (2023) and Systems, vol. 11660 (2019) 16–27. 4–12. doi: 10.1007/978-3-030-30859-9_2 [12] I. Ruban, V. Martovytskyi, S. Partyka, [3] B. Zhurakovskyi, et al., Secured Remote Classification of Methods for Detecting Update Protocol in IoT Data Exchange Anomalies in Information Systems, System, in: Workshop on Cybersecurity Systems of Armament and Military Providing in Information and Equipment 3(47) (2016) 47. Telecommunication Systems, vol. 3421 [13] V. Kharchenko, Fundamentals of (2023) 67–76 Machine Learning, Sumy State [4] O. Shevchenko, et al., Methods of the University (2023). Objects Identification and Recognition [14] X. Sun, et al., System-Level Hardware Research in the Networks with the IoT Failure Prediction using Deep Learning, Concept Support, in: Cybersecurity 56th Annual Design Automation Providing in Information and Conference 20 (2019) 1–6. doi: Telecommunication Systems, vol. 2923 10.1145/3316781.3317918. (2021) 277–282. [15] V. Malinov, et al., Biomining as an [5] M. Moshenchenko, et. al., Optimization Effective Mechanism for Utilizing the Algorithms of Smart City Wireless Bioenergy Potential of Processing Sensor Network Control, in: Enterprises in the Agricultural Sector, in: Cybersecurity Providing in Information Cybersecurity Providing in Information and Telecommunication Systems II, vol. and Telecommunication Systems Vol. 3188 (2021) 32–42. 3421 (2023) 223–230. [6] V. Sokolov, et al., Method for Increasing [16] Y. Bazak, Comparison of Machine the Various Sources Data Consistency for Learning Methods for Predicting IoT Sensors, in: IEEE 9th International Failures in a Smart Home, Abstracts of Conference on Problems of Info- the Report at the Scientific and Practical 77 Conference “Problems of Computer Engineering,” Collection of abstracts (2023) 125–127. [17] K. Kononova, Machine Learning: Methods and Models: a Textbook for Bachelors, Masters and Doctors of Philosophy in Specialty 051 “Economics,” V. N. Karazin Kharkiv National University (2020). [18] Y. Bondarenko, Manual for the Study of the Discipline “Statistical Analysis of Data,” Lira (2018). [19] K. Hureeva, O. Kudin, A. Lisnyak, A Review of Machine Learning Methods in the Task of forecasting Financial Time Series, Computer Science and Applied Mathematics (2) (2018) 18–28. [20] I. Puleko, A. Yefimenko, Architecture and Technologies of the Internet of Things: a Textbookб State University “Zhytomyr Polytechnic” (2022). [21] B. Zhurakovskyi, et al., Coding for Information Systems Security and Viability, in: Information Technologies and Security Vol. 2859 (2021) 71–84. [22] M. Moshenchenko, et al., Optimization Algorithms of Smart City Wireless Sensor Network Control, in: Cybersecurity Providing in Information and Telecommunication Systems Vol. 3188 (2021) 32–42. [23] B. Chen, et al., Smart Factory of Industry 4.0: Key Technologies, Application Case, and Challenges, IEEE Access 6 (2018) 6505–6519. doi: 10.1109/ACCESS.2017. 2783682. [24] J. Mineraud, et al., A Gap Analysis of Internet-of-Things Platforms, Computer Communications 89–90 (2016) 5–16. doi: 10.1016/j.comcom.2016.03.015. 78