In the past decade, the electrical energy consumption has experienced a rapid increase, contributing to economical problems and exacerbating environmental problems. An efective solution is energy management using energy management systems in which electrical energy consumption prediction plays a crucial role. In this paper, we present a deep learning based approach for short-term electrical energy consumption prediction. We start by introducing our proposed architecture that provides a structured framework for electrical energy prediction process. Then, we present our prediction models based on LSTM and GRU. Finally, we present an experimental study in which performance of the used models are evaluated using ”Electrical consumption profile for households in Romania ECPHR” dataset.
In the past decade, the electrical energy consumption (EEC) has experienced a rapid increase. The International Energy Agency recorded the largest increase in global electricity consumption in 2021, with a rise of 1,500 TWh, or 6%, attributed to the strong economic recovery following the Covid-19 pandemic recession in 2020. This increase represents the strongest growth since 2010 [1].
In Algeria, electricity represents the second most consumed energy source with a share of 28.3%, compared to 38.4% for natural gas [2]. Electricity demand, especially in the household sector, plays a significant role, accounting for 40% of total consumption, and this demand continues to grow [ 3]. A historic power consumption record was registered on August 14, 2022, as reported by [4], equivalent to a demand of 16,822 MW on the national grid. This record was attributed to the high usage of air conditioning units by citizens seeking relief during heatwave seasons.
However, excessive electricity consumption contributes to the depletion of fossil fuels and raises issues of energy demand management, leading to increased costs and energy tarifs, contributing to greenhouse gas emissions and exacerbating problems related to climate change [5]. Satisfying consumers’ energy needs while managing consumption peaks and minimising environmental impact represents one of the main challenges of the 21st century. Consequently, optimising EEC became highly crucial.
An efective solution is to use Energy Management Systems (EMS). These systems play a crucial role in optimising EEC by providing tools, insights, and strategies to eficiently manage and control energy usage. In such systems, predicting EEC plays a crucial role [6]. It allows predicting future electricity needs, adjusting production accordingly, and finding more sustainable solutions to meet the growing demand. Predictions have become indispensable for maintaining the balance between electricity production and consumption, especially with industrial evolution and growing demands [7].
CEUR
ceur-ws.org
Deep learning (DL) can be used to predict electricity consumption in various ways. Commonly used techniques include artificial neural networks, time series algorithms, and regression models. They can analyse historical EEC data, along with other relevant factors such as weather, holidays, economic trends, etc., to predict future electricity demand. Electricity providers can use these predictions to plan and adjust production and distribution more efectively.
In this paper, we present a DL based approach for EEC prediction. The proposed approach leverages the power of DL models to predict short-term EEC of a residential building (House). The main contributions of this research work can be summarised as follows: • Designing an architecture that provides a structured framework that organises the energy prediction process. • Developing a DL based technique based on two model, namely LSTM (Long Short-Term Memory) and Gated Recurrent Units (GRU).
The rest of the paper is organised as follows. Section 2 discusses some recent works on EEC prediction. Section 3 presents the details of the proposed approach and the used DL models. Section 4 evaluates the proposed approach, while Section 5 discusses the result and performances of the used DL models. Finally, Section 6 concludes the paper and summarises the future works.
Accurately predicting household EEC has proven to be a challenging task [8],[9], particularly given the dynamic nature of energy usage patterns. However, this topic has garnered significant attention from researchers, driven in part by the potential of DL techniques to capture intricate relationships within the data. This has led to the exploration of various prediction models that leverage DL techniques. In this section, we provide a concise yet comprehensive overview of noteworthy contributions in this dynamic domain. Early attempts at predicting household electricity load involved the utilisation of machine learning (ML) models [10]. However, recent advancements have demonstrated that DL models ofer notably improved results compared to traditional ML algorithms. For instance, [ 11] introduced an innovative scalable system named HousEEC, based on a Deep Recurrent Neural Network (DRNN). This system aims to predict the household EEC for the upcoming day. To evaluate the performance, the authors employed a dataset from the Pecan Street region, encompassing nearly four years of EEC data from multiple households. However, it was observed that the model’s efectiveness was confined to datasets with similar weather and economic conditions. In another study, [12] focused on predicting individual household electricity load using an LSTM-RNN model. The authors utilised a four-year dataset from the New South Wales region of Australia, collected through the Smart Grid Smart City (SGSC) project. By comparing the LSTM model’s performance with ELM, BPNN, and KNN models, they showcased a substantial reduction in prediction error through the LSTM architecture. Demonstrating the efectiveness of LSTM models for EEC prediction, the authors of [ 13] evaluated their model’s performance on real EEC data. The results underscored the superiority of their prediction model over classical models, with the added benefit of enhanced performance owing to the authors’ data preparation methodology. To enhance prediction accuracy, several studies have proposed hybrid models that combine the strengths of diferent algorithms [ 14]. The authors of [15] presented an approach that combines Convolutional Neural Network (CNN) and GRU models. This hybrid method was assessed on IHEPC and AEP datasets, showcasing its superiority over other ML and DL models, particularly in scenarios involving temperature, humidity and electricity consumption data. GRU was also combined with LSTM in [16]. The authors of this work utilised a stacking method based on LSTM and GRU for EEC prediction of residential buildings. The used energy data collected from buildings located in Seoul. The experimental results demonstrated that the performance of the proposed combination in term of prediction accuracy. Another combination that fuses CNN and LSTM models have emerged as prevalent strategies for time series prediction. In [17], a comparative study demonstrated the superiority of their hybrid CNN-LSTM model over various ML and DL counterparts in predictive capabilities. Similarly, in [18], the proposal of a hybrid CNN-multi-layer bidirectional LSTM (M-BLSTM) model showcased its predictive prowess, outperforming models like BLSTM, LSTM and CNN-LSTM, particularly in short-term EEC prediction. Leveraging CNN’s spatial feature extraction and LSTM’s temporal dependency modeling, [9] introduced a model that exhibited heightened accuracy compared to conventional LSTM, BPNN, KNN, and ELM models. Moreover, [19] introduced an innovative Longand Short-Term Time-Series Network (LSTNet) approach, capitalizing on CNN and LSTM to identify local connections and periodic patterns alongside an autoregressive model. This approach significantly improved accuracy for short-term load prediction compared to ARIMA, CNN-LSTM and IPSO-LSTM models. These diverse contributions collectively emphasise the significant potential of specific predictive models, particularly the widely adopted and high-performing GRU and LSTM architectures, in achieving accurate predictions of residential electricity load. Notably, the combination of these two models with CNN has been leveraged to great efect, yielding good results in the domain of energy prediction.
In this section, we present our proposed approach. We start by describing the proposed architecture for EEC prediction. Then, we give an overview on the selected DL models namely GRU and LSTM.
EEC prediction involves various stages such as data collection and preprocessing, feature extraction, model training, and prediction. A clear architecture outlines how data flows between these stages and ensures seamless integration. In this work, we elaborated a system architecture that provides a structured framework for organising diferent steps of the prediction process.
As shown in figure 1, the prediction system is structured into four distinct layers, and the functioning and role of each layer are detailed as follows:
- Data Collection Layer: this layer ensures the collection of energy data from diferent houses through smart meters and transmits it to the next layer. This layer plays an essential role in the initial collection of energy data at the local level.
- Communication Layer: this is gateway that transmit the data collected by smart meters from the data collection layer to the cloud layer via the internet, using specific technologies such as optical fiber or cellular networks.
- Cloud Layer: this layer plays an important role in our system by providing several services: • Data Storage: the large amount of energy data from diferent houses requires significant storage capacity. This is provided by the cloud layer, which ofers suitable support for fast and secure access. • Data Processing: The data processing phase involves converting raw energy data into a structured and usable format, creating a dataset suitable for training DL models. This process includes steps such as data cleaning, normalisation, feature extraction, and aggregation. By transforming raw energy data into an organised and standardised dataset, it becomes possible to feed DL models for training and prediction. The quality of the dataset greatly influences the performance and accuracy of trained models, making data processing a critical step for EEC prediction. • Data Analysis and Prediction: involves using data from the preprocessing phase to train the
DL model to predict future consumption based on users’ consumption history.
- Application Layer: this layer in energy prediction serves as a crucial interface between the predictive models and end-users, providing various functionalities and features that enhance the usability, accessibility and decision-making capabilities of the system.
This section presents an overview of the selected models for EEC prediction. After conducting a comprehensive literature review of recent advances in EEC prediction, two models have demonstrated notable performances which are LSTM and GRU.
- LSTM: introduced by [20], is a specialised type of RNN architecture designed to process, analyse, and predict sequences of data [14], particularly useful for tasks involving time-dependent information, such as EEC prediction. LSTMs take on the challenge of understanding patterns in sequences that span long periods of time, something that standard RNNs often struggle to do due to the vanishing gradient problem, where information fades as it propagates through the network [21]. The distinctive advantage of LSTMs lies in their ability to capture and retain long-term dependencies within sequences. This means they can efectively understand patterns and relationships that extend over long-time intervals, a crucial aspect when dealing with data such as EEC and production, which often exhibit trends that are not immediately obvious over short periods of time. In the context of energy prediction, where it is essential to understand past patterns and predict future trends, LSTMs excel due to their ability to process sequences of varying lengths and automatically learn relevant features from the data. Their ability to store past information over long periods and selectively focus on important details makes them a solid choice for accurately predicting EEC and production patterns over time [12].
- GRU: introduced by [22], is a type of RNN architecture similar to LSTM widely used to solve sequence-related problems. Emerging as a simplified version of LSTM, GRUs retain the ability to capture long-term dependencies in sequential data while having a simpler structure. One of the advantages of GRU lies in its simplicity. With fewer parameters and computations, GRU is easier to learn and require less memory compared to LSTM. This eficiency can be valuable when dealing with large datasets as the training time can be reduced. In addition, GRU is less likely to overfit, which can be useful when dealing with small datasets. GRUs are excellent at capturing short-term dependencies in sequences, making them a suitable choice when recent context is more important than distant context. Although their architecture is less complex than that of LSTMs, GRUs still have the ability to manage information flow and memory in a controlled manner, which facilitates eficient sequence modelling. Thus, GRU provide an optimised alternative to LSTM for sequence modelling tasks. Its simpler structure, eficient learning and ability to handle short-term dependencies make it a valuable tool in various applications such EEC prediction.
In this section, we present the experimental study conducted in order to evaluate the performances of the selected models. First, we present the dataset selected. Then, we present our models architectures. We finish with the training configuration for our models.
In this work, we used the dataset ”Electrical consumption profile for households in Romania ECPHR” [23] to train and evaluate the performance of the selected models. This dataset describes the hourly EEC of a family of two adults and one child living in a house with an area of (100 - 150 square meters) located in Romania. The collected data spans over a year from December 31, 2016, to December 31, 2017. As shown in figure 4, this dataset contains historical data of the house EEC and the detailed consumption of the house appliances, lightening, and water heater. It also provides information on residents’ habits and activities over the course of a week.
Before proceeding with data pre-processing and models training, we conducted an exploratory study of the dataset using visualisation. This later, plays a crucial role in both gaining a better understanding of the dataset and evaluating its quality. By visually representing the distribution of data of a week (Figure 2) and a month (Figure 3), we can clearly see the consistent trends in the consumption patterns of the residents, indicating the predictable nature of the data.
In order to prepare the original dataset (Figure 4) for training the three models, a pre-processing step was necessary. This step began by identifying missing values, which were then replaced with the mean of the other values using the ’fillna()’ function. Similarly, the same operation was applied to outliers, replacing them with the mean of non-outlier values. Next, a new column called ’day of week’ was introduced, indicating whether each row corresponds to a weekday or a weekend. This column captures potential variations in EEC behaviour based on the days of the week.
An additional column named ’activities’ was also added, specifying the activity associated with each hour of the day. The possible values for this column are: ’Sleeping’ (when all family members are sleeping), ’Inhouse activity’ (when energy-consuming activities occur within the house), and ’Unoccupied’ (when the house is unoccupied). To represent these values with numerical data, the ’OrdinalEncoder()’ function was used. Subsequently, the dataset was normalized using the ’MinMaxScaler’ function to bring all values to the same scale, which facilitates model learning. Figure 5 presents the dataset with the new added columns.
Before making predictions, time series problems need to be reformulated as supervised learning problems, from a sequence to pairs of input and output sequences. This was achieved using the ’series to supervised’ function, which adds columns containing lagged values of variables for past and future time steps. Figure 6 presents the dataset after applying the time series to supervised transformation.
Subsequently, the ’train_test_split’ function was used to divide the dataset into training, testing, and validation sets. Finally, the ’reshape’ function was used to resize the data and adapt it to the input shape required by the model.
In this section, we present the models architectures designed for EEC prediction. We present the diferent experimented combination before elaborating the final models architectures.
In order to design the architecture of the GRU model that gives best energy prediction, we tested diferent configurations. We started with an initial architecture, that is composed of five layers: Input layer, GRU layer, dropout layer, and two fully-connected dense layers. To determine the number of neurons (units) for the GRU layer, we tested various configurations: 32 units, 40 units, and 64 units, where the last one gave a better performance. Therefore, the initial version of the architecture starts with an input layer that considers the dimensions of the input data. Then, a GRU layer with 64 units is integrated to capture temporal dependencies in the data. To regularise the model and prevent overfitting, a dropout layer is introduced with a rate of 20%. In order to enable the model to capture nonlinear relationships in the data, a dense layer with 16 units is added. A final dense layer with a single output unit is appended to the model to predict the output value.
In order to further enhance the model’s performance, we decided to incorporate an additional GRU layer. Using the same technique, we tested various configurations: 64 units for the first GRU layer and 32, 40, and 64 units for the second GRU layer. The last configuration demonstrated the best performance. Consequently, the second version of the GRU architecture is composed of an input layer, followed by two 64 units GRU layers and a hyperbolic tangent (tanh) as an activation function. After each GRU layer, a dropout layer with a rate of 20% is added. A final dense layer with a single unit is appended to predict the output value.
To further enhance the model’s performance, we opted to integrate a third GRU layer. Using the same technique, we tested various configurations: 64 units for the first and the second GRU layer and 32, 40, and 64 units for the third GRU layer. The last configuration stood out in terms of performance. As a result, our final architecture of the GRU model consists of an input layer followed by three 64 neural units GRU layers, and utilising a hyperbolic tangent (tanh) as an activation function. The first two GRU layers return a sequence of values rather than a single value, while the last GRU layer returns a single value. The GRU layers capture sequential patterns in the input data. Between each GRU layer, a dropout layer is introduced with a rate of 20% to regularise the model by randomly deactivating some neurons during training, thereby preventing overfitting. Lastly, a dense layer with a single output unit is added to the model, generating a single output value.
Similarly to the GRU model, we tested diferent configurations in order to elaborate the LSTM model architecture for energy prediction. We started with an initial architecture, that is composed of five layers: Input layer, LSTM layer, dropout layer, and two fully-connected dense layers. We added LSTM layers with diferent number of units and tested diferent combinations progressively. As a result, our ifnal architecture of the LSTM model consists of an input layer followed by three 64 neural units LSTM layers, and utilising a hyperbolic tangent (tanh) as an activation function. Between each LSTM layer, a dropout layer is introduced with a rate of 20% to regularise the model, thereby preventing overfitting. Lastly, a dense layer with a single output unit is added to the model, generating a single output value.
In this section, we present the training configuration. It involves the various settings and parameters used during the training process of each model. In order to determine the best configuration, we performed a hyperparameter tuning. Table 1 presents the optimal training configuration for our models.
In this section, we present comprehensive view of the proposed models’ performances. This is performed using the following performance metrics: Train score, Validation score, R2 score (Coeficient of Determination), MSE (Mean Squared Error), RMSE (Root Mean Squared Error), MAE (Mean Absolute Error).
“Train score” and ”Validation score” are used to assess the model’s fitting and generalisation abilities. As shown in figure 7 the learning curves of the GRU and LSTM provide a visual representation of the model’s performance evolution during the learning process. They show that the models’ performance improves on both training and validation data as the amount of data used for training increases. This trend reflects these models’ ability to efectively generalise and learn relevant patterns present in the data.
Figure 8 presents training and validation loss curves. For both models, loss curves show both the training and validation loss decrease, stay close to each other and plateau at similar low values. This means that both models generalise well to new data, capturing meaningful patterns without getting bogged down by noise or fitting the data too closely. In other words, the absences of overfitting and underfitting.
Figure 9 depicts the real electricity consumption and the one predicted by the GRU and LSTM models over the course of a week. It is evident that the prediction curves closely track the real electricity consumption fluctuations during the day and night, hour by hour, except for a few peak hours. This variability is due to random fluctuations arising from the household occupants.
The final results summarised in Table 2demonstrate that the LSTM model stands out by delivering the best performance in electricity consumption prediction. Performance metrics clearly showcase its ability to efectively capture consumption trends, resulting in accurate predictions.
In this paper, we introduced a deep learning-based approach for short-term EEC prediction. Our contribution encompassed two key aspects: an architectural framework for EEC prediction and the design of DL prediction models LSTM and GRU. First, we presented our architecture for EEC prediction that organises the prediction process. the proposed architecture serves a high-level design that outlines the components, their interactions, and the organisation of the prediction system. Then, we presented our DL prediction models which are LSTM and GRU. Both of them have demonstrated good performances in term of short-term EEC prediction. However, LSTM model outperformed the GRU model across all metrics, showing higher R2 score and lower error values. Our future work will concentrate on refining our models’ performance by integrating additional factors that influence EEC, such as meteorological conditions and building characteristics. This enhancement aims to further elevate the accuracy and reliability of our predictions, contributing to more efective energy consumption management.
During the preparation of this work, the author(s) used GPT-4 for grammar, spelling checks and rephrasing sentences or paragraphs to improve clarity, conciseness, and style. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.
This work was partially supported by the LABEX-TA project MeFoGL: ”Méthode Formelles pour le Génie Logiciel” [1] IAEA, Contribution du nucléaire à la sécurité énergétique en augmentant la production d’électricité en 2021, https://www.iaea.org/fr/newscenter/news/, 2023. Consulté le 8 juin 2023. [2] Y. Benhamouda, H. Khelifa, Evolution de la consommation d’électricité par le secteur «ménages» en algérie: faut-il commencer la lutte contre le gaspillage? evolution of household electricity consumption in algeria: Should the fight against waste be started? (2021). [3] APRUE, https://www.aprue.org.dz/index.php/fr/communication/publication, 2019. Consulté le 3 juin 2023. [4] Sonelgaz, Demande en électricité, https://www.sonelgaz.dz/fr/5526/demande-en-electricite-2, 2022.
Consulté le 05 juin 2023. [5] R. B. Hamida, L’énergie entre les opportunités de développement et les risques de la dégradation de la qualité de l’environnement: cas du gouvernorat de Sfax (Tunisie), Ph.D. thesis, Université d’Auvergne-Clermont-Ferrand I; Université de Sfax (Tunisie), 2014. [6] M. Alanbar, A. Alfarraj, M. Alghieth, Energy consumption prediction using deep learning technique (2020). [7] F. Amara, Modélisation et prévision de la demande d’électricité résidentielle, Ph.D. thesis, Université du Québec à Trois-Rivières, 2018. [8] R. E. Edwards, J. New, L. E. Parker, Predicting future hourly residential electrical consumption: A machine learning case study, Energy and Buildings 49 (2012) 591–603. [9] M. Alhussein, K. Aurangzeb, S. I. Haider, Hybrid cnn-lstm model for short-term individual household load forecasting, Ieee Access 8 (2020) 180544–180557. [10] X. M. Zhang, K. Grolinger, M. A. M. Capretz, L. Seewald, Forecasting residential energy consumption: Single household perspective, in: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), 2018, pp. 110–117. doi:10.1109/ICMLA.2018.00024. [11] I. Kiprijanovska, S. Stankoski, I. Ilievski, S. Jovanovski, M. Gams, H. Gjoreski, Houseec: Day-ahead household electrical energy consumption forecasting using deep learning, Energies 13 (2020) 2672. [12] W. Kong, Z. Y. Dong, Y. Jia, D. J. Hill, Y. Xu, Y. Zhang, Short-term residential load forecasting based on lstm recurrent neural network, IEEE transactions on smart grid 10 (2017) 841–851. [13] D. Ageng, C.-Y. Huang, R.-G. Cheng, A short-term household load forecasting framework using lstm and data preparation, IEEE Access 9 (2021) 167911–167919. [14] D. Syed, H. Abu-Rub, A. Ghrayeb, S. S. Refaat, Household-level energy forecasting in smart buildings using a novel hybrid deep learning model, IEEE Access 9 (2021) 33498–33511. [15] M. Sajjad, Z. A. Khan, A. Ullah, T. Hussain, W. Ullah, M. Y. Lee, S. W. Baik, A novel cnn-gru-based hybrid approach for short-term residential load forecasting, Ieee Access 8 (2020) 143759–143768. [16] A.-N. Khan, N. Iqbal, A. Rizwan, R. Ahmad, D.-H. Kim, An ensemble energy consumption forecasting model based on spatial-temporal clustering analysis in residential buildings, Energies 14 (2021) 3020. [17] T.-Y. Kim, S.-B. Cho, Predicting residential energy consumption using cnn-lstm neural networks,
Energy 182 (2019) 72–81. [18] F. U. M. Ullah, A. Ullah, I. U. Haq, S. Rho, S. W. Baik, Short-term prediction of residential power energy consumption via cnn and multi-layer bi-directional lstm networks, IEEE Access 8 (2019) 123369–123380. [19] X. Guo, Y. Gao, Y. Li, D. Zheng, D. Shan, Short-term household load forecasting based on long-and short-term time-series network, Energy Reports 7 (2021) 58–64. [20] A. Chen, Z. Yu, X. Yang, Y. Guo, J. Bian, Y. Wu, Contextualized medication information extraction using transformer-based deep learning architectures, Journal of Biomedical Informatics 142 (2023) 104370. [21] R. Jozefowicz, W. Zaremba, I. Sutskever, An empirical exploration of recurrent network architectures, in: International conference on machine learning, PMLR, 2015, pp. 2342–2350. [22] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, Learning phrase representations using rnn encoder-decoder for statistical machine translation, arXiv preprint arXiv:1406.1078 (2014). [23] M. Data, Electrical consumption profile for households in romania, https://data.mendeley.com/ datasets/ykjcjsnhds/1, 2018. Consulté le 6 avril 2023.