Introduction

Predicting energy consumption for electric vehicles (EVs), especially those used in commercial heavy-duty contexts, is paramount for improving their operational efficiency and promoting sustainability. Effective energy consumption forecasts are indispensable for strategic route planning, optimising charging protocols, and ensuring that vehicle configurations align well with specific operational demands. As electric vehicles gain traction as a viable and ecofriendly alternative to internal combustion engine vehicles, the importance of precise energy consumption predictions becomes increasingly pronounced. The challenges in this domain are multifaceted, stemming from the inherent variability in driving conditions, vehicle load, and diverse environmental factors, which collectively complicate the development of accurate predictive models. Overcoming these obstacles is essential not only for enhancing the reliability and performance of EVs but also for minimising operational costs and boosting the overall efficiency of electric transport systems.

The transition to electric vehicles is a significant step towards reducing greenhouse gas emissions and achieving sustainable transportation goals. However, since limited energy storage puts unique constraints on which operations are feasible, the benefits of EVs can only be fully realised through the development of specified forecasting methods that accurately anticipate energy needs. In this context, AI and ML emerge as transformative tools. AI-driven models can analyse vast amounts of data to uncover patterns and relationships that are not immediately apparent, providing more accurate and reliable energy consumption forecasts. These models can adapt to new data, continuously improving their predictions over time.

Nevertheless, energy consumption forecasting for EVs faces critical challenges, such as dynamic driving conditions and fluctuating loads, which makes even state-of-the-art methods struggle to handle complex real-world data effectively. While the potential to learn from historical data and identify trends that influence energy consumption is the biggest strength of ML-based approaches, it is crucial to develop robust models that can generalise well across different scenarios and vehicle types.

The complexity and variability inherent in forecasting energy consumption for electric vehicles make it a relevant testing ground for cutting-edge modelling techniques that promise to handle diverse and dynamic data inputs. In particular, Multi-Task Learning (MTL) presents a compelling solution by enabling simultaneous training across multiple related tasks, thereby leveraging shared information to improve the predictive performance of each task. In contrast, training machine learning models in a traditional setting only utilize the target task. MCL is particularly beneficial in scenarios with limited training data, as it enhances generalisation by incorporating auxiliary tasks that provide additional training signals. Moreover, the efficacy of MTL can be further amplified by integrating curriculum learning (CL), which structures the learning process in a progressive manner. Curriculum learning organises tasks from simple to complex, allowing the model to build a robust foundation before tackling more challenging problems. By combining these methodologies into multi-task curriculum learning (MCL), we can efficiently train neural networks that not only perform better on individual tasks but also generalise more effectively across different contexts. MCL optimises the learning trajectory, ensuring that simpler tasks enhance the model's capability to learn more complex ones, ultimately leading to more accurate and reliable energy consumption forecasts for electric heavy-duty vehicles. This integrative approach has been shown to be a potent strategy to address the multifaceted challenges in several domains but has not been applied to EV auxiliary energy forecasting before. Thus, this paper aims to evaluate the suitability of MCL in this real-world, complex scenario. Generating a set of auxiliary tasks is a critical step in the implementation of MCL -and how to do it for forecasting energy consumption in EVs requires experimental evaluation. To create auxiliary tasks, one must first obtain an understanding of the primary task, identifying key factors and variables that influence energy consumption and the types of patterns that are indicative of future behaviour. These factors often include vehicle load, driving speed, route characteristics, weather conditions, and driver behaviour. Each of these variables can serve as the basis for an auxiliary task. For instance, an auxiliary task might involve predicting the impact of vehicle load on energy consumption under different traffic conditions or estimating the effect of varying driving speeds on battery usage. Historical data from real-world vehicle operations can be mined to extract relevant patterns and correlations, which can then be used to define these auxiliary tasks. In this paper, we have decided to focus on the patterns within the forecasted value itself instead of exploiting multivariate vehicle signals. In particular, we define several types of energy consumption characteristics as targets for the auxiliary tasks, such as questioning whether the consumption in the next time frame exceeds the global mean, whether the consumption will be higher in the next time frame compared to the current consumption, or predicting the consumption difference between the start and the end of the next time frame. These tasks are general enough to be suitable for any forecasting task, while at the same time being sufficiently closely related to the actual primary task to, hopefully, provide useful information to boost the training process.

The core contribution of this paper is the evaluation of applying several multi-task curriculum learning techniques for forecasting the energy consumption of heavy-duty electric vehicles, including the proposition of utilising key consumption characteristics as targets for generating auxiliary tasks for MCL. Comparison of MCL variations, with combinations of curriculum learning strategy (sequential learning and progressive continual learning) and auxiliary tasks, illustrates the improvements in the performance on a real-world data collected from normal operations of commercial transportation electric vehicles. The experimental results show progressive continual learning, with a logistic growth weighting function governing the learning balance between the primary and the auxiliary task, achieves the best performance; the result also shows that the first auxiliary task is the most helpful task for subsystems 1 and 4; the third auxiliary task is the most helpful task for subsystems 2 and 3. Furthermore, it is observed that MCL with the proposed auxiliary tasks can improve the learning efficiency of the model, achieving faster convergence to a point beyond which the gain from further training is limited.

Related Work

Curriculum learning enables the training of machine learning models in a meaning order, from easy samples to sets of difficult and complex samples [1]. A Common approach for CL introduces easy-to-hard ordering of samples for the training process, e.g., vanilla CL, self-paced CL, balanced CL, etc. When multiple tasks are available, the easy-to-hard ordering of the tasks to be learned can be applied as well. Multi-task learning can be applied, by sharing information across a set of related tasks in the training process, and the performance can be further improved [2] via, e.g. Gradnorm [3] balancing the losses between multiple tasks. While most multi-task learning approaches aim at learning multi-tasks simultaneously, progressive curriculum learning allows determining the best order to learn multiple tasks to maximise the final result. Study [4] presented by Pentina et al. finds the best order of tasks to be learned in a sequence based on a generalisation bound criterion to optimise the average expected classification performance over all the tasks. Work [5] by Siahpour et al. introduced a penalty coefficient, as a function of the epoch step, to govern the training process by suppressing the loss, and noise respectively, from the domain discrimination task in the early stage, to ensure the efficient training of neural networks. Shi et al. proposed progressive contrastive learning [6] based on multi-prototypes in the dataset, the training process is ordered to learn the centroid prototype first, followed by the hard prototype, and finally the dynamic prototype. In this work, we explore sequential learning and progressive continual learning with a set of auxiliary tasks generated based on key characteristics of target signal.

Problem Formulation

For a given primary learning task 𝒯 𝑖 , we create a set of auxiliary tasks 𝒯 𝑗 𝑖 , where 𝒯 𝑖 corresponds to the primary task (in our case, the forecasting of energy consumption for the 𝑖-th auxiliary subsystem in an electric truck), and 𝒯 𝑗 𝑖 corresponds to the 𝑗-th type of auxiliary task. The majority of the multi-task learning studies aim to learn all relevant tasks together to improve the performance for each task 𝒯 𝑖 . In our study, we are only interested in improving the energy forecasting tasks 𝒯 𝑖 , not the generated auxiliary tasks 𝒯 𝑗 𝑖 . All energy forecasting tasks and the auxiliary tasks are learned from the same dataset, multi-variate time series sensor readings were collected from the normal operations of several heavy-duty electric vehicles.

Let us denote data of the multivariate time series x of each vehicle 𝑣 by 𝑋 = { 𝑥 𝑘 𝑣,𝑡 | 𝑡 = 1, 2, ..., 𝑇 𝑒 (𝑣), 𝑘 = 1, 2, ..., 𝐾}, where 𝑥 𝑘 𝑣,𝑡 is the value of the 𝑘-th feature x given a vehicle/trajectory 𝑣 at time 𝑡, and 𝑇 𝑒 (𝑣) corresponds to the end of the recording. A subset of the features 𝑢 𝑖 𝑣,𝑡 reflects the energy consumption of subsystem 𝑖 at time 𝑡. The target energy consumption 𝑦 𝑖 𝑣,𝑡 0 in a future time frame 𝜏 𝑝ℎ can be approximated by summing up the energy consumed over this time frame 𝑦 𝑖 𝑣,𝑡 0 = ∑︀ 𝑡∈[𝑡 0 ,𝑡 0 +𝜏 𝑝ℎ ] 𝑝 𝑖 (𝑡) • ∆𝑡, where 𝑝 𝑖 (𝑡) is the power consumption at time 𝑡, and ∆𝑡 is the time interval between two samples.

In this study, we set 𝜏 𝑝ℎ equal to 10 minutes. For a given forecasting task 𝒯 𝑖 , a regression model 𝑓 𝑗 𝑖 (•) is trained together with one of the auxiliary tasks 𝒯 𝑗 𝑖 to estimate consumption 𝑦 𝑖 𝑣,𝑡 . In this study, neural networks, a shared feature extractor, with multiple heads, each corresponding to one task, were trained under different settings and evaluated for their performance after 200 training epochs. We explore different multi-task curriculum learning settings and auxiliary tasks for forecasting energy consumption. The MCL methods were compared to the traditional approach

Method

Auxiliary Tasks

For a given regression task 𝒯 𝑖 (forecasting energy consumption for one of the subsystems), a set of auxiliary tasks was generated to assist the learning progress. We explore the use of five types of consumption characteristics as targets for creating the auxiliary tasks: i) 𝒯 1 𝑖 : classifying whether the consumption in the next time frame exceeds the global mean for that subsystem 𝑖; ii) 𝒯 2 𝑖 : classifying whether the consumption will increase in the next time frame, compared with the current consumption; iii) 𝒯 3 𝑖 : classifying whether the consumption at the end of the next time frame exceeds the starting point; iv) 𝒯 4 𝑖 : predicting the consumption difference between the start and the end of the next time frame; v) 𝒯 5 𝑖 predicting the difference between the peak consumption and the lowest consumption in the next time frame. The first three auxiliary tasks are classification task, the other two tasks are regression task. Learning to predict these key consumption characteristics in these auxiliary tasks 𝒯 𝑗 𝑖 , along with the primary tasks 𝒯 𝑖 , under MCL, are evaluated for their usefulness.

Network Architecture

The regression model evaluated for MCL in this study builds on a multi-layer perceptron. The model is comprised of a shared feature extractor and two heads, one head carries out the main task 𝒯 𝑖 , and the other corresponds to one of the five auxiliary tasks 𝒯 𝑗 𝑖 . The network architecture is illustrated in Figure 1. For auxiliary tasks that are classification tasks, a sigmoid function was applied to the output of the corresponding head.

Curriculum Learning Strategy

The two curriculum learning strategies evaluated in this work are sequential learning (SeqL) and progressive continual learning (PCL). The overall optimisation loss ℒ can be defined as:

ℒ = 𝜆ℒ 𝒯 𝑖 + (1 − 𝜆)ℒ 𝒯 𝑗 𝑖 (1)

where ℒ 𝒯 𝑖 denotes the loss for the primary tasks, while ℒ 𝒯 𝑗 𝑖 denotes the loss for the auxiliary task 𝑗. The SeqL employed imposes a fixed ordering of the tasks, e.g. learning the auxiliary task first, before a predetermined epoch, and the primary task afterwards:

𝜆 𝑆𝑒𝑞𝐿 = {︃ 0 if 𝜂 < 𝒩 𝑒𝑝 1 if 𝜂 ≥ 𝒩 𝑒𝑝 (2)

where 𝜂 is the current training epochs, and 𝒩 𝑒𝑝 is the number of epochs predetermined to switch to another task. The PCL employs a weighting mechanism, a function of training epochs, to govern the learning process and gradually increases the weights on the loss corresponding to the primary task:

𝜆 𝑃 𝐶𝐿 = 2 1 + 𝑒𝑥𝑝(−10𝛼𝜂/𝒩 𝑡𝑜𝑡 ) − 1 (3)

where 𝛼 is a coefficient governing the change rate (see Figure 2 for an illustration), 𝜂 is the current training epochs, and 𝒩 𝑡𝑜𝑡 is the total amount of training epochs. The two curriculum learning strategies were compared with MTL without any special curriculum learning and learning only on the primary task 𝒯 𝑖 The two evaluation criteria in this study are (i) the test loss (Mean Absolute Error, MAE) after training converged, i.e., 𝑁 𝑡𝑜𝑡 epochs, and (2) whether the proposed learning strategy achieves a faster convergence time, i.e., the epoch at which the test loss has reached a saturation point (no further significant decrease in the loss afterwards). In each case, different variants of MCL are compared against a learning process without any multi-task curriculum learning. The saturation point is detected using a knee point detection algorithm [7], proposed by Satopaa et al.

Experiment Result

The energy consumption dataset was collected from several electric trucks operating in different countries for a couple of months, including sensor readings of mileage, speed, ambient temperature, and energy consumed for auxiliary subsystems, etc., from sessions of driving.

The four subsystems we forecast energy consumption for are the air compressor 𝒯 1 , the air conditioner 𝒯 2 , the cabin heater 𝒯 3 , and the heater of the energy storage system 𝒯 4 .

For the experiment conducted in this paper, the neural networks were implemented via Pytorch library [8], using an ADAM optimiser with a learning rate of 0.001. The loss function for the regression tasks is mean absolute error (MSE), and binary cross-entropy (BCE) was employed as the loss function for the classification tasks. The total number of training epochs 𝒩 𝑡𝑜𝑡 is set to 200. For the sequential learning strategy, 𝒩 𝑒𝑝 is set to 100, and for the progressive continual learning, 𝛼 value of 0.1 (i.e. a linear function) and 0.3 are tested. The experiments were conducted using 4-fold cross-validation driving session-wise, i.e. data from the same driving session would never appear in the training and the testing population together.

Table 1 and Table 2 show the training and testing losses after 200 epochs of training of the neural networks using multi-task learning with any curriculum learning (MTL), sequential learning (SeqL), progressive continual learning with an 𝛼 of 0.1 (PCL-lin), and an 𝛼 of 0.3 (PCL-exp). The baseline performance, single task learning (STL), is produced with learning only on the primary task 𝒯 𝑖 for each subsystem, shown in the parenthesis. It is shown in both tables that the lowest averaged MSE is achieved using PCL-exp. As a sanity check, Table 1 demonstrated that the training losses, after 200 epochs of training, of most MCL methods did converge to a level comparable to STL. For the testing losses shown in Table 2, applying PCL-exp on task sets {𝒯 1 , 𝒯 1 1 } and {𝒯 4 , 𝒯 1 4 } achieved lowest averaged MSE for forecasting energy consumption of subsystem 1 and 4 (i.e., the first auxiliary task appears to be the most helpful auxiliary task for subsystem 1 and 4); similarly, applying PCL-exp on task sets {𝒯 2 , 𝒯 3 2 } and {𝒯 3 , 𝒯 3 3 } achieved lowest averaged MSE for forecasting energy consumption of subsystem 2 and 3.

Figure 3 illustrates the differences between several multi-task curriculum learning strategies, focusing on the convergence speed. Specifically, we identify a reference point (epoch) beyond which the gain from further training is limited. This reference point is computed using a knee point detection method (algorithm [7] by Satopaa et al.) on the mean STL test losses (shown as grey dots and the corresponding dash line). The four plots in Figure . 3 illustrate the testing loss for learning the four primary tasks, along with their 5-th auxiliary task, i.e. 𝒯 5 𝑖 . It is observed in Figure 3: i) there is no significant difference between the four approaches for 𝒯 1 ; ii) MTL and PCL-lin drop slightly slower compared to STL and PCL-exp for 𝒯 2 ; iii) both PCL approach drop slower compared with STL and MTL for 𝒯 3 ; iv) MTL, PCL-lin, and PCL-exp drops faster compared to STL. Table 3 shows a comparison between MCL methods on the convergence time to the reference point (computed based on STL mean testing losses over the four folds). It is observed that: i) MTL outperforms STL in all four primary tasks, and converged to the reference point faster than other approaches in three out of four primary tasks; ii) the performance of PCL-lin achieved converged fast for two of the tasks; iii) PCL-exp achieved better performance compared to PCL-lin, with overall short convergence time. The result corresponding to SeqL is particularly interesting. Although a 𝒩 𝑒𝑝 of 100 epochs is adopted for SeqL (i.e. trained on one of the auxiliary tasks for the first 100 epochs before learning the primary task), the testing loss converges to the reference point within 10 epochs in the majority of the cases. From an empirical perspective, the proposed auxiliary tasks assisted the learning (of the models) for the primary task, resulting in a faster convergence time to the reference point.