In the rapidly evolving industrial landscape, sensors are integral to automation applications. Capturing and analyzing the vast amount of time series data is crucial for optimizing processes. However, analyzing this sensor data in isolation presents challenges, particularly in time series analysis, due to the influence of various external contextual factors that are not always apparent. Integrating these contextual factors with time series data is essential for time series analysis. However, these contextual factors are often heterogeneous in the time dimension due to the diverse nature of the data, makes integration challenging. Therefor, as a part of this PhD research that is currently at the beginning of the second year, we aim to introduce a systematic approach for integrating contextual factors with heterogeneous time dimensions. This integration enables the transformation of data with heterogeneous time dimensions into a format that can be efectively processed by machine learning and deep learning models for time series analysis. We use Water Distribution Systems (WDSs) as a representative use case and aim to demonstrate how this integration enhances the accuracy and reliability of time series analysis. time series analysis, heterogeneous time dimensions, time series forecasting, anomaly detection Workshop Proceedings
In this work, we use Water Distribution Systems (WDSs)
as a representative application use case, where sensors are
employed for monitoring the consumption of water. These
observations serve as crucial inputs for, e.g., detecting water
losses or estimating the water demand. WDSs are equipped
with networks of sensors and control units, such as flow
and pressure sensors, to ensure eficient resource
management [
ceur-ws.org etc. These external contextual factors play a crucial role in shaping how resources are utilized and can significantly impact the accuracy of forecasting and anomaly detection in systems like WDSs. An efective integration of this diverse mixture of data enhances the decision making process by providing a comprehensive understanding of processes. The main objective of this PhD will be on the systematic integration of contextual information with heterogeneous time dimensions (static, interval, event, and secondary time series) for the efective analysis of time series, i.e., missing values imputation, anomaly detection and forecasting.
According to a recent survey on anomaly detection in the
IoT and IIoT domain by Rodríguez et al. [
Daniel et al. [
Most of the existing works consider internal contexts while analyzing sensory data, such as day, time, yearly seasons, months, and years etc., that are typically uniform and consistent and easily handled, as it is derived from the time dimension of the time series data. On the other hand, integrating external context is more challenging and requires a transformation because the information might be represented using a diferent time dimension and often lacks uniformity.
The main objective of this thesis is to enhance the performance of machine learning and deep learning models for time series analysis by integrating contextual information with heterogeneous time dimensions together with sensory readings. By acknowledging the crucial role of context in time series analysis, this research endeavors to develop novel methodologies that leverage contextual information to significantly enhance the accuracy and robustness of these models. Through rigorous experimentation and analysis, this study aims to establish a deeper understanding of how contextual information can be efectively integrated into existing frameworks, thereby advancing the state-of-the-art in time series analysis within diverse domains. The main research questions for the PhD are as follows.
RQ1 How to combine relevant contextual information with heterogeneous time dimensions and time series data to build data driven models for time series analysis, such as anomaly detection and forecasting? RQ2 How to detect patterns in time series data that deviate in specific contexts using data driven approaches? RQ3 How to build a data driven model that efectively handles heterogeneity and adapts to time series dynamics?
Machine learning and deep learning techniques for time series analysis, such as Random Forest, Support Vector Machine (SVM), Long Short Term Memory (LSTM) etc., require input data with a uniform or homogeneous format, i.e, numerical values (vectors) recorded over regular time steps. However, contextual information with heterogeneous time dimensions lacks this uniformity. The efective integration of data with heterogeneous time dimensions involves distinct strategies for each type, and we illustrate it on the example of WDSs.
WDSs utilize networks of sensors such as flow sensor to monitoring the consumption of water, that generate time series data recorded either regularly or irregularly, with varying sampling frequencies. Along with consumption patterns (primary time series) to be analyzed, secondary time series may also be considered, such as temperature or rainfall. For example, water consumption is highly correlated with temperature, which is also time series data. Tables 1 and 2 exemplify time series data collected from a information is valid. In the domain of WDSs, interval data represents periods, such as tourism seasons that may or may not occurring annually over some period of time. It is crucial to consider tourism related information as it significantly influences water consumption. During the peak of a tourism season, water consumption increases substantially compared to of-season periods. and therefore, incorporating tourism seasonality into the analysis is essential for accurately understanding the water consumption patterns. water flow sensor and a temperature sensor. This data can be aligned with specific timestamps to ensure consistency, typically achieved through interpolation or resampling to establish regular time steps. Static data, as illustrated in Table 3, remains constant over time and may be associated with every time point, maintaining its validity throughout the period of interest. In WDSs, static data, such as sensor location and characteristics play a crucial role for cross sensor analysis. For instance the location of sensors can provide valuable insights into the population distribution of diferent regions, which can directly influence factors such as water consumption. Areas with higher population density are likely to exhibit diferent consumption patterns compared to sparsely populated regions. By considering sensor locations, models can better capture and interpret variations in water consumption data, leading to more accurate and informed analyses in decision making. Event data as illustrated in Table 4, captures specific time points of an event occurrence. Such data may be integrated by marking relevant time points with event indicators, offering a comprehensive view of discrete events over time. In WDSs, event data captures irregular occurrences such as sensor calibration, cleaning, replacement or bank holidays. Some of this events can directly influence sensor readings, potentially leading to inaccuracies in interpreting water consumption data. Therefore, it is crucial to include such type of information for accurate analysis of water consumption.
Interval data as illustrated in Table 5, are characterized by start and end time points for which a particular contextual
01-06-2023—30-10-2023 01-07-2023—30-10-2023
high high
During the data representation stage, data with heterogeneous time dimensions is transformed into a uniform or homogeneous format suitable for machine learning models that require numerical vectors over regular time steps. This stage following the sampling rate of the main time series or user parametrization extracts diferent values from the data with heterogeneous time dimensions using simple value extraction, diferent aggregation functions, or one-hot encoding (similar to resampling). For instance at each time step, for constant data it may just extract a value or a one-hot encoding, for event data it may extract a one-hot encoding and/or count of events, for interval data it may extract a count and/or an indicator if a period just started or ended, and for secondary time series it may apply a resampling based on the new frequency. This initial transformation ensures that data has the same homogeneous representation, such as for instance in Table 6 (do not consider the last two columns for now). Each row in the table indicates to a specific time step, while each column represents values extracted from the heterogeneous data. Figure 3 illustrates this transformation process to homogeneous data graphically, making it compatible with machine learning models. This step can be achieved through a combination of densification and aggregation, and some of these values extracted at each time step will be used later to extract features that are related to previous (cross) time steps.
Tohhhullloooiiirgggwwwihhhsm Mrcceaaapcillllneiiabbtanncrreneooaanmittnaiiooegnnnncte Las3t003000Cal. Dur. high020201 tourism
After data representation, the subsequent step involves feature extraction, where features from the generated values and primary time series are extracted. Similarly to data representation this stage is sensitive to the type of homogeneous time dimension from which the values were extracted. For instance, for event data, the number of events in the past or elapsed time since a specific event last occurred may be generated, e.g., see “Last Cal.” in Table 6 that records the number of hours since the last calibration event. For interval data the time since a period last ended or the duration so far may be generated, e.g., see “Dur. high tourism” in Table 6 that records the duration of high tourism in days. These additional features provide information across time to the model and help setting a given time step into context, ultimately improving model performance. For instance, sensor readings with very high values for the last calibration may be subject to higher variability or outliers, or water consumptions deep in the tourism season may be generally very high. While the previous stage was achieved through a combination of densification and aggregation, this step can generated through window functions aggregating over previously generated values. The key challenge is to identify the required sequence of dependencies.
Feature extraction may yield hundreds of features and at this stage it becomes imperative to select features with high importance. Feature selection is indeed crucial for several reasons, including reducing computational complexity and mitigating the risk of overfitting models. Features are chosen based on the model’s performance, and depending on the model’s output, features are iteratively selected, continuing this process recursively during feature selection. This iterative approach enhance the algorithm’s explainability, identifying which features are indeed valuable for decision making.
After feature extraction and selection, the final step involves the development of machine learning and deep learning models for multivariate time series analysis using the example of anomaly detection. To validate the impact of the integrated contextual information in the previous steps, the performance evaluation is twofold. Firstly, model training without incorporating contextual data to establish a baseline performance solely based on available features or signals. Secondly, contextual features are integrated to observe their impact on the model’s predictive capabilities. Comparing the model’s performance in both settings facilitates understanding of how contextual information enhances predictive accuracy and robustness.
Sensor readings often display complex properties influenced by numerous contextual factors. Integrating contextual factors in time series analysis is crucial for accurately interpreting data pattern and identifying factors behind variations. By incorporating contextual factors, we aim to improve performance of machine learning and deep learning models for time series analysis. In this PhD research, currently at the beginning of the second year, we have identified four categories of contextual information with heterogeneous time dimensions. Based on these, we will develop a systematic data management approach for the efective and eficient integration of contextual features for time series analysis.