CNN-GRU for Air Quality Index Forecasting

Abstract

Recently, air pollutant becomes an urgent problem, especially in urban areas; therefore, a system to predict future air properties is demanded to improve the quality of life. By proposing a neural network using weather and air data to perform the Air Quality Index forecasting task, the model shows a reasonable performance instead of traditional regression methods. Moreover, the dependency of the number of input days, and the hour prediction accuracy are also discussed.

1. Introduction

2. Related Work

In [ 3 ], authors use three regression models as feature extractors using preprocessed sensor data, extract two new features Part-Of-Day (cluster 24 hours of a day into 5 groups) and Is-RushHour (identify a Part-Of-Day group is rush hour or not), and employ a stack generalization technique to combine multiple regressions’ results into the final output.

In [ 4 ], Convolution Neural Network (CNN) [ 5 ] is used as a feature extractor. A modified Long Short-term Memory (LSTM) [ 6 ] called ILSTM removes the output gate to improve the hidden gate and input gate. A CIM gate is introduced to prevent saturation during training.

In this work, instead of learning from raw and noisy data, Convolution Neural Network (CNN) [ 5 ] is proposed to learn rich features. By performing 1D convolutions and Rectified Linear Unit (ReLU) [ 7 ], high dimensional features are constructed by linear and non-linear operations on nearby hour values and other pollutants values. Besides, Gated Recurrent Unit (GRU) [ 8 ] is used as an auto-regression model to avoid the "gradient explosion" and "gradient vanishing" problems in RNN [ 9 ], and reduce the number of parameters in LSTM [ 6 ] architecture.

3. Approach

3.1. Data preprocessing

The sensors are facing many errors during running time. The strategy for data preprocessing includes three steps. Firstly, handle missing and wrong behavior data records. After that, resample by hour to calculate the 1-hour-average value, and needed hour-average value for each pollutant (e.g. 24-hour-average value for PM2.5 and PM10). Then AQI, AQI level, and the ifnal AQI with the responsible pollutant. The final total dataset is interpolated by 24 hours and removed invalid records.

3.2. Datasets

The raw data is crawled from a real-time server. After data preprocessing, subsets of data (train, validation, and test) are created strategically to optimize the model’s hour-average value of pollutants. Besides, the target set is constructed for Subtask 1 evaluation metrics, and the train value set is used for pre-training the model before the train set.

The target set’s construction is based on the Subtask 1 metric mentioned in Section 3.4. It aims to use a set of valid records in a diferent number of days starting from ’2022-11-01 00:00:00’ to predict short- (D+1), mid- (D+5), and long-term (D+7) periods respectively. The test set’s time period is the same as the target set but with a continuous 7-day range to predict. For the train and validation set, initialization is similar to the test set, but the time period is before November. The proportion of the train and validation set is 9:1 because of the small number of valid 7-day periods. For pre-training the model, the train value set contains all record which has available hour-average value of pollutants for the next 7 days. For explanation, the main contributor to model performance is the hour-average value of air factors so only these value is required for the pre-training step.

3.3. Methods

3.4. Metrics

There are two main metrics used for the evaluation of Subtask 1 [ 2 ] for 6 pollutants (PM2.5, PM10, CO, NO2, SO2, O3): (1) MSE/MAE for the hour-average value of each pollutant. (2) F1-score for AQI for each pollutant.

The prediction is evaluated in short-, mid- and long-term periods which are 1, 5, and 7 days in the future respectively.

For training hour-average values, the only used metric is MSE.

3.5. Model architecture

There are 3 main components in the architecture: a feature extractor, a recurrent neural network, and an AQI calculator (Figure 1).

Feature extractor: A convolution neural network [ 5 ] includes 4 layers using 1-D filters to learn the relative hour-average value from raw hour-average values. The BatchNorm and ReLU layers are used after each 1D convolution.

Autoregressive model: Gated recurrent unit [ 8 ] is used to handle the time-series features from extractor. The input and output of GRU represent 24-hour features. Each output feature is re-weighted by a 24x24 matrice, then feed into the GRU model until 7-day output features are generated.

To achieve the pollutants’ value, each 24-hour feature which is the output of the GRU is transformed into the value vector representing the hour-value of pollutants.

AQI calculator use the input and output hour-average values to calculate the AQI value by the Equation 1 which is mentioned in Technical Assistance Document for the Reporting of Daily Air Quality – the Air Quality Index (AQI) 1.

=

− −

( − ) + Where : the index of pollutant p : the truncated concentration of pollutant p : the concentration breakpoint that is greater than or equal to : the concentration breakpoint that is less than or equal to : the AQI value corresponding to : the AQI value corresponding to (1)

4. Experiments and Results

4.1. Baseline

For the pre-training phase, the configuration includes 1 input day, Adam optimization with a learning rate of 0.01. The training result is plotted in Figure 2. The best validation loss is 0.00624 (normalized) achieved at epoch 13, then the model enters the saturation period. By reducing the learning rate to 0.001, the model converges slower and smoother with a minimal validation loss is 0.00648. The pre-trained model is tested on the test set, and the results are 0.0638 (normalized). 1https://www.airnow.gov/sites/default/files/2020-05/aqi-technical-assistance-document-sept2018.pdf (a) leaning rate = 0.01 (b) leaning rate = 0.001

The final model is also evaluated by validation, test, and target sets (Table 1). Although the test set is smaller than the validation set, the performance is reduced.

4.2. Number of input days

The model performance in the train value set is tested from 1 to 10 input days (Figure 3). With 1 input day, the loss is always minimized the best. Furthermore, the model’s convergence abilities of 2, 3, and 4 input days are the same, nearly as 1 day. There is a noticeable performance when feeding for 9 days.

4.3. Prediction hour in a day

By plotting the MSE loss of 24 hours per day in 7 prediction days in the train set (Figure 4), there is a common trend in the hour accuracy. In all 7 days, the period between 10 a.m. and 8 p.m. keeps the smallest loss. The MSE loss on days 1, 5, and 7 share the same shape.

5. Conclusion

Air Quality Index (AQI) forecasting is crucial for improving the quality of smart cities. In this work, a CNN-GRU model is optimized to predict the hour-average value of six pollutants by feeding raw input data records. The achieved results are noticeable to improve better with data preprocessing methods that reduce noisy data, outlines, and sensor problems, solving the saturation problem.

References

[1] AirNow, Air quality index (aqi) basics , https://www.airnow.gov/aqi/aqi-basics/, 2022 . Accessed: 2022 -11-28.

[2] M.-S. Dao , T.-H. Dang , T.-L. Nguyen-Tai , T.- B. Nguyen , D.-T. Dang-Nguyen, Overview of mediaeval 2022 urban air: Urban life and air pollution , MediaEval'22: Multimedia Evaluation Workshop ( 2022 ).

[3] D. Duong , Q. Le , T.-L. Nguyen-Tai , H. Nguyen , M. Dao , B. Nguyen , An Efective AQI Estimation Using Sensor Data and Stacking Mechanism , 2021 . doi: 10 .3233/FAIA210040.

[4] J. Wang , X. Li , L. Jin , J. Li , Q. Sun , H. Wang , An air quality index prediction model based on cnn-ilstm , Scientific Reports 12 ( 2022 ). doi: 10.1038/s41598-022-12355-6.

[5] Y. LeCun , B. Boser , J. S. Denker , D. Henderson , R. E. Howard , W. Hubbard , L. D. Jackel , Backpropagation applied to handwritten zip code recognition , Neural Computation 1 ( 1989 ) 541 - 551 . doi: 10 .1162/neco. 1989 . 1 .4.541.

[6] S. Hochreiter , J. Schmidhuber , Long short-term memory , Neural computation 9 ( 1997 ) 1735 - 80 . doi: 10 .1162/neco. 1997 . 9 .8.1735.

[7] V. Nair , G. E. Hinton , Rectified linear units improve restricted boltzmann machines , in: Proceedings of the 27th International Conference on International Conference on Machine Learning , ICML' 10 , Omnipress , Madison, WI , USA, 2010 , p. 807 - 814 .

[8] K. Cho , B. van Merrienboer , D. Bahdanau , Y. Bengio , On the properties of neural machine translation: Encoder-decoder approaches , CoRR abs/1409 .1259 ( 2014 ). URL: http://arxiv.org/abs/1409.1259. arXiv: 1409 . 1259 .

[9] D. E. Rumelhart , J. L. McClelland , Learning Internal Representations by Error Propagation , 1987 , pp. 318 - 362 .