Calculating an Air Quality Index (AQI) typically uses data streams from air quality sensors deployed at ifxed locations and the calculation is a real time process. If one or a number of sensors are broken or ofline, then the real time AQI value cannot be computed. Estimating AQI values for some point in the future is a predictive process and uses historical AQI values to train and build models. In this work we focus on gap filling in air quality data where the task is to predict the AQI at 1, 5 and 7 days into the future. The scenario is where one or a number of air, weather and trafic sensors are ofline and explores prediction accuracy under such situations. The work is part of the MediaEval'2022 Urban Air: Urban Life and Air Pollution task submitted by the DCU-Insight-AQ team and uses multimodal and crossmodal data consisting of AQI, weather and CCTV trafic images for air pollution prediction.
The Urban Life and Air Pollution task at MediaEval 2022 required participants to predict the air
quality index (AQI) value at +1, +5 and +7 days using an archive of air quality, weather and
images from 16 CCTV cameras, one image taken every 60 seconds [
The biggest issue in this research challenge was the significant gaps in the training data, highlighted in Figure 1 for the data we downloaded, and regarded as a common issue with climate datasets. Because participants downloaded data independently, and because data servers had diferent periods of downtime, it is likely that participants have diferent, but overlapping training data and perhaps other participants managed to download more data than we did. Even so, the amount of data we downloaded allows us to focus on the challenge of data gaps. This is directly addressed in our research methodology by first identifying the sensitivity of the data gaps and adopting a counter-measure to eliminate gap data. Our method comprises 4 steps: • Step 1. Data Analysis. This performs a statistical summary of datasets including the computation of spatial data related to the locations of air quality stations and cameras. • Step 2. Gap Filling. Elimination or maximising the reduction in the gaps in air quality data. • Step 3. Processing CCTV camera images. This step transforms each image into a set of features that can be combined with the air quality feature set. • Step 4. Model Building. This step builds an experimental platform using diferent machine learning model configurations together with diferent feature sets to identify the best performing model/feature set combination.
In the remainder of this section, we described the first 3 steps in detail and in Section 3, we describe our approach to model building for the air quality prediction task.
The downloaded air quality data are collected at 10 monitoring stations in Dalat City, Vietnam from March 2020 to 7th Nov 2022. The data includes air pollutant concentration for NO2(ppm), CO, SO2, O3, PM1.0, PM2.5, PM10 as well as environmental measures namely temperature, humidity, UV, rainfall. In addition, trafic data, in form of images, was recorded every minute from 16 CCTV cameras across Dalat City. Figure 1 shows the availability of the dataset as downloaded by our group. This shows huge gaps in data availability. In our model building we use the first 80% of available data for training machine learning models and the remaining 20% for validation.
For the CCTV data, we downloaded a total of 398,412 images from across all 16 cameras, which took approximately 215GB of storage. If all data had been available and downloaded, there would be approximately 16 cameras x 8 months x 30 days x 24 hours x 60 minutes = 5,529,600 images so our download represents 7.2% of the theoretical maximum.
We re-sized each image to 640x640 and processed each using a medium-sized YOLOv6
objectdetector [
Our approach to gap filling used 3 diferent feature sets.
• Rolling Window (FS1). We generate training data using rolling windows sliding hourly over
the data. Following some experimentation, we determine the historical data length to input
to the model (size of window) to be 2 days. To address missing data, we remove windows
with big gaps and fill windows with small gaps of less than 30% or have data from more than
4 stations.
• Rolling Window with Gap Filling (FS2). Spatial interpolation using a tree-based gradient
boosting model [
We convert all pollutant concentration values into AQI values according to the formula described
in [
We used baseline machine learning models including multi-layer perceptrons, long
shortterm memory (LSTM) [
Table 1 presents results after our own validation using bold font to highlight the best performing model with the second best results underlined. The first 2 columns show the features and models used where runs marked 1-5 represent our 5 submitted prediction attempts. The remaining 6 FS1 FS1 (run 1) FS2 (run 3) FS2 (run 4) FS2 FS2 (run 2) FS3 (run 5)
MLP LSTM LSTM TemAtt LSTM-GNN SpaTemAtt SpaTemAtt columns, which present RMSE scores, are split according to those stations that continued to provide data, “Station (recent data)”, and those that stopped supplying data due to malfunctions, “Station (no recent data)” at the station. For the latter, it was necessary to make predictions for these stations based on recent data at other stations. For both station categories, we provide our accuracy for 1-day ahead, 5-days ahead and 7-days ahead predictions. Validation uses root mean squared error, so the lower the error cost, the better the model.
Analysis. Stations with recent data far out-performed those without so we will focus our discussion on the former. As expected, the predictive accuracy for all models decreases as the forecast window increased, significantly for 5 and 7 day predictions. Feature set 2 (FS2) was the best performing across all time intervals. This highlighted the benefit of our gap filling method but also indicated that the camera data did not deliver any added performance. In terms of models, no single model performed best although our Temporal Graph model (TemAtt) was the best-performing model for 1-day predictions and the Spatio-Temporal Graph model (SpaTempAtt) was best performing for 5-day predictions. Finally, all test models, including both of our Graph models, showed significant degradation in performance as sensitivity decreases.
Some high-level conclusions from our results show that our gap filled data have a positive impact on the predictions, it approximates missing values better and adds spatial information from other locations. Time series specialised models are better at predictions in short-term as LSTM and TemAtt architectures out-perform MLP and simpler models marginally.
While the results provide answers to some questions, there are may other unanswered ones. For example it is not clear why the TemAtt model outperformed the SpaTemAtt model. It may be because the inclusion of spatial info made SpaTemAtt weaker somehow. We were surprised by the comparatively poor performance of the LSTM-GNN. For us, the CCTV images added no value as FS3 featured outside the top-2 across all prediction intervals possibly because there is little overlap between CCTV and air quality data thus little for the models to train on. One other reason for this could be that additional trafic features do not really provide new information. Even if we assume air quality is afected by trafic, was our vehicle counting too naive or is there another explanation.
This work was supported in part by Science Foundation Ireland through the the Insight Centre for Data Analytics (SFI/12/RC/2289_P2) and the Centre for Research Training in Machine Learning (18/CRT/6183). We thank the organisers for running the task.