Traffic forecasting is crucial to intelligent transportation system, and very challenging due to the uncertainty and complexity of spatial-temporal dependencies in real-world traffic network. Many existing approaches use the pre-defined graph to model spatial correlations, but they fail to capture the latent spatial evolution. Then some dynamic graph-based methods are proposed to address this issue, however they separately model spatial and temporal dependencies without internal connection. In this paper, we propose a novel Dynamic gated Spatial Temporal Graph Neural Network (DSTGNN) for traffic forecasting, which can capture time-varying spatial correlations and temporal dependencies jointly. Besides, we apply gate mechanism into residual connection between extracted spatial and temporal features. Experimental results on two real-world traffic datasets have demonstrated the effectiveness of DSTGNN, and that the proposed DSTGNN can compete with state-of-the-art baselines.
1. Introduction 1
As a core component of Intelligent Transportation System (ITS)[
In early deep learning approaches, convolutional neural networks (CNNs) are used to extract spatial
correlations and RNNs are used to model temporal dependencies. However, CNNs are only suitable for
capturing spatial features in grid-data and perform poorly in non-Euclidean space. Recently, graph
neural networks (GNNs)[
To model both temporal and spatial dependencies, many early approaches [
In this paper, we proposed an end-to-end model named Dynamic gated Spatial Temporal Graph Neural Network (DSTGNN), which models spatial-temporal dependencies jointly to address the above issues. Specifically, we stack multiple the proposed dynamic gated spatial-temporal(DGST) blocks to extract spatial-temporal features from historical traffic data. Each DGST block consists of a dynamic graph convolution module for extracting dynamic and static spatial correlations, a temporal attention module for modeling time dynamics and a gated residual connection for interaction between extracted spatial and temporal features. The contribution of this paper can be summarized as: • We propose the dynamic gated spatial-temporal block that jointly models the spatial and temporal dependencies of traffic data, and then make prediction in a non-autoregressive way. • Experiments on two real-world traffic datasets are conducted to evaluate the effectiveness of our model, and results show that our model can compete with the state-of-the-arts.
As an efficient variant of CNNs on graph-structured data, graph convolutional networks (GCNs)
have been applied into various areas and achieved state-of-the-art results. In the spectral perspective
[
Attention mechanism has been extensively utilized in many domains such as natural language
processing and graph learning (GAT[16]). Recently, researchers apply attention mechanism to model
spatial-temporal dependencies for traffic forecasting. GMAN[
In this study, a traffic network is denoted as a directed weighted graph G = (V , E, A) , where V is a set of N =| V | nodes representing sensors in traffic network; E is a set of edges indicating the connectivity among the nodes; and A ∈ N×N is the weighted adjacency matrix of graph G representing the proximity measured by the Euclidean distances between sensors via Gaussian kernel.
At each time step t , the traffic conditions can be represented as a graph signal Xt ∈ N×C on graph G , where C is the number of traffic conditions observed such as traffic speed, traffic density and so on.
Given a traffic network graph G and traffic conditions of historical P time steps Xt−P+1,, Xt ∈ P×N×C observed by the N nodes, we aim to learn a function f to forecast the traffic conditions of the next F time steps over all nodes. The process can be formulated as:
Xˆ t+1,, Xˆ t+F = f ( X t−P+1,, X t ;G ) , (1) where Xˆ t+1,, Xˆ t+F ∈ F×N×C . 3.2.
The overall architecture of the proposed DSTGNN is shown in Figure 1, which consists of three main components including input layer, multiple stacked dynamic gated spatial-temporal (DGST) blocks and output layer. We first use two-layer fully-connected network before traffic data enters the model to project the data to high-dimension space, then the multiple stacked DGST blocks extract spatial and temporal dependencies jointly from input. Finally, the output layer transforms the features with spatial-temporal information from high-dimension space to traffic speed. Besides, DSTGNN predicts future traffic conditions in a multi-step manner. The detailed modules will be introduced later.
(2) (3) module (for DGC module in the first DGST block, X S(T0) = X′S ). We first view the temporal dimension of X′S as a batch dimension, and then X′S is transformed to the query, key and value subspaces by three different linear projection, obtaining QS′KS ,VS ∈ P×N×d ，formally:
QS = X S(lT−1)Wq , KS = X S(lT−1)Wk ,VS = X S(lT−1)W , v where Wq ,Wk ,Wv ∈ d×d are all learnable weight matrix, X S(Tl−1) ∈ P×N×d is input of DGC module in lth DGST block, and d is the dimension of feature space.
Next, we use ReLU (⋅) to sparse the dense adjacency matrix derived from dot product as follows: A = Re LU (QS KST ) + I N , where A ∈ N×N is the sparse spatial correlation matrix, IN is an identity matrix, and add it to enhance the self-connection ability. Furthermore, to capture both dynamic and static spatial correlations, we combine A and the predefined adjacency matrix A by element-wise product with broadcasting mechanism. A softmax(⋅) function is then adopted to normalize the combined adjacency matrix to avoid gradient disappearing or explosion, the final graph convolution operation can be expressed formally as:
HS(l) = softmax( A A)VS , element-wise product. 3.4.
As shown in Figure 1, we apply a gate function to regulate the flow of residual information, drawing inspiration from the gate mechanism in the Gated Recurrent Unit (GRU). How much the previous residual information can influence the following module is regulated by this gate mechanism. Specifically, a gated residual shortcut path is added to DGC module, fusing the spatial correlation of the output HS(l) and the input's spatial-temporal features X S(Tl−1) ∈ P×N×d in the lth DGST block with X S(T0) = X′S , which can be formulated as: (4) (5) (6) (7) where XT(l) ∈ P×N×d is the following temporal attention module's input, gs ∈ P×N×d denotes the gate, Wres ∈ d×d denotes the linear projection weight, and Ug ∈ d×d denotes the state-to-state weight matrix, is the element-wise product, and stands for the ⋅ non-linear activation function.
Similar to DGC module, we also added a gated residual shortcut path to the following temporal attention module, which combines the spatial correlation of input XT(l) with the temporal dependence of output HT(l) , building an interaction between the extracted spatial and temporal information that can be learned to extract spatial-temporal representations X S(Tl) ∈ P×N×d . This process can be written as： where HT(l) ∈ P×N×d is output of the following temporal attention module, and gt is the gate function. where HS(l) ∈ P×N×d denotes the output of the DGC module in the lth DGST block, and stands for QT = XT(l)Wq′ , KT = XT(l)Wk′ ,VT = XT(l)Wv′ ,
VT′ = LN softmax QT KTT VT + XT ,
d HT(l) = LN (FFN (VT′ ) + VT′ ) , X S(lT) = gt HT(l) + XT(l)Wres ,
gt =σ ( XT(l)U g ) , 3.5.
The temporal attention module takes the output XT(l) ∈ P×N×d of gated residual connection in DGC module as input and views the spatial dimension of XT(l) as a batch dimension. Then it uses the transformer-based encoder to model temporal dependence from XT(l) with spatial correlation. It mainly consists of attention aggregation and FFN refining, with the former highlighting relative temporal cues and the latter updating refined features. The output tensor HT(l) ∈ P×N×d of the temporal attention module can be computed as follows: where Wq′，W′，Wv′ ∈ d×d are all weight matrix, LN indicates layer normalization, and FFN is the feed k forward network. The above calculation can also be extended in a multi-head manner. 3.6.
The output layer regards the last DGST block's output X S(Tl) ∈ P×N×d as input, then transforms it to final prediction results Yˆ ∈ F×N×C . The proposed DSTGNN can be trained in an end-to-end style via backpropagation by minimizing the mean absolute error between predicted values and ground truths: (Θ) = 1 F N
Yˆt,n − Yt,n
F × N t=1n=1 where Θ denotes all the learnable parameters in our model, and denotes the ground truth. (8)
As detailed below, we evaluate our DSTGNN on two public real-world traffic datasets, namely
PeMSD7 and PEMS-BAY. PeMSD7 collects 2-month traffic data on 228 sensors during the weekdays
of May and June 2012 in California's District 7. PEMS-BAY contains six months of traffic data from
325 sensors in the Bay area from January 1st to May 31st, 2017. Following STGCN[
Metrices. To evaluate the performance of our model, we employ three metrics: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE).
Baselines. Our DSTGNN is compared to the following baselines: ARIMA[
Table 1 shows the experimental performance of DSTGNN and baselines for 15, 30 and 60 minutes
ahead prediction on the PeMSD7 and PEMS-BAY datasets. The prediction performance comparison
results show that DSTGNN can compete with the state-of-the-art methods in both long-term and
shortterm predictions on both datasets, while outperforming predefined graph-based spatial-temporal models,
namely STGCN[
In terms of short-term prediction (<= 30 min), DSTGNN outperforms STTN[
In this paper, we propose a novel framework named dynamic gated spatial temporal graph neural network (DSTGNN) for long and short-term traffic forecasting. In DSTGNN, we adopt the DGC module to precisely integrate both static and dynamic spatial correlations and at the same use temporal attention module to capture evolution cues in time series. Besides, we add the gated residual connection to the proposed DGST block for fusing extracted spatial and temporal features. The experiments on two real traffic datasets verifies the effectiveness of DSTGNN on modeling spatial-temporal correlations from time series. In the future, we will consider our DSTGNN for more general spatial-temporal structural graph sequence forecasting tasks, such as preference prediction in recommendation systems.
This work was supported by Wuhan Institute of Technology under Grant No.CX2021277. 7. References [12] Micha ̈el Defferrard, Xavier Bresson, and Pierre Van dergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 29, 2016. [13] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [14] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016. [15] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs.
Advances in neural information processing systems, 30, 2017. [16] Petar Veliˇckovi ́c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.