Data: a Review of Deep Neural Networks and the Trajectory Data Representations to Train Them

Abstract

information). Trajectory data combines the complexities of time series, spatial data, and (sometimes irrational) movement behavior. As data availability and computing power have increased, so has the popularity of deep learning from trajectory data. This paper aims to provide an overview of deep neural networks designed to learn from trajectory data, focusing on recent work published between 2020 and 2022. We take a data-centric approach and distinguish between deep learning models trained using dense trajectories (quasi-continuous tracking data), sparse trajectories (such as check-in data), and aggregated trajectories (crowd Deep learning, data engineering, movement data, trajectories

Keywords

• Representations

1. Introduction

Deep learning has become a popular approach for developing data-driven prediction, classification, and anomaly detection solutions. Work on deep learning from trajectory data is spread out over many domains, including but not limited to computer science, geography, geographic information science, urban planning, and ecology. Consequently, it covers many use cases and corresponding trajectory dataset types.

Trajectory datasets can be categorized according to the level of detail: from dense trajectories (quasi-continuous tracking data of individual movement) to sparse trajectories (such as check-in data of individuals), and finally, aggregated trajectories (crowd-level information, typically aggregated to edges/nodes in a mobility graph, to a grid, or to a set of points of interest)[ 1, 2, 3 ]. In many cases, the titles and abstracts of papers are not suficient to determine which type of trajectory data was used to train the deep learning model. While many papers start with dense trajectories, most convert them into sparse trajectories [ 4, 5, 6, 7, 8, 9, 10 ] or even aggregate them to crowd-level [11, 12, 13]. Common approaches to turning dense trajectories into sparse trajectories include: converting them into a sequence of stop locations [ 4, 7 ] Proceedings of the Workshop on Big Mobility Data Analytics (BMDA) co-located with EDBT/ICDT 2023 Joint Conference (March 28-31, or POI2Vec [17]. Another recent review [18] is dedicated to deep learning for trafic flow prediction models, which are primarily trained on aggregated trajectory data. However, to the best of our knowledge, there is no review paper that provides an overview of the diferent neural network architectures used to learn from trajectory data.

The goal of this work is to provide a first overview of the current state of neural networks / deep learning trained with trajectory data, structured by 1. Use case category (travel time/crowd flow/location predictions, location/trajectory classifications, anomaly detection), 2. Neural network architecture (CNN, RNN, LSTM, GNN, …), and 3. Trajectory data granularity (dense, sparse, aggregated) and representation. Therefore, this review does not include classic ML approaches and does not provide an exhaustive historical analysis of the field. Due to the page limit, this paper does not fit an exhaustive list of all relevant works published in recent years. However, we provide at least one paper for each use case and network combination we identified. We specifically reviewed publications at recent events, including SIGSpatial 20221 [ 6, 7, 19, 8, 20, 12, 21, 22 ], Sussex-Huawei Locomotion (SHL) Challenge 20212 at the ACM interna

1https://sigspatial2022.sigspatial.org/accepted-papers/

2http://www.shl-dataset.org/activity-recognition-challenge-2021/ data tional joint conference on pervasive and ubiquitous com- publications is not necessarily consistent. We, therefore, puting (UbiComp) [23], Trafic4cast challenge 2021 3 at define the most important terms and abbreviations in a NeurIPS [24], and Big Movement Data Analytics work- glossary at the end of this paper. shop BMDA 20214 at EDBT [11, 25, 26].

Even though we focus explicitly on deep learning, it is worth noting that deep learning may not always be 2. Representing Trajectory Data the best approach [27]. In particular, the SHL Challenge For Deep Learning Use Cases summary [23] shows that regular machine learning models outperform deep learning models on all three metrics: This review is structured around eight use case categories F1 score, train time, and test time. of deep learning from trajectory data, as shown in Fig

This review does not attempt to compare the per- ure 1. The following subsections describe the use cases, formance of diferent deep-learning approaches. Even neural network designs used to address them, and trajecthough there are some commonly used open datasets, tory data used to train these networks. Figure 2 provides such as the Porto taxi data5, the T-Drive taxi dataset6 and an overview of the diversity of identified trajectory repGeoLife dataset7, and the Gowalla check-in data8, cross- resentations. More details on the trajectory datasets and paper comparisons outside of dedicated data challenges the data engineering steps applied to the trajectory data are notoriously dificult. For example, “Despite the Porto before they are used as input to train the neural networks dataset’s original use as a standardized benchmark for are summarized in Tables 1-3. Some works (e.g. [28, 29]) open competition, design choices in subsequent work use additional data sources in combination with trajecmake cross-paper comparison dificult. Firstly, diferent tory data to train their models. These additional data papers often augment the dataset with their metadata not sources have been omitted from our review in favor of present in the original release, which may give some mod- clarity and conciseness. els an advantage over others independent of architecture or training design.” [20] 2.1. Location classification

Due to the large range of domains working on trajectory data analysis, the terminology used in diferent

6https://www.microsoft.com/en-us/research/publication/

t-drive-trajectory-data-sample/

7https://www.microsoft.com/en-us/research/publication/ geolife-gps-trajectory-dataset-user-guide/ 8http://snap.stanford.edu/data/loc-Gowalla.html

This use case category covers the classification of locations using patterns derived from movement data. The classification of regionally dominant movement patterns 34hhttttppss::////wwwwww..idaartaais.atoc.raiet/st.roarfgfi/cb4mcadsat2/12/0B2M1-DcoAm2p1eAtcitcieopnt/ed.html may be of interest in and of itself [9] or help with the 5https://www.kaggle.com/c/pkdd-15-predict-taxi-service-trajectoryc-lia/ ssification of POIs (e.g. ports [ 4 ]) or the classification of trip destinations [19].

To detect regionally dominant movement patterns, Yang et al. [9] use direction information and density maps to generate directional flow images. They convert the trajectories into images where each pixel contains the Wang et al. (2018) Buijse et al. (2021) Zhang et al. (2022) Liatsikou et al. (2021) Tritsarolis et al. (2021) Mehri et al. (2021) Liatsikou et al. (2021) Nguyen et al. (2022) Fan et al. (2022) Carroll et al. (2022) Musleh et al. (2022) Liao et al. (2018) Feng et al. (2018) Gao et al. (2019) Li et al. (2020) Hong et al. (2022) Tenzer et al. (2022) Rao et al. (2020) Yang et al. (2018) Chen et al. (2020) Yang et al. (2022) Singh et al. (2022) Li et al. (2021)

Altan et al. (2022) Derrow-Pinion et al. (2021)

Lippert et al. (2022) Buroni et al. (2021) Gao et al. (2022) Xue et al. (2022)

Lu (2021) Wang et al. (2022) Zhang et al. (2020) Simini et al. (2021)

NN design GEO-Conv CRNN VAE-like deep generative models

LSTM-based autoencoder

RNN LSTM RNN GRU RNN

SAN Transformers GRU+attention Transformers Hypernetworks (LSTMs) LSTM+GAN

CNN

RNN GCN+LSTM (T)GNN

LSTM GCN+GRU Transformers

CNN GAN MLP

Lyu et al. [19] train a plug-in memory network to predict trip purposes based on destination locations. Their model is trained using activity, origin, and destination matrices derived from OD data using a non-negative Tucker decomposition scheme.

2.2. Arrival time prediction

This use case category covers the prediction of travel times or arrival times, such as arrival time prediction in train networks [30] and street networks [31, 32].

Since travel time often depends on historical travel times at a given time of day, recurrent mechanisms are commonly used [31, 32, 30]. For example, Derrow-Pinion et al. [31] train GNNs on aggregated trajectories to provide travel time predictions in Google Maps. The GNN graph consists of segment and supersegment-level embedding vectors. Nodes store street segment-level data (average real-time and historical segment travel speeds and times, segment length, and road class), while edges store supersegment-level data (real-time supersegment travel times).

In contrast, Wang et al. [32] introduces the GEOconvolutional network layer (GEO-Conv, also used by Buijse et la. [30]), which is trained on dense trajectories stating that “directly mapping the GPS coordinates into grid cells is not accurate enough to represent the original spatial information in the data”. The proposed GEO-Conv layer takes dense trajectories as input and applies a nonlinear mapping of each trajectory (latitude and longitude) point, followed by a GEO-Conv step with multiple kernels. The resulting feature map of local paths is appended with a final column of distances of the local paths.

2.3. Trafic volume prediction

Trajectory data representation

Raw trajectories r e s n e D Resampled trajectories

Resampled sequences of space & time deltas Generalized trajectories

Discretized trajectories l e v e ll a u d ii v d n I r e s r a p S d e t a g e r g g A /l e v e l d w o r C

Location (sequence)

embeddings Rasterized trajectories (trajectory images & directional flow images)

Temporal graphs Time series on street network edges / POIs Natural language sentences Traffic movies

OD matrices Decomposed Space

Activity Matrix

Lyu et al. (2022)

Memory network

This use case category covers trafic or crowd predictions of volumes or flows, e.g., predicting trafic volume on street segments [11, 18], human activity at specific POIs Figure 2: Overview of trajectory data representations used [28] and metropolitan areas [33, 24], or predicting animal to train neural networks movement dynamics [34].

Aggregated trajectory data in the form of trafic movies is provided in the Trafic4Cast 2021 competition which challenged participants to predict trafic under conditions directional flows. They use a CNN to classify the input of temporal domain shift (Covid-19 pandemic) and spatial image patterns and detect the dominant regional move- shift (transfer to entirely new cities). Lu [ 24] won this ments. challenge using CNN (U-Net) and multi-task learning.

An approach that makes more use of temporal informa- Their multi-task learning approach randomly samples tion is presented by Altan et al. [ 4 ]. They use a temporal from all available cities and trains the U-Net model to GNN (TGNN) to distinguish gateway ports from actual jointly predict the future trafic states for diferent cities. ports using AIS vessel movement data. After extracting Wang et al. [12] follow this trafic movie approach as well the ports (nodes) from the raw AIS messages using DB- by aggregating individual-level trajectories into a grid SCAN, they extract trips between consecutive ports and with inflow referring to the total number of incoming build a graph for each time step to generate the time- trafic entering this region from other regions during a ordered daily graph sequence for the TGNN. given time interval and outflow representing the total number of trafic leaving the region. Zhang et al. [ 13] points”. also follow the trafic movie approach, creating temporal grids of average trafic speed and taxi inflow per cell. 2.5. (Sub)trajectory classification

Li et al. [33] build a graph for their GCN by aggregating CDR data and representing spatial statistical units as This use case category considers the classification of comnodes and their relationship (physical distance, physical plete trajectories [10] or sub-trajectories [ 5, 23 ] in order movement, phone calls) as edges. Similarly, Lippert et to learn more about diferent vessel and human moveal. [34] build temporal graphs from bird migration data ment patterns. where nodes represent radar locations, and edges repre- By splitting trajectories into sub-trajectories, more sent the flows between the Voronoi tessellation cells of ifne-grained analyses are possible. Typical applications the radar locations. include the detection of movement types, such as ship

Finally, Buroni et al. [11] provide a tutorial using vehi- maneuvers [ 5 ] or the detection of transportation and cle counts derived from GPS tracks to build and train a Di- locomotion modes of smartphone users [23]. Chen et rect LSTM encoder-decoder model. The model is trained al. [ 5 ] generate colour-coded trajectory images from ship to predict counts of vehicles per network edge per time AIS data, where each pixel is assigned one of three colstep for the Belgian motorway network. Similarly, Gao ors according to the movement type (static, normal, maet al. [28] use their GPS tracks to count vehicles per POI neuvering). These trajectory images are used to train a per time step (hourly) to train a GCN+GRU model that CNN-based ship maneuver classifier. To identify diferent predicts these visit counts. And Xue et al. [21] propose a movement modes (i.e. still, walk, run, bike, car, bus, train, translator called mobility prompting which converts daily and subway) from smartphone data, the SHL Challenge POI visit counts into natural language sentences so they winner uses an AdaNet algorithm, a Tensorflow-based can use (and fine-tune) pre-trained NLP models such as framework for learning NN models and ensembling modBert, RoBERTa, GPT-2, and XLNet to predict these visit els to obtain even better models [ 46, 23 ]. counts. An example of the classification of complete trajectories is the recognition of ship types introduced by Yang 2.4. Trajectory prediction/imputation et al. [10]. It relies on the same technique as [ 5 ] for transforming the raw AIS trajectory data into colour-coded This use case category covers the prediction of trajecto- trajectory images. The resulting images show characterries in artificial [ 35], urban [ 6 ], and maritime environ- istic trajectory patterns, which can be used to identify ments [36, 26], as well as imputation of trajectories [ 8 ]. the ship vessel type with a CNN classifier.

Mehri et al. [36] generalize AIS trajectories using context-aware piecewise linear segmentation before feed- 2.6. Next location / final destination ing them into their LSTM three vertices at a time. This prediction enables their model to perform short-term trajectory predictions with high spatial detail. Tritsarolis et al. [26], on This use case category covers the prediction of the next the other hand, represent trajectories by their composi- locations or final destinations of trips [ 38, 37, 7, 39, 40, tion of diferences in space Δ , Δ , and time Δ for the 41, 21, 20]. Besides GPS tracks, a commonly used data input of their RNN-based models to predict the vessel’s source in this category are social media check-ins (e.g., position at time Δ + 1 . from Foursquare). The task then becomes to predict the

Fan et al. [ 6 ] discretize mobile phone GPS trajectories next check-in location (e.g., a POI). using the H3 hexagonal grid and use the grid cell se- Attention mechanisms have proven to be a popular quences to train their GRU. The resulting model is used approach for next location prediction. Gao et al. [37] to predict cell sequences which are afterward used to train VANext, a semi-supervised network trajectory consearch for similar high-resolution trajectories, which are volutional network, on check-in data. They convert each returned as the final trajectory prediction. individual user’s trajectory (check-in/POI sequence) into

Carroll et al. [35] use synthetic discrete movement sequences in a minimalistic grid world environment. Their 9https://www.kaggle.com/datasets/giobbu/belgium-obu transformers are trained on trajectories as sequences of 11https://coast.noaa.gov/htdata/CMSP/AISDataHandler/2017/ states, actions, and return-to-go tokens to predict tra- index.html jectories. Another work using transformers to impute 1132hhttttppss::////zseitneos.dgoo.oorggle/r.ceocmor/ds/i4te4/9y8a4n1g0dingqi/home/ trajectories is Musleh et al. [ 8 ]. They propose TrajBERT, foursquare-dataset?pli=1 a model trained using H3-discretized (tokenized) GPS 14http://www.start.umd.edu/gtd/ tracks. They “down-sample the trajectories by dropping 15https://github.com/bigdata-ufsc/petry-2020-marc/tree/master/ three-quarters of the points of each trajectory and then data1/6fhotutprssq:/u/darreiv_en.ygcoogle.com/file/d/ run TrajBERT to fill the gaps by imputing the missing 1rLJz5E0igbrmAnmnDmazdBl97UuQ0sch/view?usp=sharing Trafic volume prediction Lu (2021) CNN Trafic movies for 10 cities in 2019+2020 with [24] (U-Net) 8 dynamic channels encoding trafic speed and volume per direction and 9 static channels encoding the properties of the road maps Wang et CNN Taxi trajectories from TaxiBJ in Beijing for 17 al. (2022) months and bike trajectories from BikeNYC [12] in New York City for 6 months Mobile phone GPS tracks in Switzerland from the Green Class (GC) study covering 139 participants for a year and from the yumuv study covering 498 participants for 3 months Porto taxi tracks5 covering 1.7mio trips by 442 taxis in Porto, Portugal for 12 months Location sequences and activity/location graphs are converted into embeddings for MCARNN Location sequences are converted into timespecific and distance-specific transition matrices for ST-RNN Location sequences are generated from GPS tracks by first filtering stay locations with a stay duration >25min and then spatially aggregating stays into locations. These location sequences are converted into location, time, day, and mode embeddings which are fed into the transformers Sequences of trajectory points are converted to sequences of spatial embeddings via a geospatial encoding mechanism

Trajectories are down-sampled to 60s and represented as a sequence of vectors (lat, lon) and clipped to the first nine points to fit the autoencoder requirements Trajectories are down-sampled to 600s and converted to a four hot vectors (lat, lon, SOG, COG) with the resolutions of 0.01° for lat/lon, 1 knot for SOG, and 5° for COG.

Trajectories are resampled and interpolated at the 60s and converted into a graph with nodes representing turning points for the vessel trajectories and edges representing the sea lanes traveled by vessels Trajectories are processed by a trajectory encoding model covering trajectory point encodings (location, temporal and categorical attributes) and trajectory padding (to ensure that all trajectories have the same length) A coordinate encoding MLP converts twodimensional points into a high-dimensional representation. Then, a Bidirectional LSTM is used to encode all representations with forward and backward information for a time step Input to the model is of the origin & destination location as well as the distance between origin and destination. The output of the model is the probability to observe a trip between two locations.

Synthetic data generation Rao et LSTM- Foursquare check-ins15 in NYC covering al. (2020) GAN 193 users with 3k trajectories and 67k records [ 44 ] Zhang et al. (2022) [22] Simini et al. (2021) [ 45 ]

VAE-like deep generative models MLP

Porto taxi tracks5; T-Drive6 data consisting of 10.3k taxis for one week; and Gowalla check-ins8 England & Italy commuting flows, NY State flows16 including origin & destination geographic unit and estimated population flows between two geographic units sequence embeddings using a causal embedding method as well as graph embeddings from location-location and (similar to a high-order Markov Process). The result- activity-location graphs and train their MCARNN multiing embeddings are the input for their GRU to learn task context-aware recurrent neural network to solve the trajectory patterns. They further apply attention both activity and location prediction tasks. to the embeddings for predicting the user’s next POI. Other works use neural networks for dimensionality Feng et al. [38] tailor two attention mechanisms to gen- reduction and for creating embeddings. Liu et al. [41] inerate independent latent vectors from large and sparse corporate time and distance-specific transition matrices trajectories. These embeddings are then fed into their as temporal and spatial embeddings generated by RNNs. DeepMove GRUs and a historical attention module. The Hong et al. [ 7 ] reduce the dimensions of trajectories using learned attention weights can intuitively explain the a multilayered embedding approach for transformers to prediction based on the user’s history of movement be- predict next location and travel mode. Tenzer et al. [20] havior. Li et al. [39] introduce a spatio-temporal self- generate two geospatial and temporal embeddings by attention network (STSAN ). They generate trajectory 1. combining the random picking and the nearest neighembeddings by concatenating the temporal (activity se- bor to create sequences of spatial embeddings and 2. usquence), spatial (distance matrix of locations), and loca- ing a sinusoidal embedding to convert the timesteps to tion attentions (location sequence and their categories). temporal vectors. They train a hyper network to learn They feed these embeddings through a softmax layer and to change its weights in response to these embeddings. predict the user’s next POI. They use a federated learning setting to tackle the heterogeneity problem. Liao et al. [40] generate embeddings from location sequences

2.7. Anomaly detection

This use case category covers anomalous trajectory detection. Since the definition of anomalies is often contextdependent, ground truth labeled data is rare. Therefore, anomaly detection approaches often resort to trying to identify trajectories that are diferent compared to previously observed trajectories based on some spatial, spatiotemporal, or other metrics. Alternatively, researchers resort to using synthetically generated anomalies [25].

Liatsikou et al. [25] developed an LSTM-based network for the automatic detection of movement anomalies, such as the detection of synthetic anomalies in taxi trajectories. Since the autoencoder requires inputs of a certain fixed length, all trajectories are clipped to nine points (and shorter ones discarded).

The GeoTrackNet [42] is a model for maritime trajectory anomaly detection, which consists of a probabilistic RNN-based (Recurrent Neural Network) representation of AIS tracks and a contrario detection [ 47 ]. Detected anomalies were evaluated by AIS experts.

Singh et al. [ 43 ] present an anomaly detection system based on RNN regression models to detect anomalous trajectories, on-of switching, and unusual turns. Again, a quantitative accuracy analysis is not feasible due to the lack of ground truth data.

2.8. Synthetic data generation

This category covers the generation of synthetic movement data, such as synthetic trajectories [ 22, 44 ] and synthetic flows [ 45 ].

Rao et al. [ 44 ] focus on GeoAI-trajectory privacy protection. For this, they develop an end-to-end deep LSTMTrajGAN model to generate privacy-preserving synthetic trajectory data for data sharing and publication.

Simini et al. [ 45 ] developed an MLP model (denoted Deep Gravity) to generate mobility flow probabilities. They evaluated Deep Gravity on mobility flows in England, Italy, and New York State and achieved a good performance even for regions with no data available for training.

Zhang et al. [22] propose an end-to-end trajectory generation model for generating synthetic trajectories. The design of the model is VAE-like encoders (e.g., Globalsemantics encoder: MLPs & Bidirectional LSTM) and decoders (e.g., a prior generator based on variational recurrent structure generates noise at time by considering the noise at the previous time step).

3. Conclusion and outlook

In this work, we reviewed deep learning-based research focussing on mobility data. In most cases, even if trajectory data is used in the process, it is not ingested directly for training the neural networks. Instead, data engineering steps are applied that convert trajectories into more compact representations of individual trajectories (sparse trajectories) or aggregations of multiple trajectories. This aggregated trajectory data is commonly presented as time series of vectors, graphs, or images (movies).

On the deep learning side, we expect the popularity of GNNs to increase. For example, the Trafic4cast challenge (in its 4th year, 2022) is moving from (image/video-based) trafic forecasting to graph-based representations. Additionally, AutoML methods (e.g., AdaNet used by the SHL Challenge winner [23]) will allow users with limited DL expertise to build competitive DL models.

As far as data engineering and development is concerned, we expect further uptake of trajectory analysis libraries, such as Trackintel17 (e.g., used by [ 7 ]), MovingPandas18 (e.g., used by [36]) and scikit-mobility19 (e.g., used by [ 45 ]) since these libraries implement many common trajectory generalization, aggregation, and analysis methods and aim at a long(er) term availability. This is an important next step, as the implementations summarized in Figure 2.8 have not been substantially updated/maintained since being published. This does not only reduce the likelihood of reuse but also will lead to security issues down the road.

Future research should address the issues of model transferability, benchmark availability, and model explainability. Current work rarely addresses the issue of model transferability. Since most existing global ML models “cannot perform well locally, or be transferred to study similar problems in other regions”[ 48 ], transferability should be considered when evaluating or comparing models. Additionally, developed models, even for the same application and trajectory type, are dificult to evaluate (e.g., due to the lack of ground truth for anomaly detection) and to compare due to diferent datasets and applied metrics. Therefore, more open datasets are needed.

Finally, to better understand the why and how of using neural networks for a specific application, explainability should play a more crucial role in model development. 17https://github.com/mie-lab/trackintel 18http://movingpandas.org 19https://scikit-mobility.github.io/scikit-mobility/ //dl.acm.org/doi/10.1145/3557915.3565529. doi:10. //dl.acm.org/doi/10.1145/3139958.3140054. doi:10. 1145/3557915.3565529. 1145/3139958.3140054. [9] C. Yang, G. Gidofavi, Classification of regional [17] S. Feng, G. Cong, B. An, Y. M. Chee, POI2Vec: dominant movement patterns in trajectories with a geographical latent representation for predicting convolutional neural network, in: Martin Raubal, future visitors, in: Proc. of the Thirty-First AAAI Shaowen Wang, Mengyu Guo, David Jonietz, Peter Conf. on Artificial Intelligence, 2017, pp. 102–108. Kiefer (Eds.), Spatial Big Data and Machine Learn- [18] A. A. Kashyap, S. Raviraj, A. Devarakonda, S. R. ing in GIScience, Melbourne, Australia, 2018. Nayak K, S. K V, S. J. Bhat, Trafic flow [10] T. Yang, X. Wang, Z. Liu, Ship Type Recognition prediction models – A review of deep learnBased on Ship Navigating Trajectory and Convo- ing techniques, Cogent Engineering 9 (2022) lutional Neural Network, Journal of Marine Sci- 2010510. URL: https://www.tandfonline.com/doi/ ence and Engineering 10 (2022) 84. URL: https: full/10.1080/23311916.2021.2010510. doi:10.1080/ //www.mdpi.com/2077-1312/10/1/84. doi:10.3390/ 23311916.2021.2010510.

jmse10010084. [19] S. Lyu, T. Han, Y. Nishiyama, K. Sezaki, T. Kusakabe, [11] G. Buroni, G. Bontempi, K. Determe, A tutorial A plug-in memory network for trip purpose classion network-wide multi-horizon trafic forecast- ifcation, in: Proc. of the 30th Intern. Conf. on Ading with deep learning., in: Workshop Proc. of vances in Geographic Information Systems, ACM, the EDBT/ICDT 2021 Joint Conference, Nicosia, Seattle Washington, 2022, pp. 1–12. URL: https: Cyprus, 2021. URL: https://ceur-ws.org/Vol-2841/ //dl.acm.org/doi/10.1145/3557915.3560969. doi:10.

BMDA_6.pdf. 1145/3557915.3560969. [12] C. Wang, Y. Liang, G. Tan, Periodic residual learn- [20] M. Tenzer, Z. Rasheed, K. Shafique, N. Vasconcelos, ing for crowd flow forecasting, in: Proc. of the 30th Meta-learning over time for destination prediction Intern. Conf. on Advances in Geographic Informa- tasks, in: Proc. of the 30th Intern. Conf. on Adtion Systems, ACM, Seattle Washington, 2022, pp. vances in Geographic Information Systems, ACM, 1–10. URL: https://dl.acm.org/doi/10.1145/3557915. Seattle Washington, 2022, pp. 1–10. URL: https: 3560947. doi:10.1145/3557915.3560947. //dl.acm.org/doi/10.1145/3557915.3560980. doi:10. [13] Y. Zhang, Y. Li, X. Zhou, X. Kong, J. Luo, Curb-GAN: 1145/3557915.3560980.

Conditional Urban Trafic Estimation through [21] H. Xue, B. P. Voutharoja, F. D. Salim, LeveragSpatio-Temporal Generative Adversarial Networks, ing language foundation models for human moin: Proc. of the 26th ACM SIGKDD Intern. Conf. on bility forecasting, in: Proc. of the 30th Intern. Conf. Knowledge Discovery & Data Mining, ACM, Vir- on Advances in Geographic Information Systems, tual Event CA USA, 2020, pp. 842–852. URL: https: ACM, Seattle Washington, 2022, pp. 1–9. URL: https: //dl.acm.org/doi/10.1145/3394486.3403127. doi:10. //dl.acm.org/doi/10.1145/3557915.3561026. doi:10. 1145/3394486.3403127. 1145/3557915.3561026. [14] G. Mai, K. Janowicz, Y. Hu, S. Gao, B. Yan, R. Zhu, [22] L. Zhang, L. Zhao, D. Pfoser, Factorized deep generL. Cai, N. Lao, A review of location encoding for ative models for end-to-end trajectory generation GeoAI: methods and applications, Intern. Jour- with spatiotemporal validity constraints, in: Proc. nal of Geographical Information Science 36 (2022) of the 30th Intern. Conf. on Advances in Geographic 639–673. URL: https://www.tandfonline.com/doi/ Information Systems, ACM, Seattle Washington, full/10.1080/13658816.2021.2004602. doi:10.1080/ 2022, pp. 1–12. URL: https://dl.acm.org/doi/10.1145/ 13658816.2021.2004602. 3557915.3560994. doi:10.1145/3557915.3560994. [15] M. Zhu, W. Chen, J. Xia, Y. Ma, Y. Zhang, Y. Luo, [23] L. Wang, M. Ciliberto, H. Gjoreski, P. Lago, K. MuZ. Huang, L. Liu, Location2vec: A Situation-Aware rao, T. Okita, D. Roggen, Locomotion and TransRepresentation for Visual Exploration of Urban Lo- portation Mode Recognition from GPS and Racations, IEEE Transactions on Intelligent Trans- dio Signals: Summary of SHL Challenge 2021, portation Systems 20 (2019) 3981–3990. URL: https: in: Adjunct Proc. of the 2021 ACM Intern. Joint //ieeexplore.ieee.org/document/8664648/. doi:10. Conf. on Pervasive and Ubiquitous Computing, 1109/TITS.2019.2901117. ACM, Virtual USA, 2021, pp. 412–422. URL: https: [16] B. Yan, K. Janowicz, G. Mai, S. Gao, From ITDL //dl.acm.org/doi/10.1145/3460418.3479373. doi:10. to Place2Vec: Reasoning About Place Type Sim- 1145/3460418.3479373. ilarity and Relatedness by Learning Embeddings [24] Y. Lu, Learning to Transfer for Trafic ForecastFrom Augmented Spatial Contexts, in: Proc. of ing via Multi-task Learning (2021). URL: https: the 25th ACM SIGSPATIAL Intern. Conf. on Ad- //arxiv.org/abs/2111.15542. doi:10.48550/ARXIV. vances in Geographic Information Systems, ACM, 2111.15542.

Redondo Beach CA USA, 2017, pp. 1–10. URL: https: [25] M. Liatsikou, S. Papadopoulos, L. Apostolidis, I. Kompatsiaris, A Denoising Hybrid Model for full/10.1080/13658816.2021.1912347. doi:10.1080/ Anomaly Detection in Trajectory Sequences., in: 13658816.2021.1912347.

Workshop Proc. of the EDBT/ICDT 2021 Joint [34] F. Lippert, B. Kranstauber, P. D. Forré, E. E. Conference, Nicosia, Cyprus, 2021. URL: https:// van Loon, Learning to predict spatiotempoceur-ws.org/Vol-2841/BMDA_4.pdf. ral movement dynamics from weather radar net[26] A. Tritsarolis, E. Chondrodima, P. Tampakis, works, Methods in Ecology and Evolution (2022) A. Pikrakis, Online Co-movement Pattern Pre- 2041–210X.14007. URL: https://onlinelibrary.wiley. diction in Mobility Data, in: Workshop Proc. of com/doi/10.1111/2041-210X.14007. doi:10.1111/ the EDBT/ICDT 2021 Joint Conference, Nicosia, 2041-210X.14007.

Cyprus, 2021. URL: https://ceur-ws.org/Vol-2841/ [35] M. Carroll, J. Lin, O. Paradise, R. Georgescu, M. Sun, BMDA_15.pdf. D. Bignell, S. Milani, K. Hofmann, M. Hausknecht, [27] L. Grinsztajn, E. Oyallon, G. Varoquaux, Why do A. Dragan, S. Devlin, Towards Flexible Inference tree-based models still outperform deep learning in Sequential Decision Problems via Bidirectional on tabular data?, 2022. URL: http://arxiv.org/abs/ Transformers, 2022. URL: http://arxiv.org/abs/2204. 2207.08815. 13326. [28] Y. Gao, Y. Chiang, X. Zhang, M. Zhang, Trafic [36] S. Mehri, A. A. Alesheikh, A. Basiri, A Contexvolume prediction for scenic spots based on tual Hybrid Model for Vessel Movement Predicmulti‐source and heterogeneous data, Trans- tion, IEEE Access 9 (2021) 45600–45613. URL: https: actions in GIS 26 (2022) 2415–2439. URL: https: //ieeexplore.ieee.org/document/9380635/. doi:10. //onlinelibrary.wiley.com/doi/10.1111/tgis.12975. 1109/ACCESS.2021.3066463.

doi:10.1111/tgis.12975. [37] Q. Gao, F. Zhou, G. Trajcevski, K. Zhang, T. Zhong, [29] J. Rao, S. Gao, M. Li, Q. Huang, A privacy‐pre- F. Zhang, Predicting Human Mobility via Variaserving framework for location recommendation tional Attention, in: The World Wide Web Conf. using decentralized collaborative machine learn- on - WWW ’19, ACM Press, San Francisco, CA, ing, Transactions in GIS 25 (2021) 1153–1175. USA, 2019, pp. 2750–2756. URL: http://dl.acm.org/ URL: https://onlinelibrary.wiley.com/doi/10.1111/ citation.cfm?doid=3308558.3313610. doi:10.1145/ tgis.12769. doi:10.1111/tgis.12769. 3308558.3313610. [30] B. J. Buijse, V. Reshadat, O. W. Enzing, A [38] J. Feng, Y. Li, C. Zhang, F. Sun, F. Meng, A. Guo, Deep Learning-Based Approach for Train Arrival D. Jin, DeepMove: Predicting Human MobilTime Prediction, in: H. Y. et al. (Ed.), Intel- ity with Attentional Recurrent Networks, in: ligent Data Engineering and Automated Learn- Proc. of the 2018 World Wide Web Conf. on ing – IDEAL 2021, volume 13113, Springer, World Wide Web - WWW ’18, ACM Press, Lyon, Cham, 2021, pp. 213–222. URL: https://link.springer. France, 2018, pp. 1459–1468. URL: http://dl.acm.org/ com/10.1007/978-3-030-91608-4_22. doi:10.1007/ citation.cfm?doid=3178876.3186058. doi:10.1145/ 978-3-030-91608-4_22, series Title: Lecture 3178876.3186058.

Notes in Computer Science. [39] A. Li, S. Wang, W. Li, S. Liu, S. Zhang, Predicting [31] A. Derrow-Pinion, J. She, D. Wong et al., ETA Pre- Human Mobility with Federated Learning, in: Proc. diction with Graph Neural Networks in Google of the 28th Intern. Conf. on Advances in Geographic Maps, in: Proc. of the 30th ACM Intern. Conf. Information Systems, ACM, Seattle WA USA, 2020, on Information & Knowledge Management, ACM, pp. 441–444. URL: https://dl.acm.org/doi/10.1145/ Virtual Event Queensland Australia, 2021, pp. 3397536.3422270. doi:10.1145/3397536.3422270. 3767–3776. URL: https://dl.acm.org/doi/10.1145/ [40] D. Liao, W. Liu, Y. Zhong, J. Li, G. Wang, Predicting 3459637.3481916. doi:10.1145/3459637.3481916. activity and location with multi-task context aware [32] D. Wang, J. Zhang, W. Cao, J. Li, Y. Zheng, When recurrent neural network, in: Proc. of the 27th Will You Arrive? Estimating Travel Time Based Intern. Joint Conf. on Artificial Intelligence, 2018, on Deep Neural Networks, Proc. of the AAAI pp. 3435–3441.

Conference on Artificial Intelligence 32 (2018). [41] Q. Liu, S. Wu, L. Wang, T. Tan, Predicting the URL: https://ojs.aaai.org/index.php/AAAI/article/ next location: A recurrent model with spatial and view/11877. doi:10.1609/aaai.v32i1.11877. temporal contexts, in: Thirtieth AAAI conference [33] M. Li, S. Gao, F. Lu, K. Liu, H. Zhang, W. Tu, Pre- on artificial intelligence, 2016. diction of human activity intensity using the in- [42] D. Nguyen, R. Vadaine, G. Hajduch, R. Garello, R. Fateractions in physical and social spaces through blet, GeoTrackNet —A Maritime Anomaly Detector graph convolutional networks, Intern. Journal Using Probabilistic Neural Network Representation of Geographical Information Science 35 (2021) of AIS Tracks and A Contrario Detection, IEEE 2489–2516. URL: https://www.tandfonline.com/doi/ Transactions on Intelligent Transportation Systems

Acknowledgments

This work is mainly funded by the EU’s Horizon Europe research and innovation program under Grant No. 101070279 MobiSpaces and No. 101021797 STARLIGHT. Glossary • AE – Autoencoder • AIS – Automatic Identification System • CDR – Call Detail Records • CNN – Convolutional Neural Network • COG – Course over ground • DNN – Deep Neural Network • DL – Deep Learning • GAN – Generative Adversarial Network • GeoAI – Geospatial Artificial Intelligence • GIS – Geographic Information Science • GNN – Graph Neural Network • GPS – Global Positioning System, often used synonymously for all GNSS (incl. Galileo, GLONASS, and Beidou) • LSTM – Long Short-Term Memory • MLP – Multilayer Perceptron • NLP – Natural Language Processing • OBU – On-board Unit • OD – Origin-Destination • POI – Point of Interest • RNN – Recurrent Neural Network • SAE – Stacked Autoencoder • SAN – Self-Attention Network • SOG – Speed over ground • VAE – Variational AutoEncoder

References

[1] G. Andrienko , N. Andrienko , P. Bak , D. Keim , S. Kisilevich , S. Wrobel , A conceptual framework and taxonomy of techniques for analyzing movement , Journal of Visual Languages & Computing 22 ( 2011 ) 213 - 232 . URL: https://www.sciencedirect.com/science/ article/pii/S1045926X11000139. doi:https: //doi.org/10.1016/j.jvlc. 2011 . 02 .003.

[2] M. P. Konzack , Trajectory analysis: Bridging algorithms and visualization, Ph.D. thesis, Technische Universiteit Eindhoven , 2018 .

[3] Y. Zheng , Trajectory Data Mining: An Overview , ACM Transactions on Intelligent Systems and Technology 6 ( 2015 ) 1 - 41 . URL: http://dl.acm.org/ citation.cfm?doid= 2764959 .2743025. doi: 10 .1145/ 2743025.

[4] D. Altan , M. Etemad , D. Marijan , T. Kholodna, Discovering Gateway Ports in Maritime Using Temporal Graph Neural Network Port Classification , 2022 . URL: http://arxiv.org/abs/2204.11855.

[5] X. Chen , Y. Liu , K. Achuthan , X. Zhang, A ship movement classification based on Automatic Identification System (AIS) data using Convolutional Neural Network , Ocean Engineering 218 ( 2020 ) 108182 . URL: https://linkinghub.elsevier. com/retrieve/pii/S0029801820311124. doi: 10 .1016/ j.oceaneng. 2020 . 108182 .

[6] Z. Fan , X. Yang , W. Yuan , R. Jiang , Q. Chen , X. Song , R. Shibasaki , Online trajectory prediction for metropolitan scale mobility digital twin , in: Proc. of the 30th Intern. Conf. on Advances in Geographic Information Systems , ACM, Seattle Washington, 2022 , pp. 1 - 12 . URL: https://dl.acm.org/doi/10.1145/ 3557915.3561040. doi: 10 .1145/3557915.3561040.

[7] Y. Hong , H. Martin , M. Raubal , How do you go where?: improving next location prediction by learning travel mode information using transformers , in: Proc. of the 30th Intern. Conf. on Advances in Geographic Information Systems , ACM, Seattle Washington, 2022 , pp. 1 - 10 . URL: https: //dl.acm.org/doi/10.1145/3557915.3560996. doi: 10 . 1145/3557915.3560996.

[8] M. Musleh , Towards a unified deep model for trajectory analysis , in: Proc. of the 30th Intern. Conf. on Advances in Geographic Information Systems , ACM, Seattle Washington, 2022 , pp. 1 - 2 . URL: https: 23 ( 2022 ) 5655 - 5667 . URL: https://ieeexplore.ieee. org/document/9353410/. doi: 10 .1109/TITS. 2021 . 3055614 .

[43] S. K. Singh , J. S. Fowdur , J. Gawlikowski , D. Medina , Leveraging Graph and Deep Learning Uncertainties to Detect Anomalous Maritime Trajectories , IEEE Transactions on Intelligent Transportation Systems 23 ( 2022 ) 23488 - 23502 . URL: https://ieeexplore.ieee. org/document/9839418/. doi: 10 .1109/TITS. 2022 . 3190834 .

[44] J. Rao , S. Gao , Y. Kang , Q. Huang , LSTM-TrajGAN: A Deep Learning Approach to Trajectory Privacy Protection , in: 11th Intern. Conf. on Geographic Information Science (GIScience 2021 ) -Part I , Poznan/online, 2020 , pp. 12 : 1 - 12 : 17 . URL: http://arxiv. org/abs/ 2006 .10521.

[45] F. Simini , G. Barlacchi, M. Luca , L. Pappalardo , A Deep Gravity model for mobility flows generation , Nature Communications 12 ( 2021 ) 6576 . URL: https: //www.nature.com/articles/s41467-021-26752-4. doi: 10 .1038/s41467- 021- 26752- 4.

[46] C. Cortes , X. Gonzalvo , V. Kuznetsov , M. Mohri , S. Yang, AdaNet: Adaptive Structural Learning of Artificial Neural Networks , in: D. Precup , Y. Teh (Eds.), Proc. of the 34th Intern. Conf. on Machine Learning , volume 70 of Proc. of Machine Learning Research, PMLR , 2017 , pp. 874 - 883 . URL: https:// proceedings.mlr.press/v70/cortes17a.html.

[47] A. Desolneux , L. Moisan , J.-M. Morel , From gestalt theory to image analysis: a probabilistic approach , volume 34 , Springer Science & Business Media , 2007 .

[48] K. Janowicz , R. Zhu , J. Verstegen , G. McKenzie , B. Martins , L. Cai, Six GIScience Ideas That Must Die , AGILE: GIScience Series 3 ( 2022 ) 1 - 8 . URL: https://agile-giss.copernicus.org/articles/3/7/ 2022/. doi: 10 .5194/agile- giss- 3- 7- 2022.