=Paper=
{{Paper
|id=Vol-3632/ISWC2023_paper_501
|storemode=property
|title=Railway Track Video Knowledge Base
|pdfUrl=https://ceur-ws.org/Vol-3632/ISWC2023_paper_501.pdf
|volume=Vol-3632
|authors=Kai Herbst,Dirk Friedenberger,Rene Krieg
|dblpUrl=https://dblp.org/rec/conf/semweb/HerbstFK23
}}
==Railway Track Video Knowledge Base==
https://ceur-ws.org/Vol-3632/ISWC2023_paper_501.pdf
Railway track video Knowledge Base
Kai Herbst1,∗ , Rene Krieg1 and Dirk Friedenberger1
1
DB Systel GmbH, Frankfurt, Germany
Abstract
A video knowledge graph and a corresponding ontology was created based on railway track videos and
infrastructure data. It was shown that the heterogeneous and distributed infrastructure and track data
in the railway company, enriched with visualisations such as images and videos, implemented as an
RDF/OWL graph, fulfill the required use cases and increases business value. These use cases include
tasks such as being able to quickly query the graph for a videos in which specific infrastructure elements
occur or being able to search for video sequences that start or end with certain elements.
Keywords
knowledge graph, railway track videos, railway infrastructure, RDF graph, OWL ontology
1. Motivation
Our team, as part of a railway company, is responsible for recording, processing, and providing
videos of the German railway network. Video processing includes tasks such as anonymizing
individuals and license plates appearing in the videos. Additionally, the recordings need to
be associated with corresponding train ride data and prepared before being made available
for various projects. This process involves working with a variety of data in different formats
from different sources, stored independently. The goal of building a knowledge graph is to
harmonize the data used within our team and establish connections between information about
the infrastructure of the rail network and the recorded railway track videos.
2. Problem Statement
Monitoring the railway infrastructure is an important part of maintenance. So far, it has
been possible to use the videos to search for the elements, but it has not been possible to jump
directly to the relevant points in the video. Another requirement is the training of train drivers.
Here it is very helpful to be able to use video sequences of special constellations, e.g. passing
through a station from the entry signal to the exit signal, especially location-related to the
stationing of the train driver. To achieve this, the videos must be linked to the elements of the
infrastructure, but also to the information of the railway tracks. These heterogeneous data
Athen’23: The 22nd International Semantic Web Conference, November 06–10, 2023, Athen, Greece
∗
Corresponding author.
Envelope-Open Kai.Herbst@deutschebahn.com (K. Herbst); Rene.Krieg@deutschebahn.com (R. Krieg);
Dirk.Friedenberger@deutschebahn.com (D. Friedenberger)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
sources are usually distributed throughout the railway company. A harmonized graph-based
and queryable data view would cover a multitude of use cases beyond those mentioned.
3. Approach
Firstly the ontology was designed based on the various data sources, modeling different
entities and relationships such as signals and stations. The ontology was developed by the
team itself, instead of using an external ontology, in order to be able to adapt it exactly to the
data. The ontology is built up on the data sources mentioned below. For each train ride, we
receive data from LeiDis 1 (Leitsystem Disposition), a control system that monitors the current
operations on the rail network. That data includes information about the stations visited during
the ride, as well as the date and time at which the train arrived at certain stations. Data about
infrastructure occurring at specific kilometer marks along each railway track is derived from
so called substitute timetables. These timetables include all the necessary information for a
train driver, if he needs to drive a railway track without the modern digital version of these
information. Furthermore we can determine for every second of the video the precise location
of the train on the railway track, in terms of kilometer position. The graph also incorporates
various datasets related to tracks and stations. After converting and harmonizing these data
within the knowledge graph, complex queries can now be formed using SPARQL.
The modelling and creation of the ontology associated with the Knowledge Graph were
implemented using the Protégé software 2 from the Stanford University, which supports the W3C
standards of the RDF format and the OWL Web Ontology Language. After no existing ontology
could be found for the specific domain we were describing, we had to build one ourselves. This
has been accomplished by following the steps described in the ”Ontology Development 101”
Guide [1] from the Stanford University and ideas of the ”Generic Ontology Design Patterns”
paper [2]. In order to convert the mentioned data into rdf data, a Python module was developed
to collect, process and convert the video specific data into the rdf format. GraphDB 3 , based on
the RDF4J framework, was used as the triple store, providing a SPARQL endpoint and a web
interface for working with RDF triples. The architecture of the knowledge graph itself follows
a classical approach described in the paper ”Towards a Definition of Knowledge Graphs” [3] by
sending rdf data to GraphDB, that holds the ontology and uses it for deriving new knowledge.
To interact with the Knowledge Graph, a REST API and a user interface were created using the
FastAPI Python package4 . This additional frontend, in which various use cases are predefined,
also allows non-experts to use the information in Knowledge Graph.
1
Leitsystem Disposition
2
Protégé
3
GraphDB
4
FastAPI
4. Results
The knowledge graph enables us to quickly execute powerful queries based on the combi-
nation of data from the railway track network infrastructure and our videos. For example, a
simple query can list the various infrastructure elements occurring in a specific video. This
allows us to search for specific types of signals and list videos with timestamps containing
those elements. By referring to the timestamp in the video, the corresponding element in the
video can be found and visualised, see figure 1. By querying the knowledge graph for videos of
a particular element and sorting the results based on the video recording dates, we can also
analyze the infrastructure over a specific period. Searching for videos with specific elements
or passages through specific stations is particularly useful, especially for training purposes.
Additionally, since GPS data about infrastructure elements are also included in the graph, it is
possible to search for elements within a certain radius of a position.
Figure 1: Entry signal in a video, which was found with a query on the knowledge graph.
5. Conclusions
This knowledge graph, developed as part of a bachelor’s thesis, offers significant potential,
particularly due to the underlying RDF data format and the ease of expanding the graph,
extending far beyond its sole use within our team. The graph allows the aggregation of
different data from different sources in the rail sector. In addition the corresponding SPARQL
endpoint allows us to easily publish the data for others. A possible future improvement to the
knowledge graph could be not to use the aforementioned substitute timetables to determine
the infrastructure elements in a video, but rather to use AI-based recognition mechanisms to
extract the timestamp and position of these elements from the video itself.
The corresponding presentation will be focused on how we used semantic technology to solve
a problem, that would have been difficult to implement with traditional tools like a relational
database and how we are now able to easily extend the gathered knowledge with more data.
References
[1] N. F. Noy, D. L. McGuinness, et al., Ontology development 101: A guide to creating your
first ontology, 2001.
[2] B. Krieg-Brückner, T. Mossakowski, M. Codescu, Generic ontology design patterns: Roles
and change over time, 2020. a r X i v : 2 0 1 1 . 0 9 3 5 3 .
[3] L. Ehrlinger, W. Wöß, Towards a definition of knowledge graphs., SEMANTiCS (Posters,
Demos, SuCCESS) 48 (2016) 2.
A. Online Resources
• Protégé
• GraphDB
• FastAPI
• LeiDis