A video knowledge graph and a corresponding ontology was created based on railway track videos and infrastructure data. It was shown that the heterogeneous and distributed infrastructure and track data in the railway company, enriched with visualisations such as images and videos, implemented as an RDF/OWL graph, fulfill the required use cases and increases business value. These use cases include tasks such as being able to quickly query the graph for a videos in which specific infrastructure elements occur or being able to search for video sequences that start or end with certain elements.
CEUR ceur-ws.org
Monitoring the railway infrastructure is an important part of maintenance. So far, it has been possible to use the videos to search for the elements, but it has not been possible to jump directly to the relevant points in the video. Another requirement is the training of train drivers. Here it is very helpful to be able to use video sequences of special constellations, e.g. passing through a station from the entry signal to the exit signal, especially location-related to the stationing of the train driver. To achieve this, the videos must be linked to the elements of the infrastructure, but also to the information of the railway tracks. These heterogeneous data CEUR Workshop Proceedings sources are usually distributed throughout the railway company. A harmonized graph-based and queryable data view would cover a multitude of use cases beyond those mentioned.
Firstly the ontology was designed based on the various data sources, modeling diferent entities and relationships such as signals and stations. The ontology was developed by the team itself, instead of using an external ontology, in order to be able to adapt it exactly to the data. The ontology is built up on the data sources mentioned below. For each train ride, we receive data from LeiDis 1 (Leitsystem Disposition), a control system that monitors the current operations on the rail network. That data includes information about the stations visited during the ride, as well as the date and time at which the train arrived at certain stations. Data about infrastructure occurring at specific kilometer marks along each railway track is derived from so called substitute timetables. These timetables include all the necessary information for a train driver, if he needs to drive a railway track without the modern digital version of these information. Furthermore we can determine for every second of the video the precise location of the train on the railway track, in terms of kilometer position. The graph also incorporates various datasets related to tracks and stations. After converting and harmonizing these data within the knowledge graph, complex queries can now be formed using SPARQL.
The modelling and creation of the ontology associated with the Knowledge Graph were
implemented using the Protégé software 2 from the Stanford University, which supports the W3C
standards of the RDF format and the OWL Web Ontology Language. After no existing ontology
could be found for the specific domain we were describing, we had to build one ourselves. This
has been accomplished by following the steps described in the ”Ontology Development 101”
Guide [
The knowledge graph enables us to quickly execute powerful queries based on the combination of data from the railway track network infrastructure and our videos. For example, a simple query can list the various infrastructure elements occurring in a specific video. This allows us to search for specific types of signals and list videos with timestamps containing those elements. By referring to the timestamp in the video, the corresponding element in the video can be found and visualised, see figure 1. By querying the knowledge graph for videos of a particular element and sorting the results based on the video recording dates, we can also analyze the infrastructure over a specific period. Searching for videos with specific elements or passages through specific stations is particularly useful, especially for training purposes. Additionally, since GPS data about infrastructure elements are also included in the graph, it is possible to search for elements within a certain radius of a position.
This knowledge graph, developed as part of a bachelor’s thesis, ofers significant potential, particularly due to the underlying RDF data format and the ease of expanding the graph, extending far beyond its sole use within our team. The graph allows the aggregation of diferent data from diferent sources in the rail sector. In addition the corresponding SPARQL endpoint allows us to easily publish the data for others. A possible future improvement to the knowledge graph could be not to use the aforementioned substitute timetables to determine the infrastructure elements in a video, but rather to use AI-based recognition mechanisms to extract the timestamp and position of these elements from the video itself.
The corresponding presentation will be focused on how we used semantic technology to solve a problem, that would have been dificult to implement with traditional tools like a relational database and how we are now able to easily extend the gathered knowledge with more data.
• Protégé • GraphDB • FastAPI • LeiDis