Graph of Driving Scenes for Knowledge Completion Tasks

Abstract

Knowledge graph completion (KGC) is a problem of significant importance due to the inherent incompleteness in knowledge graphs (KGs). The current approaches for KGC using link prediction (LP) mostly rely on a common set of benchmark datasets that are quite diferent from real-world industrial KGs. Therefore, the adaptability of current LP methods for real-world KGs and domain-specific applications is questionable. To support the evaluation of current and future LP and KGC methods for industrial KGs, we introduce DSceneKG, a suite of real-world driving scene knowledge graphs that are currently being used across various industrial applications. The DSceneKG is publicly available at: https://github.com/ruwantw/DSceneKG.

Keywords

• terms of structure
• modality
• conformance to ontology
• in/out degree
• cardinality
• etc
• Industrial

-

CEUR ceur-ws.org

1. Introduction

DSceneKG is a suite of knowledge graphs, developed as a collaborative efort between Table 1: Statistics of two KG variants Bosch and the University of South Carolina, to represent real-world driving data from NuScenes Pandaset openly available, real autonomous driving #triples 6,296,378 3,301,928 datasets such as Pandaset and NuScenes. #entities 2,108,545 53,248 These datasets represent diverse driving sce- Av#gr.eilna-tidoengsree 31.40353 6129.1387 narios, including urban/rural environments, Avg. out-degree 3.0107 63.3269 various weather conditions, trafic situations, Triples/entities 2.9861 62.0104 left/right-hand driving, and scenes from different continents. The Scene Ontology defines high-level object/event classes and their relationships [ 1 ]. DSceneKG instantiates all scene elements with metadata such as spatial coordinates, time, and descriptions where available. In the Scene Ontology, scenes are categorized into two types: (1) Sequence Scene – A video of 10-20 seconds, with a location region and temporal range; (2) Frame Scene – A sampled snapshot from the video, with a location point and timestamp. Table 1 shows some statistics about two KGs developed for the automated driving domain.

3. Benchmark Tasks of DSceneKG

The versatility of DSceneKG is demonstrated through various KGC/ knowledge-based tasks both within and outside Bosch. The tasks modeled exclusively as KGC are denoted by †. 1. Knowledge-based entity prediction (KEP)† - enabling a knowledge-based approach for predicting entities in driving scenes [ 1 ] 2. Context-based method for labeling unobserved entities (CLUE)† - completing AD datasets with labels for entities[ 2 ] that may have gone unobserved or unlabeled. 3. Explainable scene clustering - typing automotive scenes into explainable, high-level semantic clusters[ 3 ] 4. Semantic-based scene similarity - identifying automotive scenes that are semantically similar, but may be visually dissimilar[ 4 ] 5. Causal discovery† - enabling root-cause analysis/ causal discovery in driving scenes[ 5 ] 6. Knowledge-based retrieval - enhancing Bird’s-Eye View (BEV) retrieval by integrating semantic representations with textual descriptions [ 6 ]

References

[1] R. Wickramarachchi , et al., Knowledge-based entity prediction for improved machine perception in autonomous systems , IEEE Intelligent Systems ( 2022 ).

[2] R. Wickramarachchi , et al., CLUE-AD: a context-based method for labeling unobserved entities in autonomous driving data , AAAI ( 2023 ).

[3] S. Nag Chowdhury , et al., Towards leveraging commonsense knowledge for autonomous driving , 2021 .

[4] R. Wickramarachchi , et al., An evaluation of knowledge graph embeddings for autonomous driving data: Experience and practice , AAAI-MAKE ( 2020 ).

[5] U. Jaimini , et al., Causalkg: Causal knowledge graph explainability using interventional and counterfactual reasoning , IEEE Internet Computing ( 2022 ).

[6] D. Wei , et al., Bev-clip: Multi-modal bev retrieval methodology for complex scene in autonomous driving , arXiv ( 2024 ).