Introduction

In order to cater for the users' need for faster access to relevant information, modern web search engines such as Ecosia, Bing, and Startpage 1 go beyond the conventional list-based display of search results by incorporating supplementary components on their result pages. One prominent example of these components are Knowledge Panels (KPs), which are typically positioned in the top right corner of the result page. These rectangular interface elements are designed to provide concise and curated information on an entity mentioned in the query sourced from dedicated knowledge bases known as Knowledge Graphs (KGs) [1,2]. By integrating such KPs, users can quickly access relevant details and insights about entities of interest without having to navigate the list of search results. As described in [3], current web search engines reliably display KPs only in response to single-entity queries, i.e., queries that mention exactly one entity, though. In contrast, for dual-entity queries, i.e., queries that mention exactly two entities, different behavior occurs: in some occasions, no KP is displayed at all, whereas sometimes only a KP for one of the entities is presented. In previous work [4,3], we explained that we regard this as a missed opportunity because KPs for dual-entity queries 2 , i.e., dual-entity KPs, could provide an explanation of the relationship between the two entities, thereby potentially serving the users' LWDA'23: Learning, Knowledge, Data, Analysis. October 09-11, 2023, Marburg, Germany Envelope leon.martin@uni-bamberg.de (L. Martin); andreas.henrich@uni-bamberg.de (A. Henrich) Orcid 0000-0002-6747-5524 (L. Martin); 0000-0002-5074-3254 (A. Henrich) information need, as well. We thus proposed a bidirectional A* search algorithm for finding meaningful paths between arbitrary entities in a KG that could serve as basis for generating an explanation of the entities' relationship [4,3]. Given the scope of web search engines, we employed Wikidata [5], an open-domain KG, to be able to handle queries mentioning entities from virtually any domain. As a follow-up to that work, the present paper also focuses on paths from Wikidata for populating KPs with explanations of entity relationships.

That being said, it has not been studied how path-based entity relationship explanations could be presented in the special context of KPs with usability in mind. As a first step in this novel line of research, the present paper introduces a testbed 3 for developing and evaluating dual-entity KPs, in particular the presentation format that is used to convey the explanation of the entity relationship to the users. To promote research in this direction, some examples of conceivable presentation formats are shown, as well.

The remainder of the present paper is structured as follows: Section 2 discusses foundations and related work. Then, Section 3 introduces the testbed itself. Section 4 demonstrates a selection of conceivable presentation formats, before Section 5 draws a conclusion.

Foundations & Related Work

Explanations of the relationship between two entities in a KG serve as the central foundation of the present paper. In [6], the task of entity relationship explanation is defined as follows:

"Given a pair of entities 𝑒 and 𝑒 ′ , provide an explanation, i.e., a textual description, supported by a KG, of how the pair of entities is related. "

In accordance with this definition, the bidirectional A* search algorithm from [4,3] detects paths in Wikidata that connect 𝑒 and 𝑒 ′ in a meaningful way. With respect to the KP context, 𝑒 and 𝑒 ′ correspond to the entities mentioned in a dual-entity query issued to a web search engine. The meaningfulness requirement, which is mandatory for generating entity relationship explanations that are useful for the users, is met by considering the semantic distances between entities as part of the search heuristics guiding the algorithm.

Since Wikidata is a KG that leverages the Resource Description Framework (RDF) [7], its information is encoded in the form of triples, each comprising a subject, a predicate, and an object. A predicate represents a property, which is a binary relation between the subject and the object that can be interpreted in both directions [8]. In Wikidata, the entities, which are part of the combined set of subjects and objects, use proprietary identifiers with a leading Q, while the predicates use proprietary identifiers with a leading P, in addition to the typical Internationalized Resource Identifiers (IRIs). Wikidata can therefore be interpreted as a graph 𝐺 = (𝑉 , 𝐸) where the vertices 𝑉 are the combined set of subjects and objects and the edges 𝐸 are the instances of predicates. Accordingly, a path 𝑃 between 𝑒 ∈ 𝑉 and 𝑒 ′ ∈ 𝑉 consists of a set of vertices 𝑉 𝑃 ⊆ 𝑉 and a set of edges 𝐸 𝑃 ⊆ 𝐸, thereby also qualifying as a (sub)graph. The length of a path is defined as the number of edges, i.e., one less than the number of entities, on the path [9].

𝑒 ... 𝑒 ′ (a) A direct path from 𝑒 to 𝑒 ′ 𝑒 ... 𝑒 ′ (b) A direct path from 𝑒 ′ to 𝑒 𝑒 ... 𝑣 𝑖 ... 𝑒 ′

(c) A path composed of a direct path from 𝑒 to an intersecting entity 𝑣 𝑖 and a direct path from 𝑒 ′ to an intersecting entity 𝑣 𝑖

Figure 1:

The graph patterns that can be found using the bidirectional A* search algorithm from [4], adopted from [3]. 𝑒 and 𝑒 ′ correspond to the entities of a dual-entity query, between which a path is searched. Nodes with ... are placeholders for series of 𝑛 ≥ 0 entities.

For exchanging RDF data, various serialization formats exist. The testbed expects paths in the Terse RDF Triple Language (Turtle) [10] as an input. Due to its characteristics, the bidirectional A* search algorithm from [4,3] can only find paths following the patterns shown in Figure 1. That being said, the testbed is able to parse Turtle-formatted paths with arbitrary patterns even though the focus of the present paper remains on paths with the depicted patterns.

To generate a dual-entity KP based on a Turtle-formatted path, the explanation of the entity relationship encoded within the path has to be presented in some way. To this end, there are various options, some of which are fundamentally different. For instance, graph-based visualizations are just as conceivable as textual representations in natural language. Thus, we use the umbrella term presentation formats to subsume the range of options for presenting the entity relationship in a dual-entity KP, disregarding the utilized modalities and media types.

Regarding the presentation of paths from Wikidata in particular, it is important to consider the particularities of the Wikidata knowledge graph. Entities and properties in Wikidata feature many (contextual) information that can be leveraged to present paths in a more user-friendly manner. Most importantly, there are triples that provide natural language labels and descriptions for each entity and property. To give an example, consider the following path from Wikidata with Q7958 (explanation) representing 𝑒 and Q46857 (scientific method) representing 𝑒 ′4 : Q7958 (explanation) −P366 (has use)→ Q352842 (teaching) −P31 (instance of)→ Q11862829 (academic discipline) −P1269 (facet of)→ Q336 (science) ←P1535 (used by)− Q46857 (scientific method) Using the testbed that will be introduced in Section 3, one can implement dual-entity KPs that leverage the presentation format from the example and evaluate it using standard methods from human-computer interaction, thereby deepening the understanding of dual-entity KP usability. In addition to labels and descriptions, there are many other properties that could be leveraged for presenting the entity relationships. The testbed imposes no restrictions on the use of supplementary information as long as it is encoded in the Turtle format.

The visualization of graphs has been studied intensively in the past [11,12]. Occasionally, one can find side notes on how individual paths in a graph visualization could be highlighted.

However, the presentation of particular paths representing entity relationships in the context of KPs has not been investigated so far. Nevertheless, insights from previous work on graph visualization should be applicable to path visualization as well, since paths themselves are, by definition, graphs, albeit simple examples of them. Disregarding the particular presentation format that is employed, fundamental principles of interface design still apply, as well. This includes, for instance, Nielsen's usability heuristics [13] but also more modern guidelines like the visual design principles postulated in [14].

Recent advances in machine learning and more specifically natural language generation enable the usage of textual presentation formats that were difficult to implement before. Previous work considered the task of natural language generation as a compound problem comprising several independent tasks that address some aspect of the generation process like the text structuring or the linguistic realization [15]. Despite significant effort, the resulting texts often lack quality. Representing the current state-of-the-art, Large Language Models (LLMs) such as GPT-4 [16] leverage a holistic approach for the natural language generation problem to generate high-quality texts. ChatGPT, the conversational AI based on GPT, allows users to easily issue specific requests via prompts. For instance, one can request a textual description of certain (semi-)structured data. For our use case, this ability can be utilized to implement a presentation format that uses an actual natural language explanation to convey the entity relationship, just as prescribed in the definition of entity relationship explanation from [6].

The Testbed

The testbed is implemented as a web application using Svelte 5 and is Docker-ized 6 for ease of use and reproducibility. As depicted in Figure 2, the user interface features a central input group allowing the selection of the two entities of a dual-entity query 7 . After the selection, users can press a button to trigger the generation of the corresponding dual-entity KP. Disregarding the presentation format, each dual-entity KP comprises the labels, the IDs, and the descriptions of the two entities. Below that, there is a dropdown menu for selecting one of the available presentation formats. By default, the presentation format called Turtle is selected, which presents the path as the raw Turtle document. To add other presentation formats, one only has to add a Svelte component containing the corresponding code 8 .

Note that the testbed is currently not connected to a pathfinding backend since its purpose is to facilitate the implementation and evaluation of presentation formats in an isolated manner. The paths that can be selected via the central input group thus originate from Turtle-formatted paths that are hard-coded into the application. This ensures that always the same paths are available in the user interface. Furthermore, custom paths with specific characteristics can be added and tested easily, which would be difficult to achieve with an actual pathfinding backend. We plan to conduct both qualitative studies, e.g., using the think-aloud method, as well as quantitative studies, e.g., using the system usability scale [17], with the testbed.

Showcase of Presentation Formats

In addition to the Turtle presentation format, the four prototypical presentation formats depicted in Figure 3 have been implemented yet. Arrow corresponds to the presentation format used in Section 2 to give an example for a path in Wikidata. While this presentation format features the human-readable labels of entities and properties from Wikidata, it suffers from the prominent positioning of their proprietary IDs, which are supposedly not useful for most users. Next, LLM presents the path as a natural language explanation generated with ChatGPT. For this purpose, ChatGPT was asked to generate a description of the path encoded in the Turtle document. To mature this presentation format, the prompts provided to the LLM of choice need to be refined to optimize content, length, and structure of the description based on user feedback. That being said, the depicted explanation already possesses a decent quality. Both the third and the fourth presentation format use a graph-based visualization of the path but leverage different layouts. While Graph: Circle arranges the nodes in a circle, Graph: Hierarchy interprets the path as a hierarchy with respect to the direction of the properties. In both layouts, hovering over the path components further triggers a popup revealing the respective ID and description. Furthermore, the entities of the dual-entity query are highlighted in blue. Still, the latter format is supposedly easier to parse for users, especially when the properties express taxonomic relations. Facilitating the evaluation of such assumptions is a primary objective of the testbed.

Future work should not only evaluate and refine the showcased presentation formats but also investigate further options. Considering that KPs can comprise various media depending on the particular entity, even exotic presentation formats might be useful for some dual-entity queries. For example, AI-generated pictures could be leveraged to explain the relationship of certain entities. Wolfram|Alpha9 , an engine for factual query answering, also generates different visualizations comparing characteristics depending on the type of the queried entities. This includes maps with locations of geographic entities and tables comparing features of entities from a similar domain, among others. Context sensitivity is thus an important topic to be explored. Moreover, combinations of different presentation formats should also be considered.

Conclusion

The present paper introduced a testbed facilitating the implementation and evaluation of dualentity KPs, i.e., KPs that are presented in response to dual-entity queries. In Section 4, a selection of prototypical presentation formats that convey an explanation of the relationship between the two entities of such queries was showcased. Even though this paper focused on paths from Wikidata as a basis for generating the explanations, the testbed can be adapted to operate on paths from other KGs if desired.

Complementary to the study of representation formats, another line of research is required that examines the usefulness of paths with respect to their characteristics. For example, paths with properties expressing taxonomic relationships might be more accessible to typical end users. In addition, paths exceeding a certain length might yield explanations with a lower perceived usefulness. The two lines of research must go hand in hand since path characteristics influence the applicability of presentation formats, as well. For instance, graph-based presentation formats are not suitable for illustrating paths beyond a certain length.