<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <issn pub-type="ppub">1613-0073</issn>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Ontology of Understanding in Explanatory Interactions</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Meisam Booshehri</string-name>
          <email>mbooshehri@techfak.uni-bielefeld.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Hendrik Buschmeier</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Philipp Cimiano</string-name>
          <email>cimiano@cit-ec.uni-bielefeld.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Workshop</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="editor">
          <string-name>Compostela, Spain</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Digital Linguistics Lab, Faculty of Linguistics and Literary Studies, Bielefeld University</institution>
          ,
          <addr-line>Bielefeld</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Semantic Computing Group, Faculty of Technology, Bielefeld University</institution>
          ,
          <addr-line>Bielefeld</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Transregional Collaborative Research Center 318 “Constructing Explainability”</institution>
          ,
          <addr-line>Paderborn and Bielefeld</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Understanding</institution>
          ,
          <addr-line>Explanation, Explainable AI, Ontology design, BFO</addr-line>
        </aff>
      </contrib-group>
      <abstract>
        <p>This work takes steps towards situating the concepts relevant to explanation and understanding in explanatory interactions within the scope of Basic Formal Ontology. We introduce novel ontological accounts of understanding and explanation in BFO-terms, which foster a shared conceptualization of explanations and explainee's understanding during explainer-explainee interactions. This approach also enables the tracking of diferent aspects of understanding and explanation through cognitive profiling of various measurable aspects under the heading of process profile in BFO. Additionally, we diferentiate between the private mental process of understanding and understanding displays. Finally, we characterize the relationship between understanding displays and explanations.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Explaining and understanding are two sides of the same coin [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. To be able to give an explanation,
human explainers must be able to understand the explanandum. Similarly, in order to make use of a given
explanation, human explainees need to understand it. Recently, it has been argued that explanations in
general are social [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] and that they are actually co-constructed by the interlocutors [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Both of these
arguments have been made in the broader context of Explainable AI (XAI), which has traditionally
focused on the explanation itself rather than the explanatory interaction. Here we are interested in
explainer–explainee interactions (human–human and XAI–human), seeking, from an interdisciplinary
perspective, the elements that provide us with indications of explainee’s understanding, e.g., based on
their linguistic and multimodal behaviour [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>
        Rohlfing et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] present a conceptual framework for the social design of XAI, where the explainer’s
task is to provide an explanation and adapt it to the current level of explainee’s understanding, and the
explainee’s task is to provide cues that actively guide the explainer. This way, explainer and explainee,
jointly help co-construct an explanation. In other words, the authors emphasize the importance of
tailoring explanations to meet the needs of the explainee. Additionally, they argue that the process of
explaining should not be overlooked, as social interaction is the key for producing a (socially) relevant
and understandable explanation [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. In such a ‘Social XAI’, multimodality, as an important property of
interaction, is postulated to play a central role [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        An underlying assumption of this paper is that ontologies and best practice principles for ontology
design are helpful for social XAI. Among the many benefits that ontologies ofer, one of the most
concretely useful is that they provide a shared vocabulary, ideally based on rigorous definitions within
a domain, thus enabling researchers to better navigate the complexity of a domain [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. In practice,
∗Corresponding author.
      </p>
      <p>CEUR</p>
      <p>ceur-ws.org
the shared vocabulary is not only useful for communicating about the topic, but also enables data
integration, making scientific results comparable. Ontologies are also used to formally describe the
entities in a domain and their relationships to each other, and can contain logical axioms. This enables
automatic reasoning in technical applications and is therefore also highly relevant to the development
of (social) XAI systems. We describe an efort to develop an ontology of ‘understanding’ in the realm of
explanations. In this, we follow the general principles of ontology design as described by Arp et al. [6,
p. 50], specifically perspectivalism and fallibilism.</p>
      <p>
        Our ontology is based on a scientific conceptualization of understanding, resulting from both the
concrete needs of research projects and a series of interdisciplinary synthesis workshops (happening
in the context of the Collaborative Research Center SFB/Transregio 318); however, we do not seek
to capture reality in its totality. We are therefore considering the principle of perspectivalism, which
recognizes that there may be multiple theories that accurately represent a scientific topic, and that it
may not be possible to reduce these perspectives to a single ontology. One solution would then be a
modular approach, with experts in each scientific discipline maintaining a module. We also consider the
notion of fallibilism, which reflects the idea that scientific statements can be revised as new evidence
emerges, while at the same time being considered as candidates for expressing the truths of reality.
Based on these two aspects we follow Arp et al. [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] and try to keep ontology development pragmatic:
we try to strike a balance between utility and realism, developing the ontology for concrete needs in
XAI research projects and grounding it in interdisciplinary scientific investigation of the concept of
understanding.
      </p>
      <p>
        On this basis, we address three fundamental research questions that arise from the social perspective
on XAI [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ]:
• RQ1: How can we ontologically model explanations in Social XAI?
• RQ2: How can we ontologically model understanding in Social XAI?
• RQ3: How to track explainees’ understanding in explanations?
      </p>
      <p>
        We are taking steps to situate the concepts relevant to understanding and explanation within the
standardized framework of Basic Formal Ontology [BFO; 7] – similar to Donohue [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]’s BFO-based
Deontic Ontology. BFO is a so-called ‘top-level’ ontology that supports the integration of research results
and enables interoperability of BFO-based domain ontologies. Following the basic types of entities
in BFO [6, p. 87], we divide the concepts relevant to understanding and explanation into continuants
(entities that continue or persist through time, such as independent objects, qualities and functions)
and occurrents (entities that occur or happen, such as processes or events).
      </p>
      <p>
        We will present work towards a BFO-based ontology of understanding in explanatory interactions.
This endeavor is part of an ongoing larger ontology development process working towards an ontology
of social explainable AI. Taken together, this paper makes three contributions:
• We provide a novel ontological account of explanations in the sense of Social XAI [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ] and
contrast it with existing ontological accounts of explanations in XAI (Section 3).
• We provide an ontological account of understanding in explanatory interactions, which allows
for capturing diferent measurable dimensions of understanding (Section 4).
• We provide an ontological account of how understanding is related to explanation processes in
terms of understanding displays and illustrate how this matches research practices in linguistic
analyses of explanatory interactions (Section 5).
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Background</title>
      <p>
        Ontologies are classification systems serving as a method for representing the structure of a specific
domain by capturing the essential entities and relationships between the entities within that domain
[
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. In their study, Confalonieri and Guizzardi [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] discuss multiple roles of ontologies in XAI, including
reference modeling. Reference modeling involves utilizing ontologies as reference models to specify
the requirements of explainable AI systems. In this capacity, ontologies can play a crucial role in the
development of explainable AI systems, e.g., by helping researchers and practitioners gain a deeper
understanding of human cognitive patterns and the interplay between human cognition and XAI [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>
        Reference modeling for XAI aligns closely with the perspective articulated by Miller [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], who
underscores that XAI can benefit from the extensive body of existing research from various areas of research,
such as philosophy, psychology, sociology and cognitive science, on “how people define, generate,
select, evaluate, and present explanations.” This basically necessitates an investigation into how human
actions are built. Goodwin [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] reveals answers to this question by emphasizing the importance of
multi-modality. He states: “… why multi-modality? It’s a term that I don’t use all that much because I
think that sometimes the way people talk about multi-modality, they make it see as a side thing, [i.e.,] the
core stuf is the language or the action or anything, and what I want to argue is that what we really want to
look at is the core structure of the human action and that the way that human action is built is by bringing
diferent kinds of meaning making resources together …”. Following Goodwin’s perspective, we do not
consider multi-modality as incidental when it comes to explanations.
      </p>
      <p>Miller, on the other hand, discusses two accounts of explanation: explanation as a product and
explanation as a process. He further assumes that there exists two types of processes in explanation: a
cognitive process and a social process. The cognitive process is “to determine an explanation for a given
event, called the explanandum, in which the causes for the event are identified, perhaps in relation to
particular counterfactual cases, and a subset of these causes is selected as the explanation (or explanans).” [2,
p. 6]. He further terms the explanation resulting from such a cognitive process as a product. Finally, he
discusses that the social process of explanation corresponds to the knowledge transfer between explainer
and explainee – a process whose main goal is to provide suficient information for the explainee to
understand the explanandum.</p>
      <p>
        In exploring the nature of explanation as a product within the realm of ontologies, a parallel can be
drawn to the conceptualization put forth by Chari et al. [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. In their ontology, the authors classify
explanation under the heading of object, where an object is “an entity that is wholly identifiable at any
instant of time during which it exists.” Such a definition of objects is very much in line with the concept
of continuants within the scope of Basic Formal Ontology [6, p. 87]. Cabitza et al. [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] also propose a
typology of explanation-related entities, defining explanation merely as the output of an XAI system. The
authors intentionally avoid epistemological considerations and instead ofer a definition of explanation
that suits their practical purposes. In contrast to this, given the multimodal nature of explanations
in the sense of Rohlfing et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] and the account of explanations as socio-cognitive processes in the
sense of Miller [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], explanations in this other facet appear to belong to the world of occurrents, and
specifically processes. To the best of our knowledge, despite the presence of fundamental frameworks
ofering insights into explanation-related entities and methods [
        <xref ref-type="bibr" rid="ref13">13, 14</xref>
        ] and valuable research at the
intersection of ontologies and XAI [
        <xref ref-type="bibr" rid="ref12 ref9">9, 12, 15</xref>
        ], so far, there has been no ontological account that attends to
explanations as multimodal socio-cognitive processes. In the current work, we deal with such concepts
using the top-level categories of BFO.
      </p>
      <p>As discussed, explanation and understanding are tied together. Considering the previously mentioned
research, it is clear that new approaches to explanation focus on the needs of the explainee and
adapting the explanation to explainee’s current level of understanding. Accordingly, for an XAI system
to efectively model an explanation process, it must be capable of assessing understanding while
interacting with humans. This necessitates addressing fundamental questions: What is understanding?
and What is it that we want to measure accordingly?</p>
      <p>A relevant avenue of investigation is the field of discursive psychology, which examines how private,
inner, mental processes manifest in social interactions. Deppermann [16] explores how understanding in
interaction is informed by temporality and retrospection. He asserts that “Understanding is a temporally
extended, sequentially organized process. Temporality, namely, the sequential relationship of turn
positions, equips participants with default mechanisms to display understandings and to expect such
displays” [16, p. 57]. He further discusses that understanding is a private mental process, which is not
available to others. Moreover, one can neither check understanding nor react to it. Consequently, in
discursive psychology, understanding is ‘respecified’ by examining “how it becomes relevant, observable
and treatable for participants in the interaction itself.”</p>
      <p>Within the realm of philosophy, Grimm and Hannon [17] argue that “Understanding is a kind of
cognitive accomplishment, and the objects of understanding […] are strikingly varied.” Speaking of
‘accomplishment’ immediately points us to a classic typology which distinguishes four types of events –
accomplishment, activity, state, and achievement [18]:</p>
      <p>An activity, such as Anita’s walking uphill, is a homogeneous event: its sub-events satisfy the
same description as the activity itself, which has no natural finishing point or culmination.
An accomplishment, such as Anita’s climbing the mountain, may have a culmination, but is
never homogeneous. An achievement, such as Anita’s reaching the top, is a culminating
event (and is therefore always instantaneous). And a state, such as Anita’s knowing the
shortest way, is homogeneous and may extend over time, but it makes no sense to ask how
long it took or whether it culminated. [18]</p>
      <p>Casati and Varzi [18] discuss that sometimes achievements are considered as events in the strict sense
and all other types of events are classified as processes (i.e., temporally extended entities). This speaks of
the fact that viewing understanding as a cognitive ‘accomplishment’ emphasizes its nature as a process.</p>
      <p>The BFO account of processes extends Zemach’s concept of ‘events’. According to Smith [20], event
in the sense of Zemach [19] refers to the whole content of a spatiotemporal region. Yet, in BFO terms,
multiple processes can occupy the same spatiotemporal region, e.g. “when a process of your running
down the street is co-located with a process of your getting warmer” [20, p. 473].</p>
      <p>
        Smith further refers to these measurable dimensions of processes as streamiform structures [21],
and introduces the top-level category of process profile into BFO (although this is not part of the BFO
ISO-standard [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] yet). Discerning streamiform structures within a complex process allows for studying
these structures in isolation, while also acknowledging the complexity of the phenomena and leaving
space for studying the impact of combination of these structures as a whole.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. On the Nature of Explanation</title>
      <p>
        The accounts of explanation proposed by Miller [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] and the framework proposed for social XAI by
Rohlfing et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] are not grounded in any ontological or philosophical theories of processes, such
as Smith [20] or Guarino and Guizzardi [22]. Consequently, we recognize the need to formalize such
accounts of explanations. In alignment with Miller’s perspective on explanation as both a socio-cognitive
process and a product [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], we conceptualize explanations as both an occurrent entity and a continuant
entity, respectively.
      </p>
      <p>Within the scope of our ontology, we use the term ‘explanation’ to denote the occurrent facet of
explanation, while reserving the term ‘explanans’ to refer to the continuant facet of explanation, which
we will explore in future work.</p>
      <p>We therefore put forth the thesis that explanation is an occurrent entity in BFO terms, precisely
categorized under BFO:process. Derived from the inherent attributes of BFO:process [6, p. 89], we can
subsequently deduce the following implications for explanations:
• Explanations unfold in successive temporal parts or phases.
• Because no two distinct phases exist simultaneously, there is no point in time at which an
explanation exists as a whole.
• An explanation exists at any given point in time only in some correspondingly short-lived stage
or slice.</p>
      <p>Note that the dialogue participants that take the role of explainer or explainee are the material entities
that participate in the explanation.</p>
      <p>
        Given the nature of explanations as processes, the acknowledgment that “the way human action is
built is by bringing diferent kinds of meaning-making resources together” [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] (possibly via diferent
modalities), emphasizes the existence of distinct dimensions of qualitative change (e.g., through a focus
on particular modalities) within a broader conglomerate of processes. Accordingly, diferent dimensions
of distinctive qualitative change can therefore be discerned, such as alterations in the cognitive mode of
engagement [23] of explainer and explainee, shifts in their facial expressions, changes in their eye gaze
[24], adaptations in the types of speech acts they utilize [25], and alterations in prosody they employ
[
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. These are in fact the streamiform structures, i.e., process profiles that we can cognitively profile
within the complex explanation process.
      </p>
      <p>To elucidate this line of thinking, let us characterize explanations as dialogues in which explainee and
explainer take turns to advance the explanation process. There exists approaches [26, 27] that analyze
the single contributions and intentions of explainees or explainers as speech/dialogue acts [28, 25],
within explanatory dialogues. From our point of view, these methods look at a specific streamiform
structure of reality in the explanation process, which is the sequence of speech acts produced by the
explainer and the explainee during the course of an explanation. By characterizing the series of speech
acts as a streamiform structure of reality, we highlight the examination of a thinner process (in the
sense of Smith [21]), representing a meaningful structure within the broader process of explanation.
Simultaneously, we recognize that explanation, as a process, may involve various modalities (i.e., process
profiles) beyond speech acts, depending on the communication medium [ 29].</p>
      <p>In the above-mentioned example, a speech act produced by explainer or explainee is a
proper_occurrent_part_of an explanation process.</p>
    </sec>
    <sec id="sec-4">
      <title>4. On the Nature of Understanding</title>
      <p>Similar to Deppermann’s concept of understanding in interaction [16] and drawing inspiration from
Grimm and Hannon’s perspective [17], we view understanding as a process. However, we aim to provide
a more comprehensive account of understanding by utilizing the theory of processes from Basic Formal
Ontology [20, 30].</p>
      <p>We propose categorizing understanding under the heading of BFO:process and the measurable
dimensions of understanding under the heading of BFO:process profile .</p>
      <p>In a first attempt for conceptualizing diferent dimensions of understanding within the scope of
social XAI, we employ diferent cognitive dimensions of the Bloom’s taxonomy [ 31] – i.e., Knowledge,
Comprehension, Application, Analysis, Synthesis, and Evaluation – as heuristics for diferent measurable
dimensions of understanding.</p>
      <p>As an example, we illustrate potential process profiles of understanding processes in terms of cognitive
dimensions of Bloom’s taxonomy with the following equation:
  = 0.5 + log  ( + 2) (1 + 0.1 sin (
   
10
))
where:
•  denotes the  -th level of Bloom’s taxonomy and ranges from 1 to 6 (1: Knowledge, 2: Comprehension,
3: Application, 4: Analysis, 5: Synthesis, and 6: Evaluation)
•   denotes the corresponding instance of process profile for the  -th level of Bloom’s taxonomy.
• (  ,   ) ∈ {(2, 0), (3, 2), (5, 4), (7, 6), (11, 8), (13, 1w0)h}ere   are the bases of the logarithm and  
are the frequencies of the sinusoidal term.</p>
      <p>•  is a variable denoting time.</p>
      <p>Note that the equation primarily serves illustrative purposes rather than precise modeling. It is
designed to represent the understanding processes of an explainee with low to medium skill levels regarding
some specific explanandum, with the constant term 0.5 vertically shifting the function to accommodate
this. The logarithmic term log  ( + 2) captures the growth in cognitive gains over time. Periodic
lfuctuations in cognitive performance are modeled by the sinusoidal modulation (1 + 0.1 sin (   )),
10
which introduces oscillations around a general trend. Figure 1 illustrates these functions within an</p>
      <p>Knowledge
Comprehension</p>
      <p>Application</p>
      <p>Analysis
Synthesis
Evaluation
0
2
4</p>
      <p>6
Time
8
10
explanation process over a period of 10 time units. As time increases, the curves demonstrate how
understanding deepens across any of the six cognitive dimensions. Each curve represents an instance
of a process profile.</p>
      <p>A further category that we can introduce to our ontology of understanding is understanding
achievement. We assume that the understanding process has culmination points (think of ‘Aha-moments’,
‘Huh?-moments’, etc.). We categorize these culmination points as ‘understanding achievement’. An
achievement is instantaneous and can be categorized as an occurrent entity, specifically under the
heading of BFO:process boundary. In BFO, beginnings, endings and thresholds in processes, that are in
fact infinitesimal or of zero length, are categorized under BFO:process boundary.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Understanding Displays</title>
      <p>
        As mentioned in Section 2, understanding is considered a private process and thus not available to others
[16]. However, in dialogical interaction generally, and explanatory interactions specifically, addressees
multimodally communicate their (private [16] and subjective [4, §2.1.3]) mental state of understanding
to speakers, and speakers monitor addressees for understanding [32]. Such ‘understanding displays’ can
take many forms and play an important role in dialogue by enabling speakers to attribute mental states
of understanding [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] to addressees and thus to incrementally adapt their ongoing speech production
to addressees’ needs – a process called recipient or audience design [33]. In explanatory interactions,
these processes of monitoring and adaptation (or ‘scafolding’, to use a term from educational and
developmental research) are thought to be crucial for co-constructing explanations by explainers and
explainees [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>The most important examples for understanding displays in verbal explanations are the explanatory
moves [26]. Some of these moves explicitly serve the purpose of providing evidence of understanding of
the interlocutors utterance, or lack thereof, specifically the moves acknowledge, (non-lexical) backchannel,
completion, signal-understanding/non-understanding. Other moves might serve similar functions when
0.5
0.0
t
produced in response to feedback elicitation requests by explainers (e.g., to test understanding questions),
or more generally (e.g., explainees asking a relevant question may implicitly display their understanding).
In multimodal face-to-face explanatory dialogues, explainees’ mental states of understanding are often
displayed in embodied ways [34] through head gestures (e.g., nods), facial expressions (e.g., eyebrow
raise), eye blinks, and gaze behaviour. Being expressed in other modalities than speech, these embodied
understanding displays can happen simultaneously (and potentially continuously) to the explainer’s
verbal explanation, enabling immediate adaptation [32] without the need to respect the turn-taking
system [35].</p>
      <p>We consider an understanding display as a proper_occurrent_part_of an explanation process.
Understanding displays are integral sub-processes of explanatory dialogues [20] central to the co-construction
of explanations. We thus categorize understanding displays under the heading of BFO:process. Their
contribution to explanations being that they make ‘understanding’ – generally not available to
others during explanatory dialogues – indirectly measurable through interpretation, thus informing the
measurable dimension of understanding (BFO:process profile ). That understanding displays fit well with
profiling of understanding, is evident from research practice in linguistics as well as in computational
modeling of listener understanding.</p>
      <p>In linguistic annotation, understanding displays in the form of multimodal listener behaviors are
annotated in time-aligned ways (illustrated with the example in Figure 2a), allowing for time-aligned
interpretation and attribution of understanding. In computational work, multimodal feedback behaviour
can be interpreted by an artificial conversational agent and modeled as a dynamic belief state of a state
of understanding attributed to the interaction partner over time (illustrated in Figure 2b).</p>
    </sec>
    <sec id="sec-6">
      <title>6. Final Considerations</title>
      <p>
        In this work we have focused on three research questions, inspired by the framework of ‘social XAI’ –
an account of XAI which focuses on explainer-explainee interactions and their co-construction of an
explanation [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>Concerning RQ1 and RQ2, which address the ontological modeling of explanation and understanding
in social XAI, respectively, we have introduced novel ontological accounts in BFO terms. This efort
aims to remedy the current lack of formalization of explanation and understanding in social XAI. We
classify both explanation and understanding under the heading of BFO:process. Such a treatment allows
for capturing diferent measurable dimensions of both understanding and explanation through process
profiles [20]. To the best of our knowledge, these ontological accounts are the first that model the
concepts specific to ‘social XAI’ in which temporality is considered as a key dimension.</p>
      <p>Such a treatment of explanation and understanding also necessitates an extension in the existing
theories that characterize their relationship with explanation being treated as a continuant. For instance,
Grimm [37] discusses one of these theories that holds: “Something E is an explanation of why Q only if
someone who possesses E understands why Q.”</p>
      <p>Regarding RQ2, for the sake of illustrative modeling, we have also described how diferent dimensions
of understanding can be conceptualized in terms of cognitive dimensions of the Bloom’s taxonomy.</p>
      <p>Regarding RQ3, which focuses on the tracking of explainee’s understanding in explanations, we
have described how understanding can be displayed through diferent explanatory moves in verbal
explanations or other modalities than speech. Finally, we have characterized the relationship between
understanding displays and explanations, where we consider an understanding display to be a
proper_occurrent_part_of an explanation process.</p>
      <p>Taken together, we hope that the ontological accounts that we have provided for explanation and
understanding in explanatory interactions can contribute to the endowment of explainable AI systems.</p>
    </sec>
    <sec id="sec-7">
      <title>Acknowledgments</title>
      <p>This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation):
TRR 318/1 2021 – 438445824, Projects INF (“Toward a framework for assessing explanation quality”)
and A02 (“Monitoring the understanding of explanations”).
explainable AI, Expert Systems with Applications 213 (2023) 118888. doi:1 0 . 1 0 1 6 / j . e s w a . 2 0 2 2 .
1 1 8 8 8 8 .
[14] C. O. Retzlaf, A. Angerschmid, A. Saranti, D. Schneeberger, R. Röttger, H. Müller, A. Holzinger,
Post-hoc vs ante-hoc explanations: xAI design guidelines for data scientists, Cognitive Systems
Research 86 (2024) 101243. doi:1 0 . 1 0 1 6 / j . c o g s y s . 2 0 2 4 . 1 0 1 2 4 3 .
[15] I. Tiddi, M. d’Aquin, E. Motta, An ontology design pattern to define explanations, in: Proceedings
of the 8th International Conference on Knowledge Capture, ACM, Palisades, NY, USA, 2015, pp.
3:1–8. doi:1 0 . 1 1 4 5 / 2 8 1 5 8 3 3 . 2 8 1 5 8 4 4 .
[16] A. Deppermann, Retrospection and understanding in interaction, in: A. Deppermann, S. Günthner
(Eds.), Temporality in Interaction, John Benjamins, Amsterdam/Philadelphia, 2015, pp. 57–94.
doi:1 0 . 1 0 7 5 / s l s i . 2 7 . 0 2 d e p .
[17] S. R. Grimm, M. Hannon, Understanding, in: Oxford Bibliographies in Philosophy, Oxford</p>
      <p>University Press, 2010. doi:1 0 . 1 0 9 3 / o b o / 9 7 8 0 1 9 5 3 9 6 5 7 7 - 0 1 2 1 .
[18] R. Casati, A. Varzi, Events, in: E. N. Zalta, U. Nodelman (Eds.), The Stanford Encyclopedia
of Philosophy, Fall 2023 ed., Metaphysics Research Lab, Stanford University, 2023. URL: https:
//plato.stanford.edu/archives/fall2023/entries/events/.
[19] E. M. Zemach, Four ontologies, The Journal of Philosophy 67 (1970) 231–247. doi:1 0 . 2 3 0 7 / 2 0 2 4 1 8 5 .
[20] B. Smith, Classifying processes: An essay in applied ontology, Ratio 25 (2012) 463–488. doi:1 0 .</p>
      <p>1 1 1 1 / j . 1 4 6 7 - 9 3 2 9 . 2 0 1 2 . 0 0 5 5 7 . x .
[21] B. Smith, Material entities and process profiles (Analytic Metaphysics Lecture, 5), 2016. URL:
https://www.youtube.com/watch?v=kAmBUNB2amU.
[22] N. Guarino, G. Guizzardi, Processes as variable embodiments, Synthese 203 (2024) 104. doi:1 0 .</p>
      <p>1 0 0 7 / s 1 1 2 2 9 - 0 2 4 - 0 4 5 0 5 - 2 .
[23] M. T. Chi, J. Adams, E. B. Bogusch, C. Bruchok, S. Kang, M. Lancaster, R. Levy, N. Li, K. L. McEldoon,
G. S. Stump, et al., Translating the ICAP theory of cognitive engagement into practice, Cognitive
Science 42 (2018) 1777–1832. doi:1 0 . 1 1 1 1 / c o g s . 1 2 6 2 6 .
[24] Y. Wang, H. Buschmeier, Does listener gaze in face-to-face interaction follow the entropy rate
constancy principle: An empirical study, in: Findings of the Association for Computational
Linguistics: EMNLP 2023, Association for Computational Linguistics, Singapore, 2023, pp. 15372–15379.
doi:1 0 . 1 8 6 5 3 / v 1 / 2 0 2 3 . f i n d i n g s - e m n l p . 1 0 2 6 .
[25] A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. V.
EssDykema, M. Meteer, Dialogue act modeling for automatic tagging and recognition of conversational
speech, Computational Linguistics 26 (2000) 339–373. doi:1 0 . 1 1 6 2 / 0 8 9 1 2 0 1 0 0 5 6 1 7 3 7 .
[26] M. Booshehri, H. Buschmeier, M. Alshomary, K. Rohlfing, H. Wachsmuth, P. Cimiano, Modeling
explanations as processes: A new analytical framework accounting for relational and structural
patterns in explanatory dialogues, 2024. doi:1 0 . 5 2 8 1 / z e n o d o . 1 3 1 5 1 3 5 0 , preprint, under review,
available at zenodo.
[27] M. Alshomary, F. Lange, M. Booshehri, M. Sengupta, P. Cimiano, H. Wachsmuth, Modeling the
quality of dialogical explanations, in: Proceedings of the 2024 Joint International Conference on
Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), ELRA and
ICCL, Torino, Italy, 2024, pp. 11523–11536. URL: https://aclanthology.org/2024.lrec-main.1007.
[28] J. R. Searle, Speech Acts: An Essay in the Philosophy of Language, Cambridge University Press,
1969. doi:1 0 . 1 0 1 7 / C B O 9 7 8 1 1 3 9 1 7 3 4 3 8 .
[29] H. H. Clark, S. E. Brennan, Grounding in communication, in: L. B. Resnick, J. M. Levine, S. D.</p>
      <p>Teasley (Eds.), Perspectives on Socially Shared Cognition, American Psychological Association,
Washington, DC, USA, 1991, p. 127–149. doi:1 0 . 1 0 3 7 / 1 0 0 9 6 - 0 0 6 .
[30] B. Smith, Against fantology, in: M. E. Reicher, J. C. Marek (Eds.), Experience and Analysis, öbv&amp;hpt,</p>
      <p>Vienna, Austria, 2005, pp. 153–170.
[31] D. R. Krathwohl, A revision of Bloom’s taxonomy: An overview, Theory Into Practice 41 (2002)
212–218. doi:1 0 . 1 2 0 7 / s 1 5 4 3 0 4 2 1 t i p 4 1 0 4 _ 2 .
[32] H. H. Clark, M. A. Krych, Speaking while monitoring addressees for understanding, Journal of</p>
      <p>Memory and Language 50 (2004) 62–81. doi:1 0 . 1 0 1 6 / j . j m l . 2 0 0 3 . 0 8 . 0 0 4 .
[33] H. H. Clark, Dogmas of understanding, Discourse Processes 23 (1997) 567–598. doi:1 0 . 1 0 8 0 /
0 1 6 3 8 5 3 9 7 0 9 5 4 5 0 0 3 .
[34] J. Allwood, S. Kopp, K. Grammer, E. Ahlsén, E. Oberzaucher, M. Koppensteiner, The analysis of
embodied communicative feedback in multimodal corpora: A prerequisite for behaviour simulation,
Language Resources and Evaluation 41 (2007) 255–272. doi:1 0 . 1 0 0 7 / s 1 0 5 7 9 - 0 0 7 - 9 0 5 6 - 2 .
[35] M. Heldner, A. Hjalmarsson, J. Edlund, Backchannel relevance spaces, in: Proceedings of Nordic</p>
      <p>Prosody XI, Tartu, Estonia, 2013, pp. 137–146.
[36] Z. Malisz, M. Włodarczak, H. Buschmeier, J. Skubisz, S. Kopp, P. Wagner, The ALICO corpus:
Analysing the active listener, Language Resources and Evaluation 50 (2016) 411–442. doi:1 0 . 1 0 0 7 /
s 1 0 5 7 9 - 0 1 6 - 9 3 5 5 - 6 .
[37] S. Grimm, Understanding, in: E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy, Fall
2023 ed., Metaphysics Research Lab, Stanford University, 2023. URL: https://plato.stanford.edu/
archives/fall2023/entries/understanding/.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>F. C.</given-names>
            <surname>Keil</surname>
          </string-name>
          , Explanation and understanding,
          <source>Annual Review of Psychology</source>
          <volume>57</volume>
          (
          <year>2006</year>
          )
          <fpage>227</fpage>
          -
          <lpage>254</lpage>
          . doi:
          <article-title>1 0 . 1 1 4 6 / a n n u r e v . p s y c</article-title>
          <source>h . 5 7 . 1 0</source>
          <volume>2 9 0 4 . 1 9 0 1 0 0 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>T.</given-names>
            <surname>Miller</surname>
          </string-name>
          ,
          <article-title>Explanation in artificial intelligence: Insights from the social sciences</article-title>
          ,
          <source>Artificial Intelligence</source>
          <volume>267</volume>
          (
          <year>2019</year>
          )
          <fpage>1</fpage>
          -
          <lpage>38</lpage>
          .
          <source>doi:1 0 . 1 0 1 6 / j . a r t i n t . 2 0 1 8 . 0 7 . 0 0 7 .</source>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>K. J.</given-names>
            <surname>Rohlfing</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Cimiano</surname>
          </string-name>
          , I. Scharlau,
          <string-name>
            <given-names>T.</given-names>
            <surname>Matzner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. M.</given-names>
            <surname>Buhl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Buschmeier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Esposito</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Grimminger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Hammer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Häb-Umbach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Horwath</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Hüllermeier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Kern</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Kopp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Thommes</surname>
          </string-name>
          , A.
          <string-name>
            <surname>-C. Ngonga Ngomo</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Schulte</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Wachsmuth</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Wrede</surname>
          </string-name>
          ,
          <article-title>Explanation as a social practice: Toward a conceptual framework for the social design of AI systems</article-title>
          ,
          <source>IEEE Transactions on Cognitive and Developmental Systems</source>
          <volume>13</volume>
          (
          <year>2021</year>
          )
          <fpage>717</fpage>
          -
          <lpage>728</lpage>
          .
          <source>doi:1 0 . 1 1</source>
          0 9 / T C D S .
          <volume>2 0 2 0 . 3 0 4 4 3 6 6 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>H.</given-names>
            <surname>Buschmeier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Attentive</given-names>
            <surname>Speaking</surname>
          </string-name>
          . From Listener Feedback to Interactive Adaptation,
          <source>Ph.D. thesis</source>
          , Faculty of Technology, Bielefeld University, Bielefeld, Germany,
          <year>2018</year>
          .
          <source>doi:1 0 . 4 1 1 9 / u n i b i / 2</source>
          <volume>9 1 8 2 9 5 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M.</given-names>
            <surname>Booshehri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Emele</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Flügel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Förster</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Frey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Frey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Glauer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hastings</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Hofmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Hoyer-Klick</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Hülk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Kleinau</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Knosala</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Kotzur</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Kuckertz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Mossakowski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Muschner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Neuhaus</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Pehl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Robinius</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Sehn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Stappel</surname>
          </string-name>
          , Introducing the Open Energy Ontology:
          <article-title>Enhancing data interpretation and interfacing in energy systems analysis</article-title>
          ,
          <source>Energy and AI</source>
          <volume>5</volume>
          (
          <year>2021</year>
          )
          <article-title>100074</article-title>
          .
          <source>doi:1 0 . 1 0</source>
          <volume>1 6</volume>
          / j . e
          <source>g y a i . 2 0</source>
          <volume>2 1 . 1 0 0 0 7 4 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>R.</given-names>
            <surname>Arp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. D.</given-names>
            <surname>Spear</surname>
          </string-name>
          ,
          <article-title>Building Ontologies with Basic Formal Ontology</article-title>
          , The MIT Press, Cambridge, MA, USA,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7] ISO/IEC, ISO/IEC 21838-2:
          <fpage>2021</fpage>
          -
          <article-title>Information technology - Top-level ontologies (TLO) - Part 2: Basic Formal Ontology (BFO), Standard 21838-2:2, International Organization for Standardization (ISO), Geneva</article-title>
          , Switzerland,
          <year>2021</year>
          . URL: https://www.iso.org/standard/74572.html.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>B.</given-names>
            <surname>Donohue</surname>
          </string-name>
          ,
          <article-title>Toward a BFO-based deontic ontology</article-title>
          ,
          <source>in: Papers of the 2nd Workshop on Representing Social</source>
          and
          <article-title>Legal Entities in the Biomedical Domain (SoLe-BD</article-title>
          <year>2017</year>
          ), volume
          <volume>2137</volume>
          , CEUR,
          <article-title>Newcastle-upon-</article-title>
          <string-name>
            <surname>Tyne</surname>
          </string-name>
          , UK,
          <year>2017</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          . URL: https://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>2137</volume>
          /ws_SoLe_paper_1.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>R.</given-names>
            <surname>Confalonieri</surname>
          </string-name>
          , G. Guizzardi,
          <source>On the multiple roles of ontologies in explainable AI</source>
          ,
          <year>2023</year>
          .
          <article-title>doi:1 0 . 4 8 5 5 0 / a r X i v . 2 3 1 1 . 0 4 7 7 8 . a r X i v : 2 3 1 1 . 0 4 7 7 8</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>D.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Abdul</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. Y.</given-names>
            <surname>Lim</surname>
          </string-name>
          ,
          <article-title>Designing theory-driven user-centric explainable AI</article-title>
          ,
          <source>in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems</source>
          , ACM, Glasgow, UK,
          <year>2019</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>15</lpage>
          .
          <source>doi:1 0 . 1 1</source>
          <volume>4 5 / 3 2 9 0 6 0 5 . 3 3 0 0 8 3 1 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>C.</given-names>
            <surname>Goodwin</surname>
          </string-name>
          ,
          <article-title>Why multimodality? Why co-operative action?, Social Interaction</article-title>
          .
          <source>Video-Based Studies of Human Sociality</source>
          <volume>1</volume>
          (
          <year>2018</year>
          ).
          <source>doi:1 0 . 7 1 4 6 / s i . v 1 i 2 . 1 1</source>
          <volume>0 0 3 9 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>S.</given-names>
            <surname>Chari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Seneviratne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. M.</given-names>
            <surname>Gruen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Foreman</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. K. Das</surname>
          </string-name>
          ,
          <string-name>
            <surname>D. L. McGuinness</surname>
          </string-name>
          ,
          <article-title>Explanation ontology: A model of explanations for user-centered AI</article-title>
          , in: International Semantic Web Conference, Springer,
          <year>2020</year>
          , pp.
          <fpage>228</fpage>
          -
          <lpage>243</lpage>
          .
          <source>doi:1 0 . 1 0</source>
          <volume>0 7 / 9 7 8 - 3 - 0 3 0 - 6 2 4 6 6 - 8</volume>
          _
          <fpage>1</fpage>
          5 .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>F.</given-names>
            <surname>Cabitza</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Campagner</surname>
          </string-name>
          , G. Malgieri,
          <string-name>
            <given-names>C.</given-names>
            <surname>Natali</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Schneeberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Stoeger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Holzinger</surname>
          </string-name>
          , Quod erat demonstrandum?
          <article-title>- Towards a typology of the concept of explanation for the design of</article-title>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>