<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Representation of Concepts in AI: Towards a Teleological Explanation</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Mattia FUMAGALLI</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Roberta FERRARIO</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Information Engineering and Computer Science (DISI) University of Trento</institution>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Laboratory for Applied Ontology, ISTC-CNR</institution>
          ,
          <addr-line>Trento</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The representation of concepts is a lively research activity in several artificial intelligence (AI) areas, such as knowledge representation, machine learning, and natural language processing. So far, many solutions have been proposed adopting different assumptions about the nature of concepts. Each of these solutions has been developed for capturing some specific features and for supporting some specific (artificial) cognitive operations. This paper provides a teleological explanation of the most widely shared approaches in AI to the representation of concepts. The paper aims at providing four main contributions: i) an overview of the mainstream philosophical theories of concepts; ii) a categorization of a wide range of AI solutions inspired by such theories of concepts; iii) the proposal of a method for a comprehensive explanation of the current approaches to concepts in AI based on a teleosemantic perspective; iv) an illustration of how the proposed explanation could constitute a contribution in the context of explainable AI.</p>
      </abstract>
      <kwd-group>
        <kwd />
        <kwd>Concepts</kwd>
        <kwd>Information</kwd>
        <kwd>Representation</kwd>
        <kwd>Cognition</kwd>
        <kwd>Semantics</kwd>
        <kwd>Artificial Intelligence</kwd>
        <kwd>Explainable AI</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        It is a widespread conviction, both in the psychological and in the philosophical literature
[
        <xref ref-type="bibr" rid="ref1 ref2">1,2</xref>
        ], that concepts are to be taken as essential tools for human thought. While there is
a lively debate on their specific nature, researchers are in total agreement on the pivotal
role of concepts in adequately explaining many cognitive phenomena like categorization,
learning, induction, language understanding, planning, decision-making, and so on [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        Within the field of Artificial Intelligence (AI), there is a huge amount of work trying
to enable most of the above mentioned cognitive phenomena in artificial agents.
Consequently, when some cognitive capabilities need to be implemented into an artificial
system, the tasks of choosing, modelling and organizing the best corresponding conceptual
system have to be carried out. So far, many architectures have been realized adopting
different approaches for the organization and the representation of artificial agents’
conceptual systems [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. All these computational architectures rely more or less explicitly on
state of the art theories that provide an interpretation of the structure of concepts as tools
for “thinking”.
      </p>
      <p>
        It can be generally observed that many different approaches to the computational
representation of concepts have gone a long way with many success stories. Anyhow,
by analyzing the current results in this area, two are the main considerations. Firstly, the
criteria for evaluating the relevance1 of a conceptual representation for enabling a certain
cognitive (artificial) phenomenon are often implicit. Secondly, all AI approaches to the
representation of concepts, considered in isolation, can efficiently account only for very
few aspects of cognition. Some models, for instance, are used for enabling systems to
reason on enormous amounts of data, but fail in accounting for trivial common-sense
reasoning [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Similarly, some conceptual representations are impressively successful
when used in well-defined domains, but they are completely inefficient in cross-domains
settings [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ].
      </p>
      <p>
        This paper offers four main contributions. (i) Based on the evidence that the work
in AI can take advantage of the philosophical research on concepts, we provide a brief
overview of the mainstream theories on concepts. Our goal is not to provide a
comprehensive survey of the state of the art theories on concepts. We refer the reader to excellent
and thorough surveys, such as [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] or [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], for that purpose. Our central aim is, instead, to
examine just some of the most relevant approaches, in order to make their assumptions
explicit and link them to a common terminological (and theoretical) ground. (ii) We
categorize a wide range of AI solutions on the basis of the introduced theories of concepts,
shedding light on what is the task (e.g., classification, learning. . . ) for which they are
selected. (iii) Leveraging a teleosemantic perspective [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], we introduce a method for a
comprehensive explanation of the current approaches to concepts in AI. This high-level
theoretically grounded explanation may serve, in our view, as a blueprint for making
explicit how a given approach to concepts may be relevant in relation to a certain artificial
(cognitive) task to be addressed. (iv) Finally, we illustrate how the proposed approach
may constitute a contribution in the context of explainable AI and may offer criteria for
devising better solutions and, eventually, virtuous combinations of existing approaches.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Theories of Concepts: a Bird’s-Eye View</title>
      <p>In this section, we will try and sketch the main philosophical theories about concepts.</p>
      <p>
        Among the most known theories of concepts, we can count the group of the
socalled classical-symbolic approaches. According to these approaches, concepts are
explicit representations codified in a language, similar to the first-order predicate calculus.
The main features of this type of representations, also called propositional
representations [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], are arbitrariness and discreteness. Concepts can be seen indeed as symbols
of the language of thought (LOT) [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. They are arbitrary in the sense that no
similarity is needed between them and what they represent. They are discrete because they are
either complex expressions separable in smaller parts, or atomic parts without any
internal structure. Arbitrariness and discreteness allow the propositional representations of
concepts to be highly formal, abstract and composititional.
      </p>
      <p>
        Developed as an alternative paradigm with respect to the classical-symbolic
approaches, the connectionist research program has a long story that dates back to the 40s
[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. The many success stories of the symbolic approach around the 50s and 60s
1“Something (A) is relevant to a task (T) if it increases the likelihood of accomplishing the goal (G), which
is implied by T” [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
put connectionism in the shade for a long period. However, in the late 80s, it began to
increase again its popularity. Connectionism shares the computational hypothesis of the
symbolic approach, but provides a different model for concepts. In particular,
according to this view, concepts can be seen as representations distributed throughout a large
number of processing elements. Concepts are embedded in a network composed by
interconnected units, which, at a certain level of abstraction, simulate the behavior of a
conglomerate of neural cells [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
      </p>
      <p>
        A third family of approaches is grounded on the so-called embodied/situated
theories, which hold that cognition is the product not only of what happens in the mind, but
also in the body and in the environment [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. The embodied theories program is quite
recent and has not yet been consolidated in a wholly systematic theory, however it is
being tested and used in many AI researches and applications (e.g., dynamical systems
[
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]).
      </p>
      <p>
        We can distinguish also a group of less widely spread theories, which, however,
play a pivotal role on the explanation of essential cognitive behaviors as well. Under this
group we have procedural theories, being firstly asserted during the 70s and holding that
it is not necessary for a concept to be explicitly represented as a mental symbol [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ].
According to these theories, concepts can be implicitly represented as a procedure, i.e.,
as the execution of a piece of an algorithm. Within this framework, having a concept
amounts to having a capability to do something. For instance, having the concept of ‘Cat’
is the same thing as having the ability of recognizing something as a ‘Cat’, or having the
capability of using it in inference processes (i.e., inferring that it is an animal).
      </p>
      <p>
        Moreover, another new interpretation of concepts was introduced around the late
60s, leading to the family of analogical theories, which purport that concepts are
analogical representations (and not propositional, like in classical-symbolic theories). These
kinds of representations are defined as mental objects that are similar to the objects they
represent, like, for instance, a picture of a cat or the image of a cat on my eye’s retina
[
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. Differently from propositional concepts, analogical concepts are not claimed to be
discrete. This means that concepts do not provide a selection of features, rather they
collect the whole perceptual information. This is a value if concreteness and completeness
are the target, but it is a problem with respect to compositionality and abstraction [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
Another interesting issue is that, with their representation of concepts, analogical
theories provide an account for simulation processes in cognition, where specific distributed
collections of information, captured through experience, may function as “collectors” of
multiple conceptualizations for a single category (see for instance the notion of proxytype
in [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] and the notion of simulator in [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]).
      </p>
      <p>
        Finally, other three approaches to concepts that deserve to be considered here are:
i) the prototypical approach; ii) the exemplar approach and the iii) theory approach
[
        <xref ref-type="bibr" rid="ref19 ref21">21,19</xref>
        ]. Very briefly, in the prototypical approach, concepts provide the representation
of the most “typical” occurrence for a given class of objects. Concepts are prototypes,
i.e., a sort of weighted set of features (e.g., the prototype for ‘apple’ is something
approximately round, green, red or yellow, with a specific range of weight, and so forth).
In the exemplar view, concepts are seen as devices storing information about specific
example occurrences for a given perceived object (e.g., the information about the apples
we encountered in our experience). Within the “theory” approach, concepts are instead
represented as (micro-)theories. For instance, having a concept for ‘apple’ means having
a (micro-)theory about apples.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. Theories of Concepts in AI</title>
      <p>After this sketchy presentation of the main philosophical theories on concepts, we will
now list in this section some AI approaches that have been inspired by such theories.</p>
      <p>
        A typical example of a computational approach to concept representation that is
influenced by classical-symbolic principles are formal ontologies. The most widely shared
definition of ontology in the computer science community is Gruber’s [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]: “a formal,
explicit specification of a shared conceptualization”2. Ontologies can be seen as complex
data structures, i.e., information artifacts, which can be designed (and formalized)
using different representational languages, as for instance first order logic (FOL), or some
computable fragment of it, like RDF3 and OWL4, following clear methodological
principles (e.g., OntoClean [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ]). All the languages used for representing these
“conceptualizations” can thus be seen as instantiations of what in the classical-symbolic frame is
taken as LOT. The main goal of these artifacts is to support knowledge representation
(KR) and integration tasks, but they can be used for other tasks as well (for instance to
drive NLP, or to provide data exchange formats).
      </p>
      <p>Neural networks are typical computational representations inspired by the
connectionist view of concepts. So far, even if they cannot be considered as proper models of
real neural systems, different types of (artificial) neural networks have been successfully
adopted for addressing specific AI tasks. These artifacts can be reduced to a set of
interconnected units, i.e., abstract representation of neurons, where any connection between
these neurons is an abstract representation of a synapse. According to these
representations, each unit is associated to a numerical value, i.e., an activation state (or firing,
namely the frequency by which a neuron sends signals through synapses). Each
connection between neuron representation units is characterized by a weight that codifies the
strength of that connection. The influence of a unit x on a unit y is given by the activation
value of the unit x multiplied by the weight of the connection from x to y. The weight
value can be positive or negative, so that the signal sent through the connection can
activate or deactivate the neuron reached by the signal. So far, a lot of neural networks have
been devised for capturing aspects of cognition, mainly connected with learning. For an
overview and a collection of related papers we refer the reader to the “Neural Network
Zoo” web page5.</p>
      <p>
        The AI approaches to concepts grounded on the embodied (or situated) theories are
usually implemented by the situated robotics research program. A key exemplification of
these approaches is the work by the research group at MIT6, headed by Rodney Brooks
[
        <xref ref-type="bibr" rid="ref25">25</xref>
        ]. This group is building robots that are equipped with simple sensory-motor devices
and a collection of modules. Each of these modules is specialized for addressing a
specific task, such as checking for the presence of an obstacle, avoiding an obstacle,
exploring, and so forth. Each of these activities is run by a processor that works together with
other processors and exchange information with the sensory-motor system and other
processors. In these models no explicit representations are provided and no data is stored.
The robots are not equipped with a mental model; they are automata that can be described
2A thorough analysis of this and other definitions of ontology may be found in [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ].
3https://www.w3.org/RDF/
4https://www.w3.org/OWL/
5http://www.asimovinstitute.org/neural-network-zoo/
6https://www.csail.mit.edu/
just through finite states [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ]. All the information used by these agents is grasped from
the environment. Here concepts can be seen only as temporary representations,
information flows, built upon the different phases of the perceptual process. The main goal is to
derive the useful information from the environment, send it to the right processors and
then produce an action. Thus, every robot can be seen just as a collection of behaviors in
competition [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ]. From an external point of view, it is possible to detect coherent
behavioral patterns. However, locally, these robots are characterized by just casual processes.
The robots devised following the situated approaches are able to reproduce the cognitive
capabilities of some insects, and, according to recent results, it seems they can be evolved
by introducing new connected processing modules.
      </p>
      <p>
        In AI the analogical approaches are well-supported by research results like [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] and
raise the issue of how some artificial cognitive process are related to imagination and
deal with mental images. The underlying assumption of these computational frameworks
is that perception and the relation with the external environment play a central role in
cognition. This leads them to focus on the relevance of simulation processes and to share
some hypothesis with the embodied approaches to representation. Though there are still
few computational frameworks implementing the analogical approach, recently, some
solutions grounded on this paradigm are being developed. For instance, the work in [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]
aims at providing a computational account of cognition in modality-specific processing
[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Examples of attempts at implementing simulations can be found in [
        <xref ref-type="bibr" rid="ref28 ref29">28,29</xref>
        ]. There
are instead more computational frameworks implementing the ideas of procedural
approaches. For those in AI starting from the procedural frame, the key idea is that
concepts can be implicitly represented as fragments of algorithms. Concepts can be reduced
to a sort of know-how that is not explicitly representable by means of data structures.
However, these algorithms need some explicit information, or data structures, to work.
Thus, procedural approaches do not exclude the possibility that the mental content is
partially built on some explicit information, but they state that such content is mainly
determined by the operations performed over it. Every representation is both involved in
a causal relation with the external environment and in a causal relation with some mental
operations. Good examples of computational frameworks linked to procedural semantics
are semantic networks like KL-ONE [
        <xref ref-type="bibr" rid="ref30">30</xref>
        ] (for a detailed description see [
        <xref ref-type="bibr" rid="ref31 ref32">31,32</xref>
        ]) and
resources like WordNet or FrameNet, inspired by Inferential Role Semantics (IRS),
Lexical Semantics (LS) or Frame Semantics [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ], i.e., semantic theories that underlie most
of the procedural assumptions.
      </p>
      <p>
        Finally, we can also find some works in AI explicitly developing some of the ideas
grounding prototypical, exemplar and theory approaches. A computational work
exploiting some of the features of prototypical and exemplar theories is the one by Lieto and
Frixione [
        <xref ref-type="bibr" rid="ref34">34</xref>
        ], which is also partially inspired by the theory of conceptual spaces [
        <xref ref-type="bibr" rid="ref35">35</xref>
        ].
Here the main goal is to combine the typicality effects of a prototypical representation
with the compositionality effects of a more classical representation of concepts [
        <xref ref-type="bibr" rid="ref36">36</xref>
        ]. The
result is a sort of hybrid architecture, i.e., what they call DUAL-PECCS [
        <xref ref-type="bibr" rid="ref37">37</xref>
        ]. This is
basically an integrated KR system aiming at supporting artificial cognitive capabilities,
such as categorization, by implementing classical, prototypical and exemplar-based
representations of concepts. For what concerns “theory” approaches, to some extent, we
may say that core ontologies are examples of their computational applications. As an
example, take the ORGANIZATION core ontology: as already shown, this is a typical
formalism grounded on the symbolic frame, however it can be also seen as a (formal)
micro-theory representing the domain-specific concepts.
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. Teleosemantics: a Teleological Approach to Concepts</title>
      <p>In the following, we introduce a further philosophical theory on concepts, based on a
view of concepts which is alternative to all the previous ones and that we would like to
exploit for the development of a framework.</p>
      <p>
        Teleosemantics provides an account of concepts and their representations by
leveraging the notion of (etiological) function and the notions of producer and consumer
devices7. According to this theory, the representations of concepts are kinds of
informational states shared between a producer and a consumer device [
        <xref ref-type="bibr" rid="ref38 ref39 ref9">9,38,39</xref>
        ], which must
be equipped with specific etiological functions. In the generation of a conceptual
representation, the function of the producer is always to generate a state (the representation)
according to a certain situation, i.e., when another state obtains (the representatum). The
function of the consumer is to act in a certain way when the conceptual representation
communicated by the producer has been received.
      </p>
      <p>
        Differently from more classical approaches, in which content of a conceptual
representation is seen as its objective referent, according to teleosemantics, content is
determined by a success condition, i.e., the condition that explains why a consumer acts
successfully once a conceptual representation has been received. This situation can be
easily illustrated by the example in Figure 1. Like many other animals, bees produce
signals to inform other individuals of the same (or other) species. In Figure 1 we have
a bee (producer device) that has the role of helping other bees in reaching a spot with
nectar. The producer bee makes a special dance describing the distance and the direction
to reach the place. The consumer bee (or some device within the bee) is responsible for
7Here the notion of device has to be taken in a broad sense, including, for instance, the perceptual apparatus
of human beings and animals, or, according to a higher level of granularity, complex organisms.
the interpretation of the producer bee’s dance. The conceptual representation is the bee’s
dance itself. In the example, the function of the producer is to generate a dance in certain
circumstances and the function of the consumer is to interpret it and reach the nectar
place. Since reaching the nectar place is only successful when the nectar is found (when
there is no nectar the trip is only a waste of energy), this is the success condition of
the actions/behavior prompted by the bee’s dance. Consequently, the bee’s dance means
something like “the nectar is exactly there” [
        <xref ref-type="bibr" rid="ref40">40</xref>
        ].
      </p>
      <p>Looking at the diagram shown in Figure 1, there is another important item to be
explained, namely the one that is labeled as “reward” (see the edge from “Success?”
to the group box), which represents what is obtained by the involved bees (devices)
when their interaction is successful and that also accounts for the likelihood of this same
strategy being adopted again in the future.</p>
      <p>In these terms, describing what a conceptual representation represents amounts to
considering the state of affairs that explains when the actions generated by the conceptual
representation are successful. Such state of affairs is the condition represented by the
conceptual representation.</p>
      <p>The model represented above in Figure 1 can be then described more precisely as
follows:</p>
      <sec id="sec-4-1">
        <title>A conceptual representation R has content x iff:</title>
        <p>R can be defined as input/output in a system consisting of a producer (device) P
and a consumer (device) C;
The function of the producer P is to produce R to obtain x;
x is the success condition, dependent on R, of C’s action caused by R.</p>
        <p>The same schema illustrated in Figure 1 can be used to explain conceptual
representations in artificial contexts and, more specifically, to classify AI approaches based
on the function they carry out or, better, the function carried out by concepts in such
approaches. Let us assume that the conceptual representation is in this case a data structure,
like an ontology representing a classification of places and locations. In this situation, the
producer can be defined as the knowledge engineer designing the ontology, who has the
role of helping other agents in reasoning about geographical information. The consumer
is responsible for the interpretation of that produced data structure. The conceptual
representation is, as it has been already said, the data structure itself. In this example, the
function of the knowledge engineer (producer) is to generate the ontology in such a way as to
enable a correct navigation of the space and the function of the consumer agent (notice
that here we assume that it may be an artificial agent as well) is to run effective inferences
about geographical information and move according to what is inferred. Since running
effective inferences about geographical information is only successful when the agent
can move correctly in a specific area by following the instructions expressed or inferred
from the data structure, this is the success condition of the actions/behavior prompted by
the ontology.</p>
        <p>
          Of course, this is only a very sketchy description of the teleosemantic framework.
However, in the next section, we will try and illustrate how this framework can be applied
to the case of Artificial Intelligence technologies, so as to classify them based on the
function they are meant to accomplish.
In our view, the latter example shows that teleosemantics can be effectively used as a
high-level model for explaining AI solutions based on specific theories of concept
representation, and not only for explaining the representation of concepts in biological
systems. The roles of producer and consumer devices may be played, indeed, by
organizations, humans or even artificial devices (e.g., software applications or robots), which
are able to produce and/or use a conceptual representation (just think about the
automatic generation of ontologies or reasoners applied to them). It is not difficult to see
each of these devices as equipped with the function in relation to which the conceptual
representation is produced or consumed. The “Conceptual Representation” category (see
Figure 1) captures AI approaches (theories and applications that operate on concepts,
as internally defined) like the ones introduced in Section 3. “Action” can be seen as the
category grouping kinds of executions of artificial cognitive tasks in which the
conceptual representations can be involved. To have an idea of the items that could populate
such category, just think about the group of AI activities introduced in [
          <xref ref-type="bibr" rid="ref41">41</xref>
          ], i.e.:
problem solving, knowledge and reasoning, acting logically, uncertain knowledge and
reasoning, learning, communicating, perceiving and acting. The “Success condition” class
collects the descriptions of the “purposes” for which the “conceptual representations”
are used by consumers. “Reward” represents what is gained once that the AI technology
is successfully used.
        </p>
        <p>The teleosemantics schema can also be used to characterize more specific AI
technologies, not only kinds of approaches, as in the following example, where a specific
data structure plays the role of “Conceptual representation”:</p>
      </sec>
      <sec id="sec-4-2">
        <title>CONCEPT-REPRESENTATION: “Google K-Graph”</title>
        <p>PRODUCER: “Google LLC”
PRODUCER-FUNCTION: “to support people general information search”
CONSUMER: “Person”
CONSUMER-FUNCTION: “to find trustworthy medical information”
ACTION: “to query Google search”
SUCCESS-CONDITION: “trustworthy medical information is found”
REWARD: “?”</p>
        <p>The above structure can be used to generate a database where different conceptual
representations are classified and characterized according to their usage. This database
can be a reference resource for people who need to assess, adopt and, eventually, combine
existing AI solutions. Notice that the kinds of conceptual representations that can be put
in the framework can be of different levels of granularity (and also that it can be applied
either to types or tokens). For instance, in the above example, Google Knowledge Graph8
could be easily replaced by the general notion of “Knowledge Graph”.</p>
        <p>
          What has been left undefined in the example is what the notion of “Reward”
classifies. In the next section we will focus both on this specific notion and on a general
framework to propose a novel perspective on Explainable AI [
          <xref ref-type="bibr" rid="ref42 ref43">42,43</xref>
          ].
        </p>
        <p>8https://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-not.
html</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>6. A Telelogical Perspective on Explainable AI</title>
      <p>Nowadays, society is experiencing a very rapid growth and increasing adoption of AI
technologies. While it is quite obvious that AI is more and more able to enhance the
quality of life of individuals and communities, researchers, practitioners, institutions
and users should also acknowledge the fact that new risks could emerge from a more
widespread use of such technologies. For this reason, at the beginning of April 2019, the
European Commission (EC) released some ethics guidelines for trustworthy AI9.</p>
      <p>The document produced by the EC is aimed at ensuring adherence to European
ethical principles and values and the implementation of AI applications, which are robust,
not only from the technical point of view, but also from the social one, as these are seen
as components of wider socio-technical systems.</p>
      <p>In what follows we will try and show how the framework we are proposing may
contribute to enable many of the requirements inspired by the ethical imperatives listed in
the guidelines. Such contribution will be deployed in two main directions: transparency
and explainability, which will then have an impact on further dimensions.</p>
      <p>Transparency10 will be ensured by the adoption of a framework that highlights and
makes explicit all the aspects of an AI application which may create concerns. Less
straightforwardly, explainability11 will be granted by the functional perspective provided
by the use of teleosemantics as a driving methodology which pinpoints the function AI
applications play for the producer and consumer agents when the success conditions are
met.</p>
      <p>Transparency and explainability are referred to in the guidelines as principle of
explicability, and this is what is explicitly addressed by the approach we are proposing.
Nonetheless, indirect contributions are foreseen also for other principles: human
autonomy could be enhanced by allowing human agents to decide to change the AI
application in use and choose a better fit to their needs in case the success conditions are not
met; prevention of harm could be improved by reducing the information asymmetries of
the involved human agents; fairness could be favored by making human actors aware of
the function the application should serve and of whether such function was successfully
achieved, otherwise the use of the application may be contested. One thing that should
also be noted is that the potential tensions between the realization of different principles
can also be better singled out by making explicit the functions that the application is
supposed to play for the different agents involved (producer and consumer).</p>
      <p>As it should have become clear by now, the categories singled out by the framework
we have produced seem especially fit to document the most important characteristics of
an AI technology12 from the point of view of its explainability: the conceptual
representation (the data structure that underlies the technology), the producer (the agent who
cre9https://ec.europa.eu/futurium/en/ai-alliance-consultation/guidelines
10Transparency is defined in the guidelines as: “This requirement is closely linked with the principle of
explicability and encompasses transparency of elements relevant to an AI system: the data, the system and the
business models.”.</p>
      <p>11Explainability is defined in the guidelines as: “Explainability concerns the ability to explain both the
technical processes of an AI system and the related human decisions (e.g. application areas of a system).
Technical explainability requires that the decisions made by an AI system can be understood and traced by
human beings”.</p>
      <p>12We use “technology” here in a very general sense, it may refer to a theoretical approach, and to an
application as well.
ated or sold it to the consumer), the consumer (the agent who is going to use it), the
function of the producer (what the producer wants the consumer to do, by the technology), the
function of the consumer (what the consumer wants to accomplish with the technology),
the action (that the consumer executes with the technology) and the success condition
(the outcome of the action that, when successful, provides the conceptual representation
with content) and, finally, the reward, what makes the producer and the consumer
understand that the communication went through and can be re-used in successive, similar
situations.</p>
      <p>But there is another sense in which our approach targets explainability in AI, i.e. by
characterizing ontologically the notion of explanation.</p>
      <p>The term “explanation” may be used to indicate both a process and an object (which
is a kind of reification13 of the process, its outcome). As a process, explanation may be
seen as a social process, a communication from the explainer (the producer) to the
explainee (the consumer), which happens through a conceptual representation whose
content is given by an action that shows that the communication was successful. The success
condition of such communication is then reified in a reward, which is the explanation as
an object.</p>
      <p>
        We believe there is some similitude between this characterization we are purporting
and the one proposed by Miller in [
        <xref ref-type="bibr" rid="ref45">45</xref>
        ], for two main reasons. The first is that Miller sees
explanation as post-hoc interpretability and in fact we may say that a representation
concept or a data structure is interpreted only once the consumer’s successful action shows
its content. In other words, content can be ascribed to representations (or AI
technologies) only ex post, after that the consumer’s actions have been successfully performed.
Thus, explanation, even in our framework, comes out as a form of abductive
reasoning. The second reason is that Miller sees explanation as connected to a weak notion of
causality, a kind of functional causality, such that the explanation is not deterministically
inferred by the cause, but is selected among many possible causes. This also means that
both causality and explanation end up to be contextual. All this is very similar to the
teleosemantic approach, in which, among the possible explanations that one could give,
the one which shows that the representation (or the technology) worked (functioned) is
selected.
      </p>
      <p>Finally, a very nice aspect of the framework we are proposing is that it is adaptable
to many scenarios in AI in which an explanation is required. This includes the examples
of the previous section, in which the explanation is the description of the successful
accomplishment of a specific task by a specific AI technology, but also our own account,
in which the explanation is the description of the successful accomplishment of a kind of
task by a kind of AI technology.</p>
    </sec>
    <sec id="sec-6">
      <title>7. Conclusions and Future Work</title>
      <p>The paper was aimed at providing a high-level model to support a comprehensive
explanation of the current approaches to concepts representation in AI, based on the
teleosemantics theory.</p>
      <p>
        13We are intending here “reification” in a technical way, similarly as in [
        <xref ref-type="bibr" rid="ref44">44</xref>
        ], where reification is applied to
roles and social concepts.
      </p>
      <p>In order to be able to capture the main teleosemantics elements of such AI
approaches, a preliminary step was to examine the most important ones at the state of the
art, together with the philosophical theories on concepts by which they were inspired.</p>
      <p>A further contribution of the paper is to show how the proposed teleological
framework could constitute a theoretical tool to foster explainable AI.</p>
      <p>Among the potential paths of research opened up by the current investigations, two
we deem particularly worth to be pursued. A first perspective concerns the population
of a database where existing AI approaches to concepts can be stored and
characterized. We plan to devise the database schema through the definition of an ontology where
the semantics of the main teleosemantic notions are made formally explicit. A second
research direction concerns the central role of the notion of “reward”. We wish to
formalize reward within the teleosemantic explanation and try to understand whether, how,
and in which case, this is related to the evaluation and usage of the stored/characterized
approaches.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Lorraine</surname>
            <given-names>K Tyler</given-names>
          </string-name>
          ,
          <article-title>HE Moss, MR Durrant-Peatfield, and JP Levy</article-title>
          .
          <article-title>Conceptual structure and the structure of concepts: A distributed account of category-specific deficits</article-title>
          .
          <source>Brain and language</source>
          ,
          <volume>75</volume>
          (
          <issue>2</issue>
          ):
          <fpage>195</fpage>
          -
          <lpage>231</lpage>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Lawrence</surname>
            <given-names>W Barsalou.</given-names>
          </string-name>
          <article-title>The human conceptual system</article-title>
          .
          <source>The Cambridge handbook of psycholinguistics</source>
          , pages
          <fpage>239</fpage>
          -
          <lpage>258</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Gregory</given-names>
            <surname>Murphy</surname>
          </string-name>
          .
          <article-title>The big book of concepts</article-title>
          . MIT press,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>David</given-names>
            <surname>Vernon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Giorgio</given-names>
            <surname>Metta</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Giulio</given-names>
            <surname>Sandini</surname>
          </string-name>
          .
          <article-title>A survey of artificial cognitive systems: Implications for the autonomous development of mental capabilities in computational agents</article-title>
          .
          <source>IEEE transactions on evolutionary computation</source>
          ,
          <volume>11</volume>
          (
          <issue>2</issue>
          ):
          <fpage>151</fpage>
          -
          <lpage>180</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Birger</given-names>
            <surname>Hjørland</surname>
          </string-name>
          and Frank S Christensen.
          <article-title>Work tasks and socio-cognitive relevance: A specific example</article-title>
          .
          <source>Journal of the American Society for Information Science and Technology</source>
          ,
          <volume>53</volume>
          (
          <issue>11</issue>
          ):
          <fpage>960</fpage>
          -
          <lpage>965</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Ernest</given-names>
            <surname>Davis</surname>
          </string-name>
          and
          <string-name>
            <given-names>Gary</given-names>
            <surname>Marcus</surname>
          </string-name>
          .
          <article-title>Commonsense reasoning and commonsense knowledge in artificial intelligence</article-title>
          .
          <source>Commun. ACM</source>
          ,
          <volume>58</volume>
          (
          <issue>9</issue>
          ):
          <fpage>92</fpage>
          -
          <lpage>103</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>David</given-names>
            <surname>Silver</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Aja</given-names>
            <surname>Huang</surname>
          </string-name>
          , Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam,
          <string-name>
            <given-names>Marc</given-names>
            <surname>Lanctot</surname>
          </string-name>
          , et al.
          <article-title>Mastering the game of go with deep neural networks and tree search</article-title>
          .
          <source>nature</source>
          ,
          <volume>529</volume>
          (
          <issue>7587</issue>
          ):
          <fpage>484</fpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>D</given-names>
            <surname>.</surname>
          </string-name>
          Braddon-Mitchell and
          <string-name>
            <given-names>F.</given-names>
            <surname>Jackson</surname>
          </string-name>
          .
          <article-title>Philosophy of Mind and Cognition: An Introduction</article-title>
          . Wiley,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>Graham</given-names>
            <surname>Macdonald</surname>
          </string-name>
          ,
          <string-name>
            <surname>David Papineau</surname>
          </string-name>
          , et al.
          <source>Teleosemantics</source>
          . Oxford University Press,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>Shaun</given-names>
            <surname>Gallagher</surname>
          </string-name>
          and
          <string-name>
            <given-names>Dan</given-names>
            <surname>Zahavi</surname>
          </string-name>
          .
          <article-title>The phenomenological mind</article-title>
          .
          <source>Routledge</source>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Jerry</surname>
            <given-names>A Fodor.</given-names>
          </string-name>
          <article-title>LOT 2: The language of thought revisited</article-title>
          . Oxford University Press on Demand,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Warren</surname>
            <given-names>S McCulloch</given-names>
          </string-name>
          and
          <string-name>
            <given-names>Walter</given-names>
            <surname>Pitts</surname>
          </string-name>
          .
          <article-title>A logical calculus of the ideas immanent in nervous activity</article-title>
          .
          <source>The bulletin of mathematical biophysics</source>
          ,
          <volume>5</volume>
          (
          <issue>4</issue>
          ):
          <fpage>115</fpage>
          -
          <lpage>133</lpage>
          ,
          <year>1943</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Donald</surname>
            <given-names>O</given-names>
          </string-name>
          <string-name>
            <surname>Hebb.</surname>
          </string-name>
          <article-title>The organization of behavior: A neuropsychological theory</article-title>
          . Psychology Press,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Jerry</surname>
            <given-names>A Fodor</given-names>
          </string-name>
          and Zenon W Pylyshyn.
          <article-title>Connectionism and cognitive architecture: A critical analysis</article-title>
          .
          <source>Cognition</source>
          ,
          <volume>28</volume>
          (
          <issue>1-2</issue>
          ):
          <fpage>3</fpage>
          -
          <lpage>71</lpage>
          ,
          <year>1988</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>F.J.</given-names>
            <surname>Varela</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Thompson</surname>
          </string-name>
          , E. Rosch, and J.
          <string-name>
            <surname>Kabat-Zinn</surname>
          </string-name>
          .
          <source>The Embodied Mind: Cognitive Science and Human Experience</source>
          . The MIT Press. MIT Press,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Randall</surname>
            <given-names>D</given-names>
          </string-name>
          <string-name>
            <surname>Beer</surname>
          </string-name>
          .
          <article-title>A dynamical systems perspective on agent-environment interaction</article-title>
          .
          <source>Artificial intelligence</source>
          ,
          <volume>72</volume>
          (
          <issue>1-2</issue>
          ):
          <fpage>173</fpage>
          -
          <lpage>215</lpage>
          ,
          <year>1995</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Philip</surname>
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Johnson-Laird</surname>
          </string-name>
          .
          <article-title>Procedural semantics</article-title>
          .
          <source>Cognition</source>
          ,
          <volume>5</volume>
          (
          <issue>3</issue>
          ):
          <fpage>189</fpage>
          -
          <lpage>214</lpage>
          ,
          <year>1977</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Roger</surname>
            <given-names>N Shepard</given-names>
          </string-name>
          and
          <string-name>
            <given-names>Jacqueline</given-names>
            <surname>Metzler</surname>
          </string-name>
          .
          <article-title>Mental rotation of three-dimensional objects</article-title>
          .
          <source>Science</source>
          ,
          <volume>171</volume>
          (
          <issue>3972</issue>
          ):
          <fpage>701</fpage>
          -
          <lpage>703</lpage>
          ,
          <year>1971</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Jesse</surname>
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Prinz</surname>
          </string-name>
          .
          <article-title>Furnishing the Mind: Concepts and Their Perceptual Basis</article-title>
          . MIT Press,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Lawrence W Barsalou</surname>
          </string-name>
          , W Kyle Simmons,
          <article-title>Aron K Barbey,</article-title>
          and
          <string-name>
            <given-names>Christine D</given-names>
            <surname>Wilson</surname>
          </string-name>
          .
          <article-title>Grounding conceptual knowledge in modality-specific systems</article-title>
          .
          <source>Trends in cognitive sciences, 7</source>
          (
          <issue>2</issue>
          ):
          <fpage>84</fpage>
          -
          <lpage>91</lpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>Eleanor</given-names>
            <surname>Rosch</surname>
          </string-name>
          .
          <article-title>Cognitive representations of semantic categories</article-title>
          .
          <source>Journal of experimental psychology: General</source>
          ,
          <volume>104</volume>
          (
          <issue>3</issue>
          ):
          <fpage>192</fpage>
          ,
          <year>1975</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Thomas</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Gruber</surname>
          </string-name>
          .
          <article-title>A translation approach to portable ontology specifications</article-title>
          .
          <source>KNOWLEDGE ACQUISITION</source>
          ,
          <volume>5</volume>
          :
          <fpage>199</fpage>
          -
          <lpage>220</lpage>
          ,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Nicola</surname>
            <given-names>Guarino</given-names>
          </string-name>
          , Daniel Oberle, and
          <string-name>
            <given-names>Steffen</given-names>
            <surname>Staab</surname>
          </string-name>
          .
          <article-title>What is an ontology? In Handbook on ontologies</article-title>
          , pages
          <fpage>1</fpage>
          -
          <lpage>17</lpage>
          . Springer,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>Nicola</given-names>
            <surname>Guarino</surname>
          </string-name>
          and
          <string-name>
            <given-names>Christopher</given-names>
            <surname>Welty</surname>
          </string-name>
          .
          <article-title>Evaluating ontological decisions with ontoclean</article-title>
          .
          <source>Commun. ACM</source>
          ,
          <volume>45</volume>
          (
          <issue>2</issue>
          ):
          <fpage>61</fpage>
          -
          <lpage>65</lpage>
          ,
          <year>February 2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Rodney</surname>
            <given-names>Allen</given-names>
          </string-name>
          <string-name>
            <surname>Brooks</surname>
          </string-name>
          .
          <article-title>Cambrian intelligence: The early history of the new AI</article-title>
          . MIT press,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>Amedeo</given-names>
            <surname>Giorgi</surname>
          </string-name>
          .
          <article-title>The phenomenological mind: An introduction to philosophy of mind and cognitive science</article-title>
          .
          <source>Journal of Phenomenological Psychology</source>
          ,
          <volume>40</volume>
          (
          <issue>1</issue>
          ):
          <fpage>107</fpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Rodney</surname>
            <given-names>A</given-names>
          </string-name>
          <string-name>
            <surname>Brooks</surname>
          </string-name>
          .
          <article-title>Intelligence without representation</article-title>
          .
          <source>Artificial intelligence</source>
          ,
          <volume>47</volume>
          (
          <issue>1-3</issue>
          ):
          <fpage>139</fpage>
          -
          <lpage>159</lpage>
          ,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>D</given-names>
            <surname>Joyce</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L Richards</given-names>
            ,
            <surname>Angelo Cangelosi</surname>
          </string-name>
          , and
          <string-name>
            <surname>Kenny R Coventry.</surname>
          </string-name>
          <article-title>On the foundations of perceptual symbol systems: Specifying embodied representations via connectionism</article-title>
          .
          <source>In The logic of cognitive systems: Fifth International Conference on Cognitive Modeling</source>
          , pages
          <fpage>147</fpage>
          -
          <lpage>152</lpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Randall C O'Reilly.</surname>
          </string-name>
          <article-title>Six principles for biologically based computational models of cortical cognition</article-title>
          .
          <source>Trends in cognitive sciences</source>
          ,
          <volume>2</volume>
          (
          <issue>11</issue>
          ):
          <fpage>455</fpage>
          -
          <lpage>462</lpage>
          ,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Ronald J Brachman and James G Schmolze</surname>
          </string-name>
          .
          <article-title>An overview of the kl-one knowledge representation system</article-title>
          .
          <source>In Readings in artificial intelligence and databases</source>
          , pages
          <fpage>207</fpage>
          -
          <lpage>230</lpage>
          . Elsevier,
          <year>1989</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Nicholas</surname>
            <given-names>V</given-names>
          </string-name>
          <string-name>
            <surname>Findler</surname>
          </string-name>
          .
          <article-title>Associative networks: Representation and use of knowledge by computers</article-title>
          . Academic Press,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Ronald</surname>
            <given-names>J</given-names>
          </string-name>
          <string-name>
            <surname>Brachman</surname>
          </string-name>
          .
          <article-title>On the epistemological status of semantic networks</article-title>
          .
          <source>In Associative networks</source>
          , pages
          <fpage>3</fpage>
          -
          <lpage>50</lpage>
          . Elsevier,
          <year>1979</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Jerry</surname>
            <given-names>A</given-names>
          </string-name>
          <string-name>
            <surname>Fodor. Concepts</surname>
          </string-name>
          :
          <article-title>Where cognitive science went wrong</article-title>
          . Oxford University Press,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Antonio</surname>
            <given-names>Lieto</given-names>
          </string-name>
          , Andrea Minieri, Alberto Piana, and
          <string-name>
            <surname>Daniele P Radicioni.</surname>
          </string-name>
          <article-title>A knowledge-based system for prototypical reasoning</article-title>
          .
          <source>Connection Science</source>
          ,
          <volume>27</volume>
          (
          <issue>2</issue>
          ):
          <fpage>137</fpage>
          -
          <lpage>152</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <given-names>Peter</given-names>
            <surname>Ga</surname>
          </string-name>
          <article-title>¨rdenfors. The geometry of meaning: Semantics based on conceptual spaces</article-title>
          . MIT Press,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Antonio</surname>
            <given-names>Lieto</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>Christian</given-names>
            <surname>Lebiere</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Alessandro</given-names>
            <surname>Oltramari</surname>
          </string-name>
          .
          <article-title>The knowledge level in cognitive architectures: Current limitations and possible developments</article-title>
          .
          <source>Cognitive Systems Research</source>
          ,
          <volume>48</volume>
          :
          <fpage>39</fpage>
          -
          <lpage>55</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <surname>Antonio</surname>
            <given-names>Lieto</given-names>
          </string-name>
          , Daniele Paolo Radicioni, and
          <string-name>
            <given-names>Valentina</given-names>
            <surname>Rho</surname>
          </string-name>
          .
          <article-title>A common-sense conceptual categorization system integrating heterogeneous proxytypes and the dual process of reasoning</article-title>
          .
          <source>In Twenty-Fourth International Joint Conference on Artificial Intelligence</source>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <given-names>Fausto</given-names>
            <surname>Giunchiglia</surname>
          </string-name>
          and
          <string-name>
            <given-names>Mattia</given-names>
            <surname>Fumagalli</surname>
          </string-name>
          .
          <article-title>Concepts as (recognition) abilities</article-title>
          . In FOIS, pages
          <fpage>153</fpage>
          -
          <lpage>166</lpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <surname>Mattia</surname>
            <given-names>Fumagalli</given-names>
          </string-name>
          , Gabor Bella, and
          <string-name>
            <given-names>Fausto</given-names>
            <surname>Giunchiglia</surname>
          </string-name>
          .
          <article-title>Towards understanding classification and identification</article-title>
          .
          <source>In PRICAI</source>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40]
          <string-name>
            <surname>Ruth</surname>
            <given-names>Garrett</given-names>
          </string-name>
          <string-name>
            <surname>Millikan</surname>
          </string-name>
          .
          <source>On Clear and Confused Ideas: An Essay About Substance Concepts</source>
          . Cambridge University Press,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [41]
          <string-name>
            <surname>Stuart</surname>
            <given-names>J</given-names>
          </string-name>
          <string-name>
            <surname>Russell</surname>
            and
            <given-names>Peter</given-names>
          </string-name>
          <string-name>
            <surname>Norvig</surname>
          </string-name>
          .
          <article-title>Artificial intelligence: a modern approach</article-title>
          . Malaysia; Pearson Education Limited,,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          [42]
          <string-name>
            <given-names>Amina</given-names>
            <surname>Adadi</surname>
          </string-name>
          and
          <string-name>
            <given-names>Mohammed</given-names>
            <surname>Berrada</surname>
          </string-name>
          .
          <article-title>Peeking inside the black-box: A survey on explainable artificial intelligence (xai)</article-title>
          .
          <source>IEEE Access</source>
          ,
          <volume>6</volume>
          :
          <fpage>52138</fpage>
          -
          <lpage>52160</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          [43]
          <string-name>
            <given-names>Filip</given-names>
            <surname>Karlo</surname>
          </string-name>
          <string-name>
            <surname>Dosˇilovic</surname>
          </string-name>
          ´, Mario Brcˇic´, and Nikica Hlupic´.
          <article-title>Explainable artificial intelligence: A survey</article-title>
          .
          <source>In 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO)</source>
          , pages
          <fpage>0210</fpage>
          -
          <lpage>0215</lpage>
          . IEEE,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          [44]
          <string-name>
            <surname>Claudio</surname>
            <given-names>Masolo</given-names>
          </string-name>
          , Laure Vieu, Emanuele Bottazzi, Carola Catenacci, Roberta Ferrario, Aldo Gangemi, and
          <string-name>
            <given-names>Nicola</given-names>
            <surname>Guarino</surname>
          </string-name>
          .
          <article-title>Social roles and their descriptions</article-title>
          .
          <source>In 6th Int. Conf. on the Principles of Knowledge Representation and Reasoning (KR-2004)</source>
          , pages
          <fpage>267</fpage>
          -
          <lpage>277</lpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref45">
        <mixed-citation>
          [45]
          <string-name>
            <given-names>Tim</given-names>
            <surname>Miller</surname>
          </string-name>
          .
          <article-title>Explanation in artificial intelligence: Insights from the social sciences</article-title>
          .
          <source>Artificial Intelligence</source>
          ,
          <volume>267</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>38</lpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref46">
        <mixed-citation>
          [46]
          <string-name>
            <given-names>Rudi</given-names>
            <surname>Studer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V Richard</given-names>
            <surname>Benjamins</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Dieter</given-names>
            <surname>Fensel</surname>
          </string-name>
          .
          <article-title>Knowledge engineering: principles and methods</article-title>
          .
          <source>Data &amp; knowledge engineering</source>
          ,
          <volume>25</volume>
          (
          <issue>1-2</issue>
          ):
          <fpage>161</fpage>
          -
          <lpage>197</lpage>
          ,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref47">
        <mixed-citation>
          [47]
          <string-name>
            <given-names>Marcello</given-names>
            <surname>Frixione</surname>
          </string-name>
          and Antonio Lieto.
          <article-title>Representing concepts in artificial systems: a clash of requirements</article-title>
          .
          <source>Proc. HCP</source>
          , pages
          <fpage>75</fpage>
          -
          <lpage>82</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref48">
        <mixed-citation>
          [48]
          <string-name>
            <given-names>Aaron</given-names>
            <surname>Sloman</surname>
          </string-name>
          .
          <article-title>How can we reduce the gulf between artificial and natural intelligence? In AIC</article-title>
          , pages
          <fpage>1</fpage>
          -
          <lpage>13</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>