<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Process Theory⋆</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Gabriella Cortellessa</string-name>
          <email>gabriella.cortellessa@istc.cnr.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="editor">
          <string-name>Cognitive Robotics, Dual Process Theory, Artificial Intelligence</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Alessandro Umbrico</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>CNR - Institute of Cognitive Sciences and Technologies (CNR-ISTC)</institution>
          ,
          <addr-line>Rome</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2022</year>
      </pub-date>
      <abstract>
        <p>Autonomous robots acting in working and social contexts require the development of cognitive capabilities necessary to realize adaptive, contextualized, and safe behaviors. Artificial Intelligence (AI) technologies well-support the implementation of relevant capabilities e.g., decision making, knowledge representation, problem-solving, or learning. The synergetic integration of heterogeneous AI technologies is crucial to endow robots with a “mind” and combine together the functions of cognition necessary to realize efective behaviors. This work discusses recent results concerning the design of a novel control architecture inspired by the Dual Process Theory. The distinction between fast and slow reasoning processes guides the integration of AI modules that reason at diferent levels of abstraction and “time scales”. We show applications of the proposed concepts in healthcare assistance and collaborative manufacturing entailing continuous and adaptive interactions between a human and a robot.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Robotics and Artificial Intelligence (AI) are two research areas that historically addressed
the challenge (among others) of building embedded intelligent systems capable of acting in
a real-world environment [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Recent technological advancements in Robotics and AI are
pushing towards the design and deployment of autonomous robots in increasingly unstructured
environments and complex scenarios. A tight integration of Robotics and AI is crucial to allow
robots to safely and reliably act in the real-world [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ]. However, technology integration and
deployment of intelligent robots in real-world scenarios is still an open research challenge.
      </p>
      <p>
        Despite the increased reliability of developed technologies e.g., sensing, manipulation,
and navigation skills of robots on the one hand, and increased solving and predictive
capabilities of AI on the other, moving from structured in-laboratory environments to
semi”AAAI 2022 FALL SYMPOSIUM SERIES, Thinking Fast and Slow and Other Cognitive Theories in AI, November 17-19,
⋆You can use this document as the template for preparing your publication. We recommend using the latest version
of the ceurart style.
structured/unstructured real-world environments still poses non-negligible research challenges.
This is especially true when considering scenarios entailing the co-existence and/or continuous
direct/indirect interactions with human users. The presence of a human introduces a significant
source of uncertainty afecting robot control. The behavior of a human is uncontrollable as well
as his/her intentions, desires and objectives. Robot controllers cannot reliably predict actions
and behavior of humans and should therefore synthesize suitable strategies to carry out actions
safely. Furthermore, robot controllers should take into account a number of “non-functional”
qualities of the resulting behaviors when interacting with humans [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Indeed, doing the right
thing is not always suficient when robots act in “social contexts”. It is seamlessly important to
do the thing right and reason about how a robot should interact with humans [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ].
      </p>
      <p>Robot controllers should evolve towards an advanced “Perception, Reason, Act” paradigm to
achieve a higher level of awareness and contextualization. It is therefore necessary to endow
robots with a number of cognitive capabilities to implement behaviors that are valid from both
a technical and social point of view. In other words, robots need a mind to: (i) perceive and
build abstractions about the state of the environment; (ii) contextualize their skills; (iii) reason
about possible objectives; (iv) synthesize suitable actions and; (v) execute actions taking into
account the observed state of the environment and interacting humans.</p>
      <p>
        We propose an AI-based cognitive architecture which takes inspiration from the Dual Process
Theory [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. It organizes a number of AI modules into two reasoning layers (System 1 and
System 2) that cooperate to realize flexible and contextualized robot behaviors [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. These AI
modules implement the reasoning processes of the artificial mind of a robot. The architecture
is the result of research eforts concerning the development (and deployment) of AI-based
robot controllers in several Human-Robot Interaction (HRI) scenarios ranging from healthcare
and domestic assistance to collaborative manufacturing. The combination of a fast reasoning
layer and a slow reasoning layer supports a flexible composition of the underlying AI-modules
that realize the needed cognitive capabilities (e.g., decision making, knowledge representation,
problem solving, abstraction).
      </p>
      <p>This paper thus focuses on the integration of heterogeneous AI modules whose combination
supports the synthesis of flexible robot behaviors. Interestingly, the same architecture would
support diferent acting and interacting styles of a robot (reactive, proactive or deliberative),
depending on the diferent features and needs of a HRI scenario.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Towards a Mind for Autonomous Robots</title>
      <p>
        Artificial Intelligence, Cognitive Sciences, Neuroscience and Robotics have all contributed to the
understanding of minds by focusing at diferent levels of abstraction. While Cognitive Sciences
mostly focus on understanding cognitive processes and Neuroscience focus on the structure
and physiology of the brain (i.e., the physiological and physical correlates of mental processes),
Robotics and AI focus on understanding how minds work in order to provide software or
physical agents with intelligent behaviors. Although with diferent perspectives and levels of
abstraction, the common objective is to understand and simulate the functioning of a mind and
the related cognitive functions [
        <xref ref-type="bibr" rid="ref10 ref9">9, 10</xref>
        ].
      </p>
      <p>
        According to [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], a mind is a functional entity that can think and thus support intelligent
behaviors. The work [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] indeed represents a first efort aiming at developing a standard
model of the mind, taking into account a cognitive perspective. From a structural point of view
the proposed model is made of a number of independent modules “encapsulating” diferent
(cognitive) functionalities. This section briefly discusses some related works that have addressed
the problem of endowing robots with advanced cognitive capabilities. Although from diferent
perspectives, these works show the importance of integrating hybrid reasoning technologies to
realize efective, safe, and acceptable behaviors.
      </p>
      <sec id="sec-2-1">
        <title>2.1. Context Aware Robots</title>
        <p>
          To enhance the level of awareness, robots should be endowed with proper perception and
abstraction capabilities in order to understand observed situations. To this aim the integration
of ontology with AI and robot architectures seems promising. Researchers have investigated
diferent aspects of the reasoning and interaction capabilities of robots and used ontologies to
enhance them from diferent perspectives. The integration of semantic technologies with robot
controllers indeed has been widely studied in the literature [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ].
        </p>
        <p>
          Some works for examples have focused on providing robots with knowledge about the objects
of the environment in order to enhance manipulation skills or reason about possible use of
such object to perform complex tasks [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. KnowRob [
          <xref ref-type="bibr" rid="ref14 ref15">14, 15</xref>
          ] is a well-known framework
supporting advanced perception, reasoning and control. The framework provides robots with a
logical representation of a number of entities ranging from robotic parts and objects (with their
composition and functionalities) to tasks and actions. This framework focuses on manipulation
tasks and allows robots to perceive objects of the environment, reason about their afordances
[
          <xref ref-type="bibr" rid="ref16">16</xref>
          ], and decide how to use them by synthesizing a suitable sequence of (STRIPS/PDDL) actions
[
          <xref ref-type="bibr" rid="ref17">17</xref>
          ].
        </p>
        <p>
          An ontological model characterizing object manipulation tasks of robots has been also
considered within the PMK framework [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. Similar to KnowRob [
          <xref ref-type="bibr" rid="ref14 ref15">14, 15</xref>
          ], PMK supports a
“standardized” representation of the environment defining a “common language” to exchange
information between a human and a robot. It also models sensory capabilities to perceive
objects in the environment, linking perception outcomes to the ontological models of related
objects.
        </p>
        <p>
          Other works have focused on the “social dimension” of the interactions between humans
and robots to realize behaviors that are compliant with social norms. The ORO framework [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]
develops a knowledge reasoning framework endowing robots with common sense reasoning
capabilities to autonomously operate in semantically-rich human environments. ORO addresses
the control problem from a cognitive perspective and realizes a general cognitive architecture
deployed on diferent robotic platforms and assessed on diferent cognitive scenarios [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. This
architecture has been specifically developed to support advanced cognitive skills (e.g., theory of
mind capabilities) and supports flexible and adaptive human-robot interactions [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ].
        </p>
        <p>
          The work [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ] uses knowledge reasoning to represent social norms and allow a social robot
to implement acceptable behaviors for social tasks. More specifically, the work proposes a
formal description of the functional afordances of objects to reason about their possible use
and thus infer those that are “socially acceptable” to accomplish the requested social task (i.e.,
serving cofee to guests using the right object). The work [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] proposes the use of knowledge
reasoning to adapt human-robot interactions to the cultural knowledge of diferent contexts
and people. This is another example of how ontology-based reasoning can enhance awareness
of robots by implementing suitable cognitive capabilities. In this case, such capabilities are used
to reason about non-functional qualitative aspects of human-robot interactions and synthesize
socially-compliant and acceptable behaviors.
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Cognitive Design of Robot Controllers</title>
        <p>
          Several researchers have investigated the development of cognitive architectures based on
a functional model of the human mind. Following the review [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ], a common agreement
among AI researchers sees cognitive architectures classified in symbolic, connectionist, or hybrid
ones. Symbolic architectures use production rules and represent concepts using symbols that
can be manipulated using a predefined instruction set. Although they excel at planning and
reasoning, these architectures do not suficiently support robustness which is necessary to deal
with a changing environment and perceptual processing. Connectionist architectures address
adaptability and learning aspects by building parallel models that are organized in networks.
Although efective, the resulting system loses transparency, since knowledge is no longer a set
of symbolic entities and is distributed throughout the network. Hybrid architectures attempt to
combine elements of both symbolic and connectionist approaches, by including both symbolic
and sub-symbolic components with the aim to match human cognition.
        </p>
        <p>
          Within the hybrid architectures category some research eforts have been inspired by the dual
process theory and combined symbolic and sub-symbolic components in the attempt to simulate
the System 1 and the System 2. The work [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ] endows a social robot with a computational
explanation module based on two components: a System 1 component (S1) responsible for the
fast categorization and for the perceptual based recognition of gestures in a social context, based
on deep neural network architecture; a System 2 component (S2) responsible for providing a
high level model that can be exploited to extract an explanation about the high level features
that characterize the categorized output provided by S1 exploiting an ontology.
        </p>
        <p>
          Diferently, [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ] developed a model able to handle both symbolic and sub-symbolic reasoning,
by means of an architecture based on two memory systems: (i) a long-term memory, an
autonomous system that develops automatically through interactions with the environment,
and (ii) a working memory, a memory system used to define the notion of (resource-bounded)
computation. The long-term memory is modeled as a transparent neural network that develops
autonomously by interacting with the environment, while the working memory is modeled as a
bufer containing nodes of the long-term memory. The Clarion architecture instead is a hybrid
cognitive architecture with both connectionist and symbolic representations, that combines
implicit and explicit psychological processes, and integrates cognition (in the narrow sense) and
other psychological processes. Overall, Clarion is a modular cognitive architecture consisting of
a number of distinct subsystems, with a dual representational structure in each subsystem [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ].
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Integrated AI for Fast and Slow Reasoning</title>
      <p>
        A key point in the design of cognitive architectures is the management of diferent source
of knowledge and the basic capabilities needed to access and process such knowledge. The
work by [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] systematizes a number of cognitive capabilities that are relevant to an autonomous
system: (i) recognition and categorization; (ii) reasoning and belief maintenance; (iii) prediction
and monitoring; (iv) problem solving and planning; (v) decision making and choices; (vi) execution
and action.
      </p>
      <p>
        Robots need a suitable integration of these capabilities to efectively act in HRI scenarios.
Well established AI technologies like Machine learning, knowledge representation and reasoning,
automated planning and execution can play an important role in this context. We here propose a
hybrid cognitive architecture which integrates heterogeneous AI modules following the Dual
Process Theory. Figure 1 shows the structure of this architecture, initially introduced in [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ].
      </p>
      <p>The architecture is articulated into a fast and slow reasoning layers emulating the functioning
of System 1 and System 2 of the Dual Process Theory. Both layers consist of a pipeline of
AI modules implementing reasoning processes at two diferent levels of abstraction. Broadly
speaking, the System 1 layer supports fast reasoning capabilities that are directly linked to the
perception and acting skills of a robot. This layer deals with a high level of uncertainty generally
given by a non-perfect knowledge of the environment and unpredictable (and uncontrollable)
behaviors of humans. Implemented cognitive processes therefore reason on a short time horizon
in order to realize (fast) reactive behaviors.</p>
      <p>The System 2 layer supports slow reasoning capabilities that are directly linked to domain
knowledge and the objectives of a scenario. This layer relies on semantic abstractions and
models of the dynamics (either physical or social) of a HRI domain. It generally pursues an
optimization perspective aiming at synthesizing interaction strategies (or plans) to achieve
complex goals. Goals can be “manually” set by domain experts or opportunistically inferred by
System 2 taking into account observed situations.</p>
      <p>The reasoning processes implemented by System 1 and System 2 work in parallel and both
contribute to the synthesis of robot behaviors. Depending on the domain and observed
situations one can see a particular System “dominating” the other leading to behaviors with diferent
features. In general, a robot follows a deliberative behavior when the System 2 predominates
on System 1. Vice versa, a robot would follow a reactive behavior when the System 1
predominates on System 2. This structure supports a flexible synthesis of robot behaviors that can be
dynamically adapted to the evolving state of the environment.</p>
      <sec id="sec-3-1">
        <title>3.1. Deliberative Behavior</title>
        <p>The System 2 is composed by AI modules mainly relying on symbolic technologies ranging
from knowledge representation &amp; reasoning and automated planning. The integration of these
technologies according to the pipeline of AI modules in Figure 1 (System 2) implements
deliberative reasoning suitable to contextualize robot acting skills and synthesize plans that achieve
complex goals in the “long term”. The Semantic Module encapsulates domain knowledge and
contextualize information, events and situations collected by data streams processed by the
System 1. This module thus builds and continuously refines an abstraction of the scenario
integrating knowledge about the social context, robot capabilities, features and needs of human
users and operational requirements. The Opportunistic Module implements goal reasoning
capabilities to dynamically evaluate afordances and opportunities of action that may lead to
the achievement of new or additional goals. This module enriches knowledge reasoning by
integrating contextual knowledge about the (sub)set of goals that can be actually achieved by
the robot in a known scenario.</p>
        <p>The reasoning processes carried out in conjunction by the Semantic Module and the
Opportunistic Module support personalization and contextualization of robot behaviors. From a
HRI perspective, they allow a robot to autonomously reason about who is the target of the
interactions (e.g., human user), which objectives are suitable in a given scenario and how such
objectives should be achieved by the robot, taking into account social, human and technical
perspectives. Inferred (and selected) goals are then passed to the Strategic Module which is in
charge of deciding which actions are necessary and when they should be executed. This module
relies on automated planning technologies in order to optimize strategies/plans according to a
number of metrics (either general or domain specific). The reasoning processes of this layer are
slow since they process symbolic information and generally span over a long time horizon.</p>
        <p>The outcome of the reasoning processes implemented at System 2 level are passed to the
System 1 in shape of plans characterizing desired robot behaviors from a high abstraction
level. The System 1 is then in charge of physically implementing this behavior by synthesizing
planned actions in physical operations/interactions performed by the robot in the real-world.
System 2 thus provides System 1 with a general description of the behaviors that should be
implemented to achieve complex (domain relevant) objectives.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Reactive Behavior</title>
        <p>The System 1 is composed by AI modules mainly relying on sub-symbolic technologies ranging
from machine learning, natural language processing, computer vision and other control-related
technologies e.g., motion/path planning or autonomous navigation. The integration of these
technologies according to the pipeline of AI modules in Figure 1 (System 1) implements reactive
reasoning suitable to allow a robot to perceive the environment, receive input and physical
interact with objects, humans and other domain entities. These modules support the interacting
skills of the robot controller. They should therefore realize fast reasoning processes to quickly
adapt physical behaviors of robots to exogenous events and unknown/unexpected states of
the environment. These capabilities are crucial to safely interact with humans and carry out
long-term deliberated plans in a reliable way. Examples are reasoning and data processing
capabilities necessary to avoid obstacles during robot navigation or avoid collisions with humans
during the execution of robot motions.</p>
        <p>The Perception Module elaborate inputs from users and streams of data gathered from sensing
devices to produce clean and useful information. Depending on the specific needs of a HRI
scenario, this module would thus encapsulates data fusion techniques to process data from
multiple input channels e.g., robot camera, input voice or text, environmental sensors etc. Produced
information would be used by System 2 for knowledge abstraction and by the Learning Module
to infer patterns and similar situations. Namely, this module would thus enrich perception
capabilities of the robot with an associative network suitable to incrementally build “experience”
and autonomously recognize recurrent events or situations.</p>
        <p>Learned experience is useful at both System 2 and System 1 level. System 2 would consider
learned patterns in the deliberative process and thus synthesize in advance plans that are reliable
with respect to possible (known) situations. System 1 would consider such patterns to enhance
the adaptation of robot behaviors and thus generate more accurate polices leading to more
eficient and robust behaviors. These policies are indeed used by the Acting Module which
is in charge of dealing with the physical implementation and actual execution of the actions
composing the plans synthesized by System 2. The reasoning processes implemented at System
1 level thus decide the best way of execution actions according to the observed state of the
environment (included the observed behavior of human users).</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Enhancing Human-Robot Interactions</title>
      <p>
        The architecture of Figure 1 is the result of research eforts concerning the development (and
deployment) of AI-based robot controllers in HRI scenarios ranging from healthcare [
        <xref ref-type="bibr" rid="ref27">27, 28, 29</xref>
        ]
to manufacturing [30, 31, 32]. We made a first attempt of integrating the developed reasoning
capabilities within a “Dual Process” inspired architecture with the work [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ].
      </p>
      <p>We first investigated cognitive control for reconfigurable manufacturing systems to endow
a number of transportation modules with a level of self-awareness suitable to autonomously
adapt their capabilities to the evolving state of a production context [32]. Although not in
HRI, this work gave us the possibility of investigating the integration of perception, knowledge
reasoning and planning for the synthesis of flexible robot behaviors in a multi-agent setting
[33]. Perception data from transportation modules were interpreted and contextualized into a
knowledge base following a formal ontological model of agents’ capabilities. Each transportation
module (i.e., agent) was thus capable of autonomously infer the set of transportation capabilities
supported according to its configuration and local topology (i.e., local connections with other
transportation modules). This knowledge was automatically synthesized in a planning model
to support dynamic reconfiguration of each agent and reliable functioning of the whole system.</p>
      <p>
        We have then further investigated this first result evolving towards a cognitive approach for
the synthesis of flexible behaviors. In collaborative manufacturing, we have investigated the
integration of knowledge reasoning, perception, task and motion planning to realize optimal,
safe and human-aware collaborative processes [34, 31, 30]. In healthcare assistance, we have
investigated the integration of knowledge reasoning, planning, learning and natural language
processing to realize personalized, proactive and adaptive assistance to human users [
        <xref ref-type="bibr" rid="ref27">27, 35, 29</xref>
        ].
These results all rely on the integration of heterogeneous AI modules based on a two-layered
structured as the one proposed in Figure 1. Next subsections discuss with more details the
diferent features of the robot behaviors implemented in the cited works and show how the
proposed architecture (and the underlying AI modules) realizes the needed cognitive capabilities.
      </p>
      <sec id="sec-4-1">
        <title>4.1. Human-Awareness and Personalization</title>
        <p>Human-aware and personalization are two qualities crucial for the efective interactions between
the human and the robot. Cognitive processes supporting these qualities mainly work at System
2 level of the designed architecture. The Semantic Module of Figure 1 should be enriched
with domain knowledge suitable to characterize the relevant features of humans, interaction
capabilities of robots and the way they afect these features.</p>
        <p>Depending on the application scenario we have developed ontological models suitable to
completely characterize relevant knowledge. In manufacturing for example we have integrated
a domain ontology into the Semantic Module. The ontology characterizes production dynamics
of collaborative scenarios and working skills of robots and human works [36]. The integration
of this knowledge with the Strategic Module and underlying planning technologies allows a
robot to reason about: (i) production operations necessary for accomplishing collaborative tasks;
(ii) the set of operations the human and the robot could perform according to their skills; (iii)
most suitable allocation of tasks/operations taking into account expected qualities concerning
the execution of assigned operations (e.g., average execution time, time variance etc.). Resulting
collaborative plans are then executed through the Acting Module to physically control robot
motions and coordinate them according to the observed behavior of the human. The integration
of these two modules thus allows a robot to adapt the synthesis of collaborative processes
according to the known skills and qualities of their human working fellow [34].</p>
        <p>In healthcare assistance we have developed a Semantic Module integrating a domain ontology
of cognitive impairments of persons in order to personalize cognitive stimulation therapy
[35, 28]. The ontology encapsulates a formal description of the ICF classification 1 and allows a
robot to characterize the functioning of human users and automatically infer impairments.</p>
        <p>The Opportunistic Module then matches health impairments of users with stimulation
capabilities of robots (e.g., known cognitive stimulation exercises) to generate recommendations about
the (sub)set of exercises that best fit the stimulation needs of a user. This module in particular
introduces the concept of afordances to reason about opportunities of stimulation resulting by
matching assistive capabilities of robots and health needs of users [28]. The Strategy Module then
further elaborates this knowledge and generated recommendations to synthesize personalized
interventions. Intervention plans consist of stimulation exercises that are administrated to a
user through the Acting Module of System 1 layer. The administration of such exercises indeed
requires fast adaptation capabilities in order to deal with user feedback [29].</p>
        <p>As shown in [35], these reasoning capabilities in particular allow a robot to support decision
making of clinicians. On the one hand, the developed Semantic Module allows a robot to
1https://www.who.int/standards/classifications/international-classification-of-functioning-disability-and-health
understand user knowledge obtained through standard screening procedures like the
MiniMental State Examination (MMSE) and build suitable user profiles . On the other hand, the
resulting reasoning mechanisms allow a robot to efectively support therapists in making
decisions about how to “shape” interventions.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Proactivity and Adaptation</title>
        <p>The integration the Perception Module, Learning Module and Semantic Module within System 1
and System 2 layers allows a robot to continuously “monitor” the state of the environment and
(autonomously) evaluate the opportunity of performing actions and thus achieve goals. This
integration supports qualities like proactivity and adaptation that are crucial to reliably and
efectively act in real-world scenarios.</p>
        <p>
          In the context of domestic assistance to seniors through social robots for example we have
developed a Perception Module to integrate data streams gathered from environmental sensing
devices [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ]. Obtained information (observations) are then collected, interpreted and
contextualized by the Semantic Module which maintains a semantic module of the house and state of
both the user and the robot. The Opportunistic Module further reasons about this knowledge
in order to identify situations or conditions requiring the proactive execution of some actions
or assistive routine by the robot. Examples are situations concerning the health state of the
user (e.g., anomalous heart rate) that would trigger warnings and procedures to assist the user
through the robot. This triggers would thus generate goals that are give to the Strategic Module
which generates suitable assistive plans executed through the Acting Module.
        </p>
        <p>Considering again a scenario of cognitive stimulation, the reactive capabilities of System 1
are crucial to adapt robot behaviors to the evolving state of human supporting efective (and
engaging) interactions [29]. Given a personalized stimulation plan generated by the System 2
indeed the integration of Perception Module, Learning Module and Acting Module at System 1
level is crucial to efectively carry out the execution of such a plan, dealing with the evolving
state of a human user. The administration of cognitive exercises and the adaptation of the way
a robot interact and stimulate a person would be dynamically adapted to the feedback and to
the perceived state of the user (e.g., mood, personality traits, etc.). This level of adaptation is
crucial to achieve engaging and efective interactions between the human and the robot.</p>
      </sec>
      <sec id="sec-4-3">
        <title>4.3. Cooperation and Optimization</title>
        <p>Discussed HRI scenario see the robot and the human interacting together while playing two
distinct and diferent roles. In scenarios like collaborative manufacturing instead the human
and the robot can be seen as two peer agents working together to achieve a common objective.
In this context the proposed architecture supports a flexible and adaptive coordination between
the human and the robot [31, 30]. The distinction between the two reasoning levels well
support the design of advanced task and motion planning capabilities suitable to reason about
working skills of the two working fellows, optimize collaborative process and safely execute
planned actions. Reasoning processes at System 2 level mainly deal with domain knowledge
concerning production procedures and known skills of the human and the robot. According
to this knowledge the integrated Semantic Module and Strategic Module optimize the resulting
collaborative process. The Strategic Module in particular reasons about possible assignments of
tasks/operations to the human and the robot following a multi-objective optimization of the
whole process taking into account both cycle time and risk of collision.</p>
        <p>The Acting Module then in combination with the Perception Module executes planned actions
taking into account feedback and the observed behavior of the worker. In particular, the Acting
Module implements safe procedures to control robot motions avoiding collisions with the human.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions</title>
      <p>This work proposes an AI-based cognitive architecture inspired by the Dual Process Theory. The
architecture is the result of research eforts concerning the development (and deployment) of
AIbased robot controllers in diferent HRI scenarios. The combination of a fast reasoning layer and
a slow reasoning layer supports a flexible composition of the underlying AI-modules that realize
the needed cognitive capabilities. The proposed architecture systematizes the integration of AI
modules developed to support flexible and adaptive interactions and collaborations between
humans and robots. We show how the proposed composition supports both deliberative and
reactive behaviors of the robot that can be dynamically adapted and “interleaved” according to
the specific application needs. Future works will further investigate the design and integration
of AI modules leading to a modular robotic architecture evaluated on diferent robotic platforms
and working scenarios.
617–637.
[28] A. Umbrico, G. Cortellessa, A. Orlandini, A. Cesta, Modeling Afordances and Functioning
for Personalized Robotic Assistance, in: Principles of Knowledge Representation and
Reasoning: Proceedings of the Sixteenth International Conference, AAAI Press, 2020.
[29] R. De Benedictis, A. Umbrico, F. Fracasso, G. Cortellessa, A. Orlandini, A. Cesta, A
two-layered approach to adaptive dialogues for robotic assistance, in: 2020 29th IEEE
International Conference on Robot and Human Interactive Communication (RO-MAN),
2020, pp. 82–89.
[30] S. Pellegrinelli, A. Orlandini, N. Pedrocchi, A. Umbrico, T. Tollio, Motion planning nad
scheduling for human and industrial-robot collaboration, CIRP Annals - Manufacturing
Technology. To appear. (2017).
[31] M. Faroni, M. Beschi, S. Ghidini, N. Pedrocchi, A. Umbrico, A. Orlandini, A. Cesta, A
layered control approach to human-aware task and motion planning for human-robot
collaboration, in: 2020 29th IEEE International Conference on Robot and Human Interactive
Communication (RO-MAN), 2020, pp. 1204–1210.
[32] S. Borgo, A. Cesta, A. Orlandini, A. Umbrico, Knowledge-based Adaptive Agents for</p>
      <p>Manufacturing Domains, Engineering with Computers 35 (2019) 755–779.
[33] E. Carpanzano, A. Cesta, A. Orlandini, R. Rasconi, M. Suriano, A. Umbrico, A. Valente,
Design and implementation of a distributed part-routing algorithm for reconfigurable
transportation systems, Int. J. Computer Integrated Manufacturing 29 (2016) 1317–1334.
[34] A. Umbrico, A. Orlandini, A. Cesta, M. Faroni, M. Beschi, N. Pedrocchi, A. Scala,
P. Tavormina, S. Koukas, A. Zalonis, N. Fourtakas, P. S. Kotsaris, D. Andronas, S. Makris,
Design of advanced human-robot collaborative cells for personalized human-robot
collaborations, Applied Sciences 12 (2022).
[35] A. Sorrentino, L. Fiorini, G. Mancioppi, F. Cavallo, A. Umbrico, A. Cesta, A. Orlandini,
Personalizing care through robotic assistance and clinical supervision, Frontiers in Robotics
and AI 9 (2022).
[36] A. Umbrico, A. Orlandini, A. Cesta, An ontology for human-robot collaboration, Procedia
CIRP 93 (2020) 1097 – 1102.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>K.</given-names>
            <surname>Rajan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Safiotti</surname>
          </string-name>
          ,
          <article-title>Towards a science of integrated ai and robotics</article-title>
          ,
          <source>Artificial Intelligence</source>
          <volume>247</volume>
          (
          <year>2017</year>
          )
          <fpage>1</fpage>
          -
          <lpage>9</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>F.</given-names>
            <surname>Ingrand</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ghallab</surname>
          </string-name>
          ,
          <article-title>Deliberation for autonomous robots: A survey</article-title>
          ,
          <source>Artificial Intelligence</source>
          <volume>247</volume>
          (
          <year>2017</year>
          )
          <fpage>10</fpage>
          -
          <lpage>44</lpage>
          . Special Issue on AI and Robotics.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>M.</given-names>
            <surname>Ghallab</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Nau</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Traverso</surname>
          </string-name>
          ,
          <article-title>The actor's view of automated planning and acting: A position paper</article-title>
          ,
          <source>Artificial Intelligence</source>
          <volume>208</volume>
          (
          <year>2014</year>
          )
          <fpage>1</fpage>
          -
          <lpage>17</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>S.</given-names>
            <surname>Rossi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Ferland</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. Tapus,</surname>
          </string-name>
          <article-title>User profiling and behavioral adaptation for hri: A survey</article-title>
          ,
          <source>Pattern Recognition Letters</source>
          <volume>99</volume>
          (
          <year>2017</year>
          )
          <fpage>3</fpage>
          -
          <lpage>12</lpage>
          .
          <article-title>User Profiling and Behavior Adaptation for Human-Robot Interaction</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>K.</given-names>
            <surname>Dautenhahn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Walters</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Woods</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. L.</given-names>
            <surname>Koay</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. L.</given-names>
            <surname>Nehaniv</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Sisbot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Alami</surname>
          </string-name>
          , T. Siméon,
          <article-title>How may i serve you? a robot companion approaching a seated person in a helping context</article-title>
          ,
          <source>in: Proceedings of the 1st ACM SIGCHI/SIGART Conference on HumanRobot Interaction</source>
          , HRI '06,
          <string-name>
            <surname>Association</surname>
          </string-name>
          for Computing Machinery, New York, NY, USA,
          <year>2006</year>
          , p.
          <fpage>172</fpage>
          -
          <lpage>179</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>B.</given-names>
            <surname>Bruno</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. T.</given-names>
            <surname>Recchiuto</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Papadopoulos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Safiotti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Koulouglioti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Menicatti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Mastrogiovanni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Zaccaria</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Sgorbissa</surname>
          </string-name>
          ,
          <article-title>Knowledge representation for culturally competent personal robots: Requirements, design principles, implementation, and assessment</article-title>
          ,
          <source>International Journal of Social Robotics</source>
          <volume>11</volume>
          (
          <year>2019</year>
          )
          <fpage>515</fpage>
          -
          <lpage>538</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>P.</given-names>
            <surname>Wason</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Evans</surname>
          </string-name>
          ,
          <source>Dual processes in reasoning?, Cognition</source>
          <volume>3</volume>
          (
          <year>1974</year>
          )
          <fpage>141</fpage>
          -
          <lpage>154</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>D.</given-names>
            <surname>Kahneman</surname>
          </string-name>
          , Thinking, fast and slow, Macmillan,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>P.</given-names>
            <surname>Langley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. E.</given-names>
            <surname>Laird</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Rogers</surname>
          </string-name>
          ,
          <article-title>Cognitive architectures: Research issues and challenges</article-title>
          ,
          <source>Cognitive Systems Research</source>
          <volume>10</volume>
          (
          <year>2009</year>
          )
          <fpage>141</fpage>
          -
          <lpage>160</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>A.</given-names>
            <surname>Lieto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Bhatt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Oltramari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Vernon</surname>
          </string-name>
          ,
          <article-title>The role of cognitive architectures in general artificial intelligence</article-title>
          ,
          <source>Cognitive Systems Research</source>
          <volume>48</volume>
          (
          <year>2018</year>
          )
          <fpage>1</fpage>
          -
          <lpage>3</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>J. E.</given-names>
            <surname>Laird</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Lebiere</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. S.</given-names>
            <surname>Rosenbloom</surname>
          </string-name>
          ,
          <article-title>A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics</article-title>
          ,
          <source>AI</source>
          Magazine
          <volume>38</volume>
          (
          <year>2017</year>
          )
          <fpage>13</fpage>
          -
          <lpage>26</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>S.</given-names>
            <surname>Manzoor</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y. G.</given-names>
            <surname>Rocha</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.-H.</given-names>
            <surname>Joo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.-H.</given-names>
            <surname>Bae</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.-J.</given-names>
            <surname>Kim</surname>
          </string-name>
          ,
          <string-name>
            <surname>K.-J. Joo</surname>
          </string-name>
          , T.-Y. Kuc,
          <article-title>Ontology-based knowledge representation in robotic systems: A survey oriented toward applications</article-title>
          ,
          <source>Applied Sciences</source>
          <volume>11</volume>
          (
          <year>2021</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>D.</given-names>
            <surname>Beßler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Pomarlan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Beetz</surname>
          </string-name>
          ,
          <article-title>Owl-enabled assembly planning for robotic agents</article-title>
          ,
          <source>in: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems</source>
          , AAMAS '18,
          <string-name>
            <surname>International</surname>
            <given-names>Foundation</given-names>
          </string-name>
          <source>for Autonomous Agents and Multiagent Systems</source>
          , Richland,
          <string-name>
            <surname>SC</surname>
          </string-name>
          ,
          <year>2018</year>
          , p.
          <fpage>1684</fpage>
          -
          <lpage>1692</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M.</given-names>
            <surname>Tenorth</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Beetz</surname>
          </string-name>
          ,
          <article-title>Representations for robot knowledge in the KnowRob framework</article-title>
          ,
          <source>Artificial Intelligence</source>
          <volume>247</volume>
          (
          <year>2017</year>
          )
          <fpage>151</fpage>
          -
          <lpage>169</lpage>
          . Special Issue on AI and Robotics.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>M.</given-names>
            <surname>Beetz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Beßler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Haidu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Pomarlan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. K.</given-names>
            <surname>Bozcuoğlu</surname>
          </string-name>
          , G. Bartels,
          <source>Know rob 2</source>
          .0
          <article-title>- a 2nd generation knowledge processing framework for cognition-enabled robotic agents</article-title>
          ,
          <source>in: 2018 IEEE International Conference on Robotics and Automation (ICRA)</source>
          ,
          <year>2018</year>
          , pp.
          <fpage>512</fpage>
          -
          <lpage>519</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>D.</given-names>
            <surname>Beßler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Porzel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Pomarlan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Beetz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Malaka</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Bateman</surname>
          </string-name>
          ,
          <article-title>A formal model of afordances for flexible robotic task execution</article-title>
          ,
          <source>in: ECAI</source>
          <year>2020</year>
          , IOS Press,
          <year>2020</year>
          , pp.
          <fpage>2425</fpage>
          -
          <lpage>2432</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>D.</given-names>
            <surname>Mcdermott</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ghallab</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Howe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Knoblock</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ram</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Veloso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Weld</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Wilkins</surname>
          </string-name>
          ,
          <string-name>
            <surname>PDDL - The Planning Domain Definition Language</surname>
          </string-name>
          ,
          <source>Technical Report, Yale Center for Computational Vision and Control</source>
          ,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>M.</given-names>
            <surname>Diab</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Akbari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. Ud</given-names>
            <surname>Din</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Rosell</surname>
          </string-name>
          ,
          <article-title>Pmk-a knowledge processing framework for autonomous robotics perception and manipulation</article-title>
          ,
          <source>Sensors</source>
          <volume>19</volume>
          (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lemaignan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Ros</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Mosenlechner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Alami</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          <article-title>Beetz, ORO, a knowledge management platform for cognitive architectures in robotics</article-title>
          ,
          <source>in: Intelligent Robots and Systems (IROS)</source>
          ,
          <year>2010</year>
          IEEE/RSJ International Conference on,
          <year>2010</year>
          , pp.
          <fpage>3548</fpage>
          -
          <lpage>3553</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lemaignan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Warnier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. A.</given-names>
            <surname>Sisbot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Clodic</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Alami</surname>
          </string-name>
          ,
          <article-title>Artificial cognition for social human-robot interaction: An implementation</article-title>
          ,
          <source>Artificial Intelligence</source>
          <volume>247</volume>
          (
          <year>2017</year>
          )
          <fpage>45</fpage>
          -
          <lpage>69</lpage>
          . Special Issue on AI and Robotics.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>I.</given-names>
            <surname>Awaad</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. K.</given-names>
            <surname>Kraetzschmar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hertzberg</surname>
          </string-name>
          ,
          <article-title>The Role of Functional Afordances in Socializing Robots</article-title>
          ,
          <source>International Journal of Social Robotics</source>
          <volume>7</volume>
          (
          <year>2015</year>
          )
          <fpage>421</fpage>
          -
          <lpage>438</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>I.</given-names>
            <surname>Kotseruba</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. K.</given-names>
            <surname>Tsotsos</surname>
          </string-name>
          ,
          <article-title>40 years of cognitive architectures: core cognitive abilities and practical applications</article-title>
          ,
          <source>Artificial Intelligence Review</source>
          <volume>53</volume>
          (
          <year>2018</year>
          )
          <fpage>17</fpage>
          -
          <lpage>94</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>A.</given-names>
            <surname>Augello</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Infantino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Lieto</surname>
          </string-name>
          , U. Maniscalco,
          <string-name>
            <given-names>G.</given-names>
            <surname>Pilato</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Vella</surname>
          </string-name>
          ,
          <article-title>Towards A Dual Process Approach to Computational Explanation inHuman-Robot Social Interaction</article-title>
          ,
          <source>in: Proceedings of the 1st CAID workshop at IJCAI</source>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>C.</given-names>
            <surname>Strannegård</surname>
          </string-name>
          , R. von Haugwitz,
          <string-name>
            <given-names>J.</given-names>
            <surname>Wessberg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Balkenius</surname>
          </string-name>
          ,
          <source>A Cognitive Architecture Based on Dual Process Theory, in: Artificial General Intelligence</source>
          , Springer Berlin Heidelberg,
          <year>2013</year>
          , pp.
          <fpage>140</fpage>
          -
          <lpage>149</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>R.</given-names>
            <surname>Sun</surname>
          </string-name>
          ,
          <source>The CLARION Cognitive Architecture</source>
          , Oxford University Press,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>A.</given-names>
            <surname>Umbrico</surname>
          </string-name>
          , R. De Benedictis,
          <string-name>
            <given-names>F.</given-names>
            <surname>Fracasso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Cesta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Orlandini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Cortellessa</surname>
          </string-name>
          ,
          <article-title>A mind-inspired architecture for adaptive hri</article-title>
          ,
          <source>International Journal of Social Robotics</source>
          (
          <year>2022</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>A.</given-names>
            <surname>Umbrico</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Cesta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Cortellessa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Orlandini</surname>
          </string-name>
          ,
          <article-title>A holistic approach to behavior adaptation for socially assistive robots</article-title>
          ,
          <source>International Journal of Social Robotics</source>
          <volume>12</volume>
          (
          <year>2020</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>