Ambient Assisted Living aims at developing systems that assist people in their daily tasks. In this context, assistance and care are delegated to the intelligence embedded in the environment that, in our opinion, should provide not only a task-oriented support but also an interface able to establish a social empathic relation with the user. This is what we call a “caring home”. In this paper we report our experience in designing a caring home environment that combines C@sa, a multi-agent system for managing the smart home behavior, and NICA, a conversational interface embodied in a social robot able to recognize the user attitude and to provide the appropriate social response.
One of the research goals in the context of Ambient Assisted Living ( AAL) concerns
the integration of new technologies with the social environment to support people in
their daily activities and increase their quality of life [
Natural Interaction of the user with the information and services offered by the system. Such an interface has two fundamental and interconnected objectives: being a means to interact with the environment and being, for the user, a friendly caring agent. For this reason it is important to understand not only the meaning of the communication but also the conveyed emotions and the user’s attitude during the interaction. This requires the emotional analysis of the linguistic and prosodic aspects of the user's vocal input, of her facial expressions and gestures.
In this paper we propose the integration of a social empathic agent, acting as a
virtual caregiver, in C@sa, an agent based architecture for handling the smart behavior
of the home environment [
The paper is structured as follows: in Section 2 we briefly describe the C@sa
multiagent architecture, and in Section 3 we illustrate the main features of NICA. Then, in
Section 4 we provide an example on how NICA can serve as a caring agent of C@sa;
finally we conclude the paper with discussion and directions for future work.
2
A smart environment should observe lifestyle and desires of its inhabitants to learn
how to anticipate and accommodate their needs by using Machine Learning
techniques [24]. The environment, then, must be able to reason on the situation of the
user so as to understand his/her needs and goals and satisfy them through the
composition of the most appropriate services. To enforce this view we have developed
C@sa [
As its main tasks, the butler must know the habits of the house inhabitants,
perceive the situation of the house, and coordinate the housestaff. To this aim we have
designed the following classes of agents:
- Sensor Agents (SA): are used for providing information about context parameters
and features (e.g., temperature, light level, humidity, etc.) at a higher abstraction
level than raw sensor data.
- Butler Agent (BA): reasons on the user’s goals and devises the workflow to
satisfy them [
All agents are endowed with two main behaviors, reasoning and learning (see Figure 2). Although all agents share the same architecture, they differ in level of complexity, techniques that can be exploited by the reasoning functionality, tools that implement the techniques, and theories used for reasoning. Of course, different agents work on different portions of knowledge on the domain and may require different Fig. 2. The architecture underlying the C@sa agents effort and pose different problems.
Reasoning uses the agent’s knowledge to perform inferences that determine how the agent achieves its objectives. Learning exploits possible feedback on the agent’s decisions to improve that knowledge, making the agent adaptive to the specific user needs and to their evolution in time. The learning behavior returns new knowledge gained from experience, that extends or refines the model on which the reasoning behavior is based. The reasoning behavior mixes mathematical and statistical processing techniques with more powerful kinds of reasoning and knowledge management based on First-Order Logic, in order to handle the complexity of real-world environments where relationships among several entities and possible situations play a significant role. The input to an agent is processed by its reasoning layer, for: - deciding which signals are to be ignored and which ones are to be sent to other entities that can understand and exploit them (e.g. agents or user or devices, depen ding on the kind of agent) and/or to its learning functionality. - processing and combining input data to detect significant patterns and produce more complex information, using different kinds of inference techniques. - deciding which part of this information is to be ignored and which part is to be forwarded to other entities (see above) and/or to the learning functionality.
The learning behavior, on the other hand, is used by an agent to refine and improve its future performance. For the specific needs of adaptation posed by the present application, an incremental approach to learning new information is mandatory, because the continuous availability of new data and the evolving environment require continuous adaptation and refinement of the available knowledge. An incremental ILP system that is able to exploit different kinds of inference strategies (induction, deduction, abduction, abstraction), and hence fits the above requirements, is InTheLEx [25]; also abstraction and abduction theories can be learned automatically [26]. The main inference strategy that characterizes the learning layer of our agents is induction, although a cooperation with other strategies, such as those exploited in the reasoning behavior, is strongly advised, for a better integration of the new knowledge with the reasoning engine. 3 NICA: a Socially Intelligent Caring Agent In order to show how the considerations outlined in the Introduction can be successfully employed in designing and implementing a social and empathic virtual caregiver [27, 28], we have developed a caring agent named NICA (Natural Interaction with a Caring Agent). In order to provide assistance and, at the same time, to settle a social long-term relation with the user, NICA, starting from the multimodal analysis of the user behavior (see [29] for more details on the recognition framework) provides proactively and reactively the needed assistance by acting as a social conversational interface between the user and the home services. To this aim, NICA has to: i) start from the interpretation of the user multimodal input; ii) reason on the information the user intends to convey (emotion, social attitude, performative, content, etc.) and then trigger communicative goals according to the current belief representation of the state of the world; iii) achieve these goals through a set of communicative plans (“what to say”) that can then be rendered as a combination of voice and animations of the agent’s body (“how to say”); iv) keep in its social memory information about which are the antecedents of emotions for the user, that is what triggers the emotions (events, situations, thoughts, etc.).
In order to handle dialogs in a dynamic environment, NICA has been modeled as a BDI (Belief, Desire, Intention) agent and interleaves reactive and deliberative behaviors [30].
- Interpretation: evaluates changes in the world and user’s state that are relevant to the agent’s reasoning and transforms them into a set of agent’s beliefs. In particular it interprets the user’s input.
- Goal Activation: conversational goals are triggered based on the current beliefs. - Planning and Execution: once a goal has been triggered it is achieved through the execution of a communicative plan appropriate to the situation.
In order to adapt the robot’s behavior to the user’s needs and preferences, NICA’s mental state reasons on and stores different types of knowledge:
- the World Model that represents a set of relevant beliefs about the current environment context.
- the User Model that contains the representation of beliefs of various type. In particular, we model long-term factors concerning stable user data (i.e. sex, age, chronic diseases, allergies, main personality traits, interests, etc.) and short-term factors concerning belief about the current user situation, affective state, etc. - the Conversational Resources to be used to handle the dialog.
- the Agent Social Memory stores structured information about feelings associated with events. It is used to remember relations about events and the user affective state. The importance of this piece of knowledge in the agent’s mind is related to the need of establishing empathy with the elderly person and this was outlined several times by the human caregivers during the data collection phase.
As the agent reasons and updates its beliefs, infers goals, plans conversational behaviors and executes them, it keeps an image of this process in its mental state. In order to deal with the uncertainty typical of this domain (e.g. dealing with exceptional situations or with the smooth evolution of the user’s affective state over time), we employ probabilistic models to reason on the user and to decide which behavior to adopt, that is the most appropriate set of actions to perform for satisfying the inferred user’s goal. At the moment, as shown in Figure 4, we simulate the interaction between NICA, the user and the environment in a toy house equipped with a robot (Lego MindStorm with sensors for detecting its position in the house), light, temperature and presence sensors, and a microphone for capturing the user’s voice.
Moreover, we developed an interface for setting some parameters concerning the
world state and some other user data that at the moment we are not capturing in real
time (i.e. physical parameters such as fever, blood pressure, etc.). We do not consider
this as a strong limitation of our approach since many wearable and wireless devices
are coming out on the market and therefore in a real setting we will be able to receive
these data. However, we are aware that in real settings the appearance of Lego
Mindstorm could provoke a negative effect in the user, but before testing the system
in real settings we wanted to be sure that the agent’s mind was reasoning in a reliable
and consistent way and therefore we employed the toy house scenario. Our research
group used the AIBO robot or the conversational agent Valentina [31] in other
projects and they might be used also in this case in future experiment with elderly
users. Moreover, of particular interest to our research is the approach adopted in the
Florence project (http://www.florence-project.eu/). In Florence the Social Robot was
built using low-cost parts that have been assembled in order to allow movements in
the house, communication with the SHE architecture and a friendly conversational
interface. From the implementation point of view, changing the embodiment of our
agent is not a problem since we adopt the mind-body architecture developed in a
previous project [31]. Therefore the plan computed by the “mind” module contains
the meaning to express and the “body” has to convey this meaning according to its
communicative capabilities. For space reasons, in this paper we will focus on the
description of the affective component of the user model. For more details about
NICA, dialog and affect modeling see [
The user’s move is a rich information source that allows extracting knowledge about her intention, social attitude, emotional state, and so on. In our approach, the user’s model maintained by the agent allows to reason on the user’s beliefs (i.e. the user’s move “I love fruit!” will be transformed into the corresponding belief that can be used to adapt the dialog strategy) and on the user’s attitude during the dialog. Beliefs on knowledge, preferences and interests of the user are inferred according to an approach previously employed in another system [31] and to the one about the user’s affective state that is recognized and monitored with a dynamic model based on Belief Network (DBN) [32]. In fact, when modeling affective phenomena we have to take into account the fact that affective states smoothly evolve during the interaction, from one step to the subsequent one. As a consequence, the affective state should be monitored and modeled as a temporal phenomenon, whose value at every time of the interaction depends on the value it assumes in the previous dialogue turn. For this reason, the DBN formalism is particularly suitable for representing situations that gradually evolve from a dialog step to the next one.
Figure 5 shows an example of DBN in which low-level beliefs, deriving from per ceptions, can be used to infer beliefs about the affective state of the user. In particular, this network can be used to infer the probability that the user is in a negative, neutral or positive affective state or the probability that the user feels a particular emotion. In this model we consider only sadness, anxiety, anger and happiness that are relevant for the purpose of the system since they were reported in the diary of the human care givers.
This dynamic model allows us to take into account the influence of the user’s state at the previous step. For instance, in the DBN in Figure 5 this is expressed by a temporal link between the Bel(AffectiveState)Prev variable and the Bel(AffectiveState). Analogously, the evidences and the probability values of the root nodes of the BN may be extracted from other modules. This allows to manage the complexity of the network and to integrate in the model evidence deriving from different modules performing different and independent analysis.
For example, the Voice node in Figure 5 is evaluated according to the results of the acoustic analysis of the user’s utterances. Research in emotional speech has shown that acoustic and prosodic features can be extracted from the speech signal and used to develop models for recognizing emotions and attitudes [33] In fact, the effects of emotion in speech tend to alter the pitch, timing, voice quality, and articulation of the speech signal and reliable acoustic features can be extracted from speech that vary with the speaker's affective state. We used Praat functions in order to perform a ma cro-prosodic or global analysis and to extract from the audio file of each move features related to the variation of the fundamental frequency (f0), energy (RM), harmoni city, central spectral moment and speech rate.
The sensors responsible for capturing the speech classify the user’s affective state and attitude towards the system, adopting an approach analogous to the one described in [34]. At present, our classifier exploits the Nearest Neighbor with generalization (NNge) algorithm and recognizes the valence with an accuracy of 85%, evaluated on a dataset of 4 speakers and 748 user moves overall. The accuracy of the classifer has been validated with a 10-fold Cross Validation technique.
The aim of the model described in Figure 5 is twofold: on one hand the model is employed to guess which specific emotional state the user is experiencing at every step of the interaction; on the other hand it is used to monitor the overall evolution of the user’s affective state (i.e. the agent’s belief about the positive or negative affective state of the user). In particular, every time a new user move is entered, its linguistic and acoustic features are analyzed with respect to the context variable and the resulting evidence is introduced and propagated in the network to recognize the user’s emotion and the overall polarity of her affective state. The new probabilities of individual emotions are read and contribute to formulate the next move of the agent; the probability of the dynamic variable (Bel(AffectiveState)), representing the valence of user’s affective state, is used by the agent to check the consistency between its persistent goal of maintaining the user in a positive affective state and the actual emotional state the user is in at time t, thus causing the activation or the revision of high-level planning of the agent’s behavior. Then, the values of the node Vocal Input derive from a module that integrates the linguistic content of the user move with the recognition of its acoustic features to recognize the actual user’s communicative goal [34].
At present we concentrated on spoken interaction and we are not working at the recognition of facial expressions, gestures, postures, and so on. For this reason the values of the Physical Situation and Emotional Action nodes of the network are simula ted using an interface for setting parameters for running a scenario simulation like the one that will be described later on.
Starting from what has been inferred by the user model component, the dialog management module computes the agent’s move using a strategy based on the information state approach [35]. It represents and stores beliefs about the current state of the world, the user, the dialog, the dialog history, the current dialog move and the move scheduled for execution.
The dialog manager, and in particular the deliberative module, decides which goals to trigger and to pursue during the dialog, starting from the interpretation of the user’s move and the recognized affective state. In order to handle interruptions, variation of the user’s situation (for instance the recognized emotion), the agent has a Reactive Layer. The idea is that the agent has an initial list of goals, each with its priority, some of which are inactive: every goal is linked, by an application condition, to a plan that the agent can perform to achieve it. The communicative actions corresponding to active plans are put in the agenda maintained by the information state. The agent starts the dialog by executing these actions but, as we said in the Introduction, the agent ap plies some form reasoning on the user’s move. The recognized social attitude and the emotion triggered in the agent’s mind are used to implement social and emotion-based dynamic revision of goals and consequently of the dialog. To achieve the selected communicative goal we use plans represented as context-adapted recipes.
With these rules, we formalize a situation of empathic reaction in which the agent
temporarily substitutes the presumed goals of the user for its own, when these goals
are due to an emotional state of the user. If an undesirable event occurs to the users,
what they are presumed to need is to be convinced that the agent understands the
situation and does its best to solve the problem. If something desirable occurs to them,
they need to know that the agent shares their positive experience. If, on the contrary,
the undesirable event does not concern the users, they probably want to be sure that
this will not interfere negatively with the dialog.
Let us now provide an example of how the proposed systems can be integrated in
order to support a “caring home” for assisted living. Consider the following scenario,
depicted from the analysis of data from the diaries that we collected with human care
givers and elderly people [
It’s morning and Maria, a 73 y.o. woman, is at home alone. She has a cold and fe ver. She is a bit sad since she cannot go to the market downtown and talk with her friends, like she does every morning. Maria is sitting on the bench in her living room in front of the TV. The living room is equipped with sensors, which can catch sound/noise in the air, time, temperature, status of the window (open/close) and of the radio and TV (on/off), and the current activity of the user, and with effectors, acting and controlling windows, radio and TV and also the execution of digital services that may be visualized on communication devices. According to the situation, the BA infers possible user’s goals and triggers the appropriate workflow whose tasks are executed by effectors agents. When a communicative or an interaction task is required, NICA, that is close to the user, acts as an IA. While the selected workflow is executed, NICA has to check Maria’s health state and recommend some medicine to her. After a while Maria starts whispering and moaning and says: “Oh My…oh my…”. This utterance is perceived by NICA that interprets it and activates the most appropriate emphatic behavior.
Let’s see how this scenario is simulated in our system.
The physical sensors send in real-time the values they gather to the reasoning behavior of the associated SA, which uses abstraction to strip off details that are useless for the specific current tasks and objectives. For instance, the SA providing information about temperature will abstract the centigrade value into a higher level represen tation such as “warm”, “cold”, and so on. This abstraction process may be done according to the observed specific user’s needs and preferences (e.g. the same temperature might be cold for a user but acceptable for another). For instance, let us denote the fact that the user Y is cold in a given situation X with cold(X,Y). This fact can be derived from the specific temperature using a rule of the form: cold(X,Y) :- temperature(X,T), T<18, user(Y), present(X,Y), maria(Y). (it is cold for user Maria if she is present in a situation in which the temperature is lo wer than 18 degrees). In turn, the above rule can be directly provided by an expert (or by the user herself), or can be learned (and possibly later refined) directly from observation of user interaction [36]. For instance, assume that the following events have been recorded in the past: temperature 28 16 8 20 32 18 37 26 22 19 29 23 12 25 4 action
C H H - C H C C - - C - H - H where the first row reports a set of temperatures sensed in situations where Maria was present, and the second row reports her action in those situations (C = cooling, H = heating, - = no action). Then, the SA controlling the temperature may automatically learn that the user turns on heating (i.e., she is cold) whenever the temperature is be low 19 degrees, and turns on the cooling system (presumably because she is warm) whenever the temperature is above 25 degrees: cold(X,Y) :- temperature(X,T), T<19, user(Y), present(X,Y), maria(Y). warm(X,Y) :- temperature(X,T), T>25, user(Y), present(X,Y), maria(Y). Starting from percepts received by SAs, the BA infers user goals and selects a workflow that integrates elementary services suitable for the particular situation. Situations can be formally described as conjunctive logic formulas under the Closed World Assumption (what is not explicitly stated is assumed to be false). A model consists of a set of Horn clauses whose heads describe the target concepts and whose bodies describe the pre-conditions for those targets to be detected. A very simple model might be: improveHealth(X) :- present(X,Y), user(Y), has_fever(Y). improveHealth(X):- present(X,Y), user(Y), has_headache(Y), cold(X,Y). improveHealth(X) :- present(X,Y), user(Y), has_flu(Y). improveMind(X) :- present(X,Y), user(Y), sad(Y). improveMind(X) :- present(X,Y), user(Y), bored(Y). meaning “A user Y that is present in situation X and has a fever, or has a headache and has a cold, or has a flu, might want to improve his health” and “A user Y that is present in situation X and is sad or bored might want to improve his mind”, respectively. A sample observation might be:
morning(t0),closedWindow(t0),present(t0,m),maria(m),user(m), temperature(t0,14), has_flu(m), sad(m). (i.e., “in situation at time t0 it is morning, the window is closed and the temperature is 14°; user Maria is present and she has flu”). Reasoning infers that Maria is cold: cold(t0,m). Being all the preconditions of the third and fourth rules in the model satisfied by this situation for X = t0 and Y = m, the user goals improveHealth and improveMind are recognized for Maria at time t0, which may cause activation of suitable workflows aimed at attaining those results. Conversely, the other rules in the model are not satisfied – e.g., considering the last rule, user Maria is present, but she is assumed not to be bored. Although predicates such as fever(X), headache(X) and flu(X) are already abstractions of the specific value provided by SAs, further levels of generalization can be automatically performed by the reasoning layer, e.g. using a predicate has_disease(Y), defined as has_disease(X) :- has_fever(X). has_disease(X) :- has_flu(X).
such that the first and third rule in the model can be reduced to: improveHealth(X) :- present(X,Y), user(Y), has_disease(Y).
making it applicable to other kinds of diseases, in addition to just fever and flu. Re ferring back to the previous observation, the reasoning behavior would infer that Maria has a disease – has_disease(m) – from the fact that she has a flu.
The BA reasons not only on goals but also on workflows. Indeed, once a goal is triggered, it selects the appropriate workflow by performing a semantic matchmaking between the semantic IOPE (Input Output Preconditions Effects) description of the user's high-level goal and the semantic profiles of all the workflows available in the knowledge base of the system [37]. As a result, this process will return a (possibly empty) set of workflows that are semantically consistent with the goal (possibly ranked by a function of semantic similarity with the goal). For instance, in Figure 6 the semantic matchmaking process leads to two different workflows associated, respectively, to the two high-level goals improveHealth and improveMind previously recognized. The semantic matchmaking process starts from these goals and leads to the desired workflow.
Fig 6. An example of a Smart Service Workflow composed by the Butler Agent Complex workflows may involve both simple actions and other sub-goals corresponding to subflows, which are again processed according to the matchmaking phase described above. In our case, the main workflow includes two goals that need to be executed by selecting two different subflows corresponding, respectively, to improveHealth and improveMind. These subflows include both simple actions, that can be directly executed, and subflows that need to be satisfied, such as setTemperature. In turn, the subgoal setTemperature is satisfied by applying once more the matching process to find a suitable workflow. Using this workflow, the reasoning behavior of the BA will process the information collected by the temperature sensors in order to understand whether to raise or reduce the environment temperature: doReduceTemperature(X) :- present(X,Y), user(Y), warm(X,Y). doRaiseTemperature(X) :- present(X,Y), user(Y), cold(X,Y).
This hierarchical matchmaking process stops when the resulting workflow is composed of goals that can be directly satisfied by invoking a net-centric service or throu gh simple actions performed on the effectors. In both cases, the BA asks to the HA which EAs can satisfy each planned action and sends the specific request to the EA in charge for handling actions regarding changes of a particular parameter (i.e. temperature, light, etc.). In particular, when the goal regards a communicative action, its execution is delegated to the IAs, and the HA returns to the BA the list of agents that are responsible for implementing the interaction with the user through different modalities (e.g. on a touch screen, on the smartphone or by using the social robot).
In order to find the best solution to satisfy the user needs, EAs reason about differ ent possible solutions to attain the same goal in the current context. For instance, if the goal is reducing the temperature, the EA in charge of temperature control may decide whether turning on air conditioning or opening the window; additionally, it decides how to control those devices (in the former case, which fan speed to select; in the latter case, how widely the window must be opened).
When the environment has to perform a communicative task it delegates it to the IA, that in our case is instantiated in NICA. There are several communicative goals that this agent may carry out. In Information Seeking the IA exploits interaction with the user to get hints on how to attain a simple goal and, based on this, possibly learns new preferences of the user with respect to the given context and situation, in order to continuously and dynamically improve adaptation. Information Providing is used to answer and provide explanations about the SHE appliances behavior or about the decision to include some specific subflows in the main workflow built by the BA. For example, referring to the previous scenario, the user may ask the robot to provide more information to justify the choice for a given medicine. Through Remind NICA may remind to the user to perfom an action or about an appointment, and so on. Social and Affective Support is used for communications.
In the following we refer to the previously described scenario and we propose an example of interaction. In particular the subflow called ‘Medicine Recommender’ included in the workflow in Figure 6. It may be satisfied differently according to the specific situation of the user. Let’s suppose, for example, that the information about the user’s disease is not complete, that is the BA knows that the user needs to improve her health because she has got a flu but it is necessary to decide which drug is the most suitable for her to take. In such a situation, it is necessary to further investigate the user’s physical state in order to select the most appropriate drug to suggest to him.
In the following we provide an example of interaction with the social intelligent robot present in the house.
NICA: ‘Hi Maria, how are you doing?’ Maria: ‘I have got flu, I think I need some medicine’ NICA: ‘Let’s check the fever’.
NICA: ‘You have 37°C. I would suggest you to take a pill of paracetamol’ Maria: ‘Are you sure? I don’t feel like having fever, I rather have some headache and sore throat’ NICA: ‘Well, paracetamol is ok for headache too. But, if you prefer, you can take a pill of anti-inflammatory, which could solve both problems’ … after a while Maria: Oh My …oh my… NICA: Oh I’m sorry to hear that you are anxious. Why do you feel so? Maria: I’m worried for my health.
NICA: Don’t worry, you have just a flu, you will soon feel good. If you won’t feel better by tomorrow I will call the doctor.
The last utterance by Maria is recognized as having the meaning expressing an affective state and its prosody identifies a negative state and, in particular, the recognized emotion is sadness. NICA does not know what caused sadness in Maria. The initial probability values of the nodes Voice and Vocal Input are derived automatically by the speech recognition module of NICA, while the setting of the other values of the root nodes in the DBN is done through the framework interface. This evidence is propagated in the DBN and the belief about the affective state of the user has a negative valence with a high probability (66,89) (see Figure 5). This belief triggers “encoura ge” as the most convenient goal to pursue in this situation, since it is the one with the highest probability. Then, the most appropriate plan is selected according to its precondition and the execution of its communicative actions begins. The plan includes the following actions, since NICA does not know the reason for that affective state and it will ask the user about it:
MoveTo(N,MARIA) Express(N,Sorry-for(N,MARIA)) Ask(N,MARIA,Why(MARIA,Anxious))
Express(N,MARIA,Encurage(N,MARIA))
We presented a preliminary work towards the integration of a MAS aimed at handling the situation-aware adaptation of a Smart Environment behavior and NICA, a Social Robot for handling interaction in the SHE. In the MAS different types of agents cooperate to the adaptation process: Sensor Agents, a Butler Agent, a Housekeeper, Effector and Interactor Agents. The main peculiarity of the proposed architecture lies in the fact that all kinds of agents in the MAS are a specialization of an abstract class endowed with both reasoning and learning behavior. In particular, interaction with the user is implemented using NICA, a conversational social interface to the SHE ser vices. In our opinion, besides assisting the elderly user in performing tasks, the agent has to establish a social long-term relationship with the user in order to enforce trust and confidence. The underlying idea of our work, in fact, is that endowing the agent with a socially intelligent behavior is fundamental when the devices of a smart home are integrated pervasively in everyday life environments. As an example, we illustrated how NICA, a social agent acting as a caring assistant for elderly people living in a smart home, has been developed. At present we evaluated the agent’s behavior using a toy house, and hence we could not evaluate the effectiveness of the communication with real elderly users, which requires a smart home environment and a suitable social robot like for instance the Nao [40]. This kind of experiment should aim at assessing the impact of the use of a social robot on seamless interaction with the environment services. Another important issue to be addressed in our future work concerns the interpretation of multimodal human communicative actions in order to recognize the user’s attitude. In [29] we started working on the recognition of the user’s social attitude from a combination of spoken and gestural communication using a probabilistic approach that allows accommodating for the uncertainty typical of this domain. Moreover we are developing a model for giving NICA the capability to infer the user’s activity in the house [38].
This work fulfils the research objectives of the PON02_00563_3489339 project "PUGLIA@SERVICE - Internet-based Service Engineering enabling Smart Territory structural development" funded by the Italian Ministry of University and Research (MIUR). 22. Broekens J., Heerink M., Rosendal H.: Assistive social robots in elderly care: a review.
Gerontechnology, 8(2):94-103, 2009. 23. Reeves B. and Nass C.: The media equation: how people treat computers, television, and new media like real people and places, Cambridge University Press, 1996. 24. Muggleton S.H.: Inductive Logic Programming. New Generation Computing, 8(4):295-318, 1991. 25. Esposito F., Fanizzi N., Ferilli S., Basile T.M.A. and Di Mauro N.: Multistrategy Operators for Relational Learning and Their Cooperation. Fundamenta Informaticae Journal, 69(4):389-409, IOS Press, 2006. 26. Ferilli S., Esposito F., Basile T.M.A. and Di Mauro N.: Automatic Induction of Abduction and Abstraction Theories from Observations. In: S. Kramer, B. Pfahringer. Inductive Logic Programming, 103-120, Springer, 2005. 27. Cearreta I., López J.M., López De Ipiña K., Hernandez M., Garay N., Graña M. and Álvarez A.: Affective Computing as a Component of Ambient Intelligence. INFORMATION SCIENCES 2007, Proceedings of the 10th Joint Conference, Salt Lake City, Utah, USA, 18 - 24 July 2007, 1580-1586, 2007. 28. Leite I., Martinho C. and Paiva A.: Social Robots for Long-Term Interaction: A Survey.
International Journal of Social Robotics, 1-18, January, 2013 29. De Carolis B., Ferilli S. and Novielli N.: Recognizing the User Social Attitude in
Multimodal Interaction in Smart Environments. AmI 2012: 240-255, 2012. 30. Castelfranchi C. and Paglieri F., The role of beliefs in goal dynamics: Prolegomena to a constructive theory of intentions, Synthese 155, 237-263, 2007. 31. De Carolis B., Mazzotta I. and Novielli N.: Enhancing Conversational Access to Information through a Socially Intelligent Agent. In G. Armano, M. de Gemmis, G. Semeraro and E. Vargiu (Eds): Intelligent Information Access, Studies in Computational Intelligence 301, 1-20, Springer, 2010. 32. Jensen F.V.: Bayesian Networks and Decision Graphs, Statistics for Engineering and
Information Science, Springer, 2001. 33. Sundberg J., Patel S., Björkner E. and Scherer K.R.: Interdependencies among voice source parameters in emotional speech. IEEE Transactions on Affective Computing, 2(3), 2011. 34. De Carolis B. and Cozzolongo G.: Interpretation of user’s feedback in human-robot interaction, Journal of Physical Agents, 3(2):47-58, 2009. 35. Larsson S. and Traum D.R.: Information state and dialogue management in the TRINDI dialogue move engine toolkit. Nat. Lang. Eng. 6, 3-4, 323-340, 2000. 36. Ferilli S., Basile T.M.A., Di Mauro N., Esposito F.: On the LearnAbility of Abstraction Theories from Observations for Relational Learning. In: J. Gama, R. Camacho, G. Gerig, P.
Brazdil, A. Jorge, L. Torgo. Machine Learning: ECML 2005, 120-132, Springer, 2005. 37. Meyer H.: On the Semantics of Service Compositions. Lecture Notes in Computer Science 4524, 31-42, 2007. 38. Ferilli S., De Carolis B. and Redavid D.: Logic-based Incremental Process Mining in Smart Environments. In M. Ali, T. Bosse, K.V. Hindriks, M. Hoogendoorn, C.M. Jonker, J. Treur (Eds.), Recent Trends in Artificial Intelligence (IEA-AIE 2013), Lecture Notes in Artificial Intelligence 7906, 392-401, Springer, 2013. 39. ksera.ieis.tue.nl/ 40. http://www.aldebaran-robotics.com/ (last visited Oct 3, 2012).