In this paper we present an Ambient Intelligent System, the iGenda, and the integration of a wearable device. The aim is to detect emotional states through the wearable device and ultimately represent and manage the social emotion of a group of entities. The advantage of this action is that its usability is in line with retirement homes and similar places, where the community is extended and an harmonious environment is imperative. The iGenda serves has the visual interface and the information centre, receiving the information from the wearable device and managing the community emotion by sending information to the care-receivers, caregivers, or changing home parameters (like music or lighting) to achieve an specific emotion (such as calm or excitement). Thus the goal is to provide an affective system that directly interacts with humans by discreetly improving their lifestyle.
Ambient Assisted Living (AAL) is an booming area where a large number of
projects are being developed with the aim of providing assistance to elderly and
disabled people [
These changes are usually associated to certain emotional states [
A solution would be the adoption of an adaptive system that is able to perceive these emotional changes and adjust to them. Furthermore, with sufficient information, emotional profiles could be created in order to mimic the user common response and pre-emptively use this information to respond to decisions.
They way that humans perceive the world influences their emotional state and that has a repercussion on the physical level. While most can hide facial expressions and body movements (like hiding a state of surprise) there are lowlevel signals that the human body sends inadvertently like skin/muscle tensioning, pupil dilatation and micro-movements. These most of these signals have a corresponding bio-electrical impulse, thus they can be captured by sensors.
We propose the usage of a wearable device in form of a wristband, named Emotional Smart Wristband (ESW) that feed information to the iGenda in order to schedule new tasks according the emotional status.
The paper is structure as follows: section 2 presents the ESW architecture, the emotional model and the virtual actor concept; section 3 presents the conclusion and future developments. 2
The ESW monitors the Galvanic Skin Response and the Photoplethysmogram,
processes that information locally and sends that information to other systems
like the iGenda [
For instance, it is well known that there is typically one or a few more
leaders in elderly communities [
The ESW platform is composed of agents that calculate the group emotion (or
Social Emotion) of the participants, through the Social Emotion Agent (SEtA)
and resorting to machine learning techniques and makes it available for
consumption [
For instance, consider a bar, it contains a group of friends and has music playing. Now consider that some of these friends have different musical tastes. If a music is playing that is enjoyable to 90% of the group we are able to notice a emotional progression towards "satisfied", while if it is the opposite (90% dislikes) we can observe movement in the opposite direction. One of the possible aim of this analysis is obtaining a playlist that is enjoyable to most of the group with a specific alignment of songs that cater to each individual like. This specificity is used to avoid reaching a tension point where one or more person has reached a saturation point and refuses any further change or abandons the environment.
The task of the iGenda is to consume this information a use its scheduling
feature to change the events of each participant, based on his/her profile, to
achieve a specific emotion, thus guiding the group to a common emotional state.
The emotional states are measured and placed in the PAD [
The model seen in Figure 1 is a granular visual representation of the PAD model
where the Valence replaces the Pleasure as in the psychological area is more
representative and has a larger range of values [
We use a fuzzy logic process that transforms quantitative values to qualitative values that are able to be placed in the model showed in Figure 1. Thereon, a neural network is trained using the DEAPdataset 3 as to achieve a real-world result from real humans. The correlation between the data captured and the dataset is established in the neural network meaning that each processed electrical input can be translated into an emotion. 2.3
A new development is the inclusion of virtual actors in the system that emulate
the real participants. This feature aims to enhance of the decision-making process
of the system, detecting in advance the possible emotional states and
preparing changes to the participants surrounding environment to accommodate these
3 http://www.eecs.qmul.ac.uk/mmv/datasets/deap/
changes and proactively shift them to other states if the expected state is not
desirable [
In this case specifically, when related to the elderly there is little margin
for experimentation, as they have a large number of conditionings and are very
fragile. The emotional profile helps to preview the possible reactions to changes
on their daily routines and their daily emotion when performing specific
activities. The group emotion is very useful in environments like nursing homes and
residential elderly communities. In this type of environment is very common to
perform group activities and generate teams of elderly people. One common
issue is the user verbal response and the real feeling, e.g., a person may tell that
they are enjoying an activity but that be only a "kind" response due to peer
pressure or to social engagement with the caregivers [
Another benefit from using virtual actors is the possibility of introducing Immersive Virtual Environments (IVE). In this specific case, the IVE’s would be used to project the environments that the users reside and simulate all of the possible components, meaning that every sensor/actuator is mapped as an agent and has agency, thus it is able to directly interact with the human agents. This forwards our research by having an safe environment were there is the ability of testing multiple outcomes of real interactions in fractions of seconds and project the optimal actions to achieve a certain outcome.
The main goal of this project is to create cohesive environments in which the users are happy and feel accomplished. This is an arduous task as people is composed of individuals that most of the time are very different to the rest. Furthermore, there are several different applications to the ESW ecosystem apart from the healthcare area, like: crowd control, visual interfaces management, stress and fatigue monitoring, virtual environments societies, among others. 4
This work is partially supported by the MINECO/FEDER TIN2015-65515-C41-R and the FPI grant AP2013-01276 awarded to Jaime-Andres Rincon. This work is supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT Fundação para a Ciência e Tecnología within the projects UID/CEC/00319/2013 and Post-Doc scholarship SFRH/BPD/102696/2014 (A. Costa)