Workshop Overview

In the last few years, a growing interest has been seen in the development of autonomous interactive robots to enhance our daily lives in our homes and at work. Applications are plentiful: companions for children or elderly people, partners in industries, guides in public or personal spaces, educational tutors at school and so on. Robots able to face such broad range of social situations require social-cognitive capabilities that promote fluid and effective interactions. These include the automatic understanding of the users actions, behaviors, and mental and emotional states and the coherent production of multimodal, verbal and non-verbal communication skills.

Designing and developing multimodal communication skills for robots in order to provide more natural and powerful interactive experiences is a significant challenge in practice due to limitations both in technology and in our understanding of how different modalities must work together to convey human-like levels of social intelligence. The area of social interaction and multimodal expression for socially intelligent robots remains very much an active research area with many challenges and open research questions. For example, to what degree, and how precisely, different modalities (e.g., eye gaze, touch, vocal, body, and facial expressions) might be involved in human interaction with intelligent robots remains largely unknown. Likewise, whether combining multimodal elements of emotional expressions result in enhanced recognition of the emotion is still an open question. Furthermore, current research is predominantly focused on visual and auditory senses, neglecting other modalities such as touch that are an integral part of how we experience the real world around us.

The scope of this workshop is to present rigorous scientific advances on social interaction and multimodal expression for socially intelligent robots. Previous research findings show that this challenge cannot be solely approached from a pure engineering perspective. Human sciences, social sciences, and cognitive sciences play a primary role in the development and the enhancement of social interaction skills for socially intelligent robots. This workshop fosters interdisciplinary collaboration between researchers on the domain, addressing both the study of human-human interactions as well as human-machine interactions. The analysis of human-human interactions offers researchers the opportunity to understand how humans interact with the world and with one another multimodally, through both parallel and sequential use of multiple modalities (e.g., eye gaze, touch, vocal, body, and facial expressions), and to develop guidelines on how to design robot behaviors. On the other hand, results achieved by researchers studying human-robot interactions are particularly important to understand how the multimodal communication skills developed for social robots are perceived by uninformed interaction partners (e.g., children, elderly) and how they influence the interaction process (e.g., regarding usability and acceptance).

Submission Categories

In order to enable a diverse program, the workshop invites a variety of submission categories in the call for papers. Topics included, but where not limited to:

(a) Contributions of fundamental nature -Psychophysical studies and empirical research about multimodality (b) Technical contributions on multimodal interaction -Novel strategies of multimodal human-robot interactions -Dialogue management using multimodal output -Work focusing on novel modalities (e.g., touch) (c) Multimodal interaction evaluation -Evaluation and benchmarking of multimodal human-robot interactions -Empirical HRI studies with (partial) functional systems -Methodologies for the recording, annotation, and analysis of multimodal interactions (d) Applications for multimodal interaction with social robots -Novel application domains for multimodal interaction (e) Position papers and reviews of the state-of-the-art and ongoing research

Program Overview

The workshop consists of three invited talks and four talks based on paper submissions covering topics such as verbal and non-verbal interaction with robots, design of affective expressions and applications for multimodal interaction with social robots. The talks were followed by a brainstorming discussion. Table 1 presents an outline of the program.

Invited Speakers

Ana Paiva -Instituto Superior Tcnico, Technical University of Lisbon Short Bio: Ana Paiva is an Associate Professor in the Department of Computer Science and Engineering (Departamento de Engenharia Informtica) of Instituto Superior Tcnico from the Technical University of Lisbon (Universidade Tcnica de Lisboa). She is also the group leader of GAIPS (Grupo de Agentes Inteligentes e Personagens Sintticas), a research group on agents and synthetic characters at INESC-ID. Her main scientific interests lay in the area of Autonomous Agents, Embodied Conversational Agents and Robots and Multiagent Simulation Systems. Prof. Ana Paiva has been researching in the area of artificial intelligence for the past twenty years, having also taught at IST during that period.

Talk: Robots that listen to the users heart: The role of emotions in multimodal communication with social robots Andrea Bonarini -Department of Electronics, Information, and Bioengineering, Politecnico di Milano Short Bio: Andrea Bonarini is full professor and coordinator of the AI and Robotics Lab at Politecnico di Milano. His current research interests are on machine learning and emotional human-robot interaction, with special focus on robotic games and toys for entertainment and social, psychological and physical development of people with and without special needs. Since 1989, he has realized with his collaborators and students more than 60 autonomous robots. In 2015, he co-funded Nova Labs, to share experience and tools to produce professional robots in a short time at low cost.

Talk: Speechless Social Robotic Toys: Emotional Relations Without Words Abstract: More and more sophisticated toys that can perceive signals from the world and physically act in consequence, i.e. robotic toys, are entering the market and are initiating a new way of playing: they cannot any longer treated as props (as traditional plushes and dolls are), but they are agents that can decide their actions, to interact with. This arrangement opens challenges concerning believability and appropriateness of the robots behavior, which cannot really exploit speech because of both technological and privacy reasons. Multimodal interaction is relevant for this application, and should keep into account many dimensions, from psychology, to technology, and cost. Talk: BUM -Bayesian User Model for Distributed Social Robots Abstract: In this work we present a Bayesian User Model for inferring the characteristics and inter-personal relationships of a population users. The model can receive evidence gathered by various interactive devices, such as social robots or wearable devices. The system is modular, with each module being responsible for gathering information and observations from persons present in the systems operation scenario. This information enables each module to determine a single characteristic of the person. New observations and measurements received by the system are fused with previous knowledge by a sub-process based on an information theory technique. This allows the system to be implemented in diverse heterogeneous distributed system topologies, extending beyond robotics. We have conducted experiments involving a simulated team of social robots and user population with four sets of person types. Our experiments have shown that the system is able to learn and classify the persons characteristics, and to find relevant user groups via clustering. This system can potentially be used to gather information on a large set of persons, as well as to be an information source for user-adaptive applications in areas such as Robotics, AAL and Internet of Things. Astrid Weiss is a postdoctoral research fellow in HRI at the Vision4Robotics group at the ACIN Institute of Automation and Control at Vienna University of Technology (Austria). She holds a masters degree in sociology and a PhD in social sciences from the University of Salzburg. Her current research focuses on user-centered design and evaluation studies for Human-Robot Interaction, with a special interest in the impact technology has on the everyday life and what makes people accept or reject technology. She co-organized workshops on a variety of HRI-related topics at the following conferences: RO-MAN2008, HRI2009, HRI2011, ICSR2013, and HRI2014. Moreover, she is regularly member of Program and Organizing Committees related to HRI research.

Sten Hanke has a Master in electrical engineering and a PhD in medical science. He is a researcher for the Austrian Institue of Technology (AIT) biomedical engineering in smart biomedical systems. He joined AIT in the year 2005. He is working on European and national projects in the field of ambient assisted living, social robotics, interaction with ICT systems, smart systems as well as psycho-physiological assessments since more than 10 years. At the moment is he technical manager of the FP7 Miraculous-Life project10 dealing with development of a virtual support partner as well as workpackage leader of the H2020 GrowMeUp project11 developing an ICT service robot for ambient assisted living environments. He is a working member of the international ISO/IEEE 11073, European ISO/TC 215 and CEN/TC 251 health care and medical device communication working group as well as the national ON-AG238.4 group. Currently he is in the Ambient Assisted Living Open Association (AALOA) Governing Board.

Julian Angel-Fernandez is a postdoctoral research at the Visrion4Robotics at the Institute of Automation and Control (ACIN) at University of Technology (Austria). His interests twofold in representation and selection of emotions in robotics, and study of methodologies to increase the interest of students in STEM careers. Currently, he is working as scientific coordinator in the European project ER4STEM, which aims to maintain childrens curiosity in technology through the use of robotics. He holds a European PhD with merit in Information and Technology (2016), from Politecnico di Milano Italy. He also holds a bachelor in Electronic Engineering (2008); Bachelor in Computer Science (2009); Master in Computer Science (2010) from Universidad de los Andes, Bogota Colombia. He has worked as temporal professor at Potificia Universidad Javeriana at the department of computer engineering (Colombia). During this period, he was part of the research group Takina and taught: programming courses (I and II), as programming languages and introduction to artificial intelligence.

Luis Santos holds a PhD from the University of Coimbra (UC), Portugal. He is currently an Invited Researcher at University of Coimbra and is acting as a Project and Technical Manager for the H2020 GrowMeUp Project (GA 643647 12 . He held a postdoctoral position at the Institute of Systems and Robotics of the University of Coimbra from October 2014 to March 2015. He acted as a seconded researcher from the UC to a company, Citard Services Ltd., within the scope of the Marie Curie IAPP Project Social Robot (GA 285870 13 , till the 30th of September, 2014. He holds a Pre-Bologna M.Sc. degree in Electrical and Computer Engineering, specialization in Robotics and Automation by the Faculty of Science and Technology from the University of Coimbra. He worked as researcher in the area of human-robot interaction and action recognition using Laban Movement Analysis, within the BACS European Project from November 2007 to February 2010. His current research topics mainly focus on machine learning methods for autonomous cognitive systems and human motion analysis towards end-user applications.

Conclusion

In summary, the first workshop on "Social Interaction and Multimodal Expression for Socially Intelligent Robots" brings together a multidisciplinary audience to discuss open questions, address difficult challenges and elaborate on novel ways to advance research in the field of mutlimodal interaction, based on theories from human-human interaction and on empirical findings validated human-robot interaction studies.

Advancing the development of socially intelligent robots opens new social and economic opportunities for the application of robots in our daily lives. We hope that the papers and the outcomes of the brainstorming discussion will contribute towards the establishment of a set of standard guidelines for the production of coherent multimodal, verbal and non-verbal communication skills for robots.