=Paper=
{{Paper
|id=Vol-3825/prefaceW3
|storemode=property
|title=Communication in Human-AI Interaction (Preface)
|pdfUrl=https://ceur-ws.org/Vol-3825/prefaceW3.pdf
|volume=Vol-3825
|authors=Jennifer Renoux,Jasmin Grosinge,Marta Romeo,Kiran M. Sabu,Kim Baraka,Victor Kaptelinin
|dblpUrl=https://dblp.org/rec/conf/hhai/RenouxGRSBK24
}}
==Communication in Human-AI Interaction (Preface)==
https://ceur-ws.org/Vol-3825/prefaceW3.pdf
Communication in Human-AI Interaction - CHAI
(preface)
Jennifer Renoux1,* , Jasmin Grosinger1 , Marta Romeo2 , Kiran M. Sabu1 , Kim Baraka3
and Victor Kaptelinin4
1
Örebro University, Sweden
2
Heriot-Watt University, United Kingdom
3
Vrije Universiteit, Amsterdam, The Netherland
4
Umeå University, Sweden
Abstract
As Artificially Intelligent systems are becoming more and more present in our surroundings, our ways
of interacting with them are also changing. From commercial chatbots to home assistants and robot
companions, machines are progressively taking up the role of “communicators”, provided with their
own agency, and able to interact with their human counterparts in new ways. This workshop aimed
at gathering experts in fields relevant to the study of AI systems as communicators, including but not
limited to Human-Computer Interaction, Artificial Intelligence, Human-Robot and Human-AI Interaction.
It was organized in order to discuss new challenges brought by this recent shift, compare methods and
perspectives between different fields, and foster long-term collaborations.
Keywords
Human-AI Communication, AI Communicators, Multimodal Interaction, Embodied AI, Human-Centered
Design
1. Introduction
Human Interactions with Artificial Intelligence (AI) systems are becoming part of our everyday
life. Generating text and images from prompts, asking for help from a website chatbot, or asking
a voice assistant to play our favorite playlists are only a few of the possibilities that interaction
with AI systems provide. If designed well, these interactions have the potential to enhance
human work, abilities, and well-being. In this workshop, we decided to take the particular
viewpoint in which AI systems are not merely a tool for expression or communication, but
in which they take the role of “communicators”, meaning system with which humans create
meaning [1]. This shift creates many new challenges and opportunities to design new ways for
humans and AI systems to interact. For instance, such AI communicators may have the agency
to initiate communication interactions, and should contribute to such interactions efficiently.
HHAI-WS 2024: Workshops at the Third International Conference on Hybrid Human-Artificial Intelligence (HHAI), June
10—14, 2024, Malmö, Sweden
*
Corresponding author.
$ jennifer.renoux@oru.se (J. Renoux); jasmin.grosinger@oru.se (J. Grosinger); m.romeo@hw.ac.uk (M. Romeo);
kiran.mini-sabu@oru.se (K. M. Sabu); k.baraka@vu.nl (K. Baraka); victor.kaptelinin@umu.se (V. Kaptelinin)
0000-0002-2385-9470 (J. Renoux); 0000-0003-3726-4176 (J. Grosinger); 0000-0003-4438-0255 (M. Romeo);
0009-0005-5111-1629 (K. M. Sabu); 0000-0003-4381-4234 (K. Baraka); 0000-0002-5326-7054 (V. Kaptelinin)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
From the AI-development side, the concept of “machines as communicators” have been greatly
explored albeit mostly between AI systems, as a way to reduce complexity [2] or coordinate a
team of robots [3]. Humans have recently entered the picture [4], especially through the rise of
Human-Robot Interaction (HRI) and social robotics, and more recently through the development
and fast expansion of Large Language Models. With the perspective of having social robots
and conversational agents entering our households or workplaces, developing effective models
of interpersonal and human-machine communication becomes a priority [5]. For this reason,
the need for a deeper connection between communication theory, HRI and social robotics is
more and more acknowledged by the community. Embodied communicators (through social
robots or virtual avatars) also present both challenges and opportunities in the broader field of
communication, as their embodiment provides room to explore a more complex multi-modality
of the interaction [6]. Multi-modality can be used to better convey a concept, to meet the needs
of a diverse pool of users who might have difficulties interacting via a uni-modal channel, and
to try and recover from a failure happening in one of the different channels. However, it is a
challenge in itself and an interdisciplinary effort is needed to direct the design of a multi-modal
communication medium [7].
The study of AI communicators should also include a Human-Computer Interaction (HCI)
perspective. Indeed, the study of communication as a facet of human-technology interaction
have a long history in HCI and semiotic theories informed the engineering of human-computer
interaction [8]. The HCI field is making use of advanced user research methods and concepts,
design approaches, and conceptual frameworks for analyzing UX and social contexts, which are
necessary for the study of human-AI communication.
The primary goal of this workshop was to bridge disciplinary boundaries between various
fields, included but not limited to AI, HRI, and HCI, in order to gather a multi-perspective
view on the topic of Communication in Human-AI Interaction. In particular, we are interested
in exploring the core characteristics of AI communicators and human-AI communication,
exchanging research methods, and fostering long-term collaboration between practitioners
of different fields. As the study of communication in human-AI interaction is by essence a
multidisciplinary approach, we aimed for this workshop to be a multidisciplinary platform where
researchers can learn to work together and pave the way to impacting research. We also wished
to use this opportunity to draw a tentative disciplinary map of the topic of Communication in
Human-AI Interaction, describing different perspectives, research directions, methods, and how
these perspectives can be related to one another within the research area as a whole.
The workshop’s topics of interests included but were not limited to:
• Concepts and theories of communication in human-AI interaction
• Human-AI communication design
• Blended social contexts [9], comprising both human and technological communication
• Communication in multi-user interaction with intelligent agents
• Embodied multi-modal human-AI communication (including physical robots)
• Verbal and non-verbal human-AI communication
• Communication for human-AI collaboration
• Establishing common ground in human-AI communication
• Inclusion and Diversity in Human-AI Communication
2. Organization
2.1. Workshop Chairs
Jennifer Renoux is a senior researcher in Human-Machine Communication and Collaboration
at the Center for Applied Autonomous Sensor Systems, Örebro University, Sweden. Her main
research interest is adaptive communication planning, or how automated systems can create
and adapt their communication strategies to various users, respecting their skills, expertise,
individuality, and preferences, to enable efficient and comfortable collaboration. She is also
interested in critical approaches on Human-AI Communication, and inclusive Human-AI Inter-
action. She was part of the organizing committee of the first edition of the CHAI workshop,
held during IJCAI 2022.
Jasmin Grosinger is a researcher at the Cognitive Robotic Systems lab at the Center for
Applied Autonomous Sensor Systems, Örebro University, Sweden. Her field of research is AI,
specifically the investigation of how to make autonomous (robotic) agents proactive, that is,
able to autonomously initiate actions, including learning actions, that are anticipatory. Such
reasoning spans multiple cognitive abilities such as context awareness, prediction, preference
reasoning, mental simulations of actions, epistemic reasoning, and more. Her methods are
primarily based on formal methods. She was part of the organizing committee of first edition of
the CHAI workshop, at IJCAI 2022.
Marta Romeo is an Assistant Professor in Computer Science for the School of Mathematical
and Computer Sciences at Heriot-Watt University. She got her PhD from the University of
Manchester on human-robot interaction and deep learning for companionship in elderly care.
She then stayed at the University of Manchester as a postdoc for the UKRI Node on Trust,
working on how trust in human-robot interactions is built, maintained and recovered when
lost. Her research focuses on developing social intelligent robots, adapting to their users
for an increased acceptability and usability. She is interested in human-robot interaction,
social robotics, failures and repairs in interactions between humans and robots, and healthcare
technologies.
Kiran M. Sabu is a PhD student at the Center for Applied Autonomous Sensor Systems,
Örebro University, Sweden. He has a Master’s degree in Artificial Intelligence from Vrije
University, Amsterdam, The Netherlands. In his PhD research, he is focusing on developing a
general framework that allows AI agents to plan their communication actions to multiple users
in a Human-Agent collaboration setting, satisfying communication and task-oriented goals.
Kim Baraka is a tenured assistant professor in the Computer Science Department at the
Free University (VU) in Amsterdam, and member of the Social AI group. Before joining the VU,
he was a postdoctoral fellow in the Socially Intelligent Machines Lab at UT Austin. He holds
a dual Ph.D. in Robotics from Carnegie Mellon University (CMU) and the Instituto Superior
Técnico in Lisbon, Portugal and a M.S. in Robotics from CMU. His research focuses on enabling
robots and humans to teach and learn from each other through situated social interactions. As
a professionally trained contemporary dancer, he is also interested in new frontiers in robotics
that draw inspiration from the performing arts.
Victor Kaptelinin is professor of HCI at Umeå University, Sweden. His research interests
include HCI theory, activity-centric computing, robotic telepresence, and social perception
of intelligent agents. His current research focuses on perceived politeness and fairness in
multi-user interaction with embodied intelligent agents. Victor has organized workshops at
CHI, DIS, and ECCE
3. Summary of the workshop
3.1. Submissions
The workshop received a total of 6 submissions. Each paper was peer-reviewed in a single-blind
process by two members of the organizing committee without any conflict of interest with
the authors. The reviewers were instructed to consider how relevant to the workshop the
submission was as well as its potential to initiate interesting and fruitful discussions. The
committee decided to accept 3 papers. Authors of these papers were asked to bring posters.
3.2. Detailed Program
The workshop was highly interdisciplinary and designed to encourage interaction and discussion.
The morning started with a round of introduction from all participants. Then, Dr. Ilaria
Torre, Assistant Professor in Human-Robot Interaction at Chalmers University of Technology,
gave a keynote titled "Voices from the future: creating appropriate verbal and nonverbal
communication methods for Human-Robot Interaction." The remaining of the morning was
filled with a networking and poster session for participants to learn about each-other’s research
and interest and create connections.
Three posters were presented during this session:
1. Frédéric Elisei, Léa Haefflinger and Gérard Bailly, RoboTrio2: Annotated Interactions of a
Teleoperated Robot and Human Dyads for Data-Driven Behavioral Models.
2. Alexander Berman, Argumentative Dialogue As Basis For Human-AI Collaboration.
3. Hadi Banaee, Franziska Klügl, Fjollë Novakazi and Stephanie Lowry, Intention Recognition
and Communication for Human-Robot Collaboration.
The afternoon was organized as a World Café, with three tables and discussion points:
1. What is Human-AI Communication?
2. What are the problems encountered when studying Human-AI Communication?
3. What research methodologies could / should be applied to the study of Human-AI Com-
munication?
3.3. Summary of the presentations
The first poster presented an annotated multimodal corpus of interactions between an
autonomous-looking robot and two humans. The presenter also described the process to collect
the data, which consisted in an immersive teleoperation system using Virtual Reality (VR). The
human operator is equipped with a VR helmet that recreates the visual perception of the robot
(cameras). The operator’s head, chin, lips, and eye movement are transmitted to the robot in
real-time through motion capture and eye tracking. The result is a more natural interaction
between the two human and the teleoperated robot. The present argued that such a method
allows bringing the social know-how, language understanding, and sensory-motor abilities of a
human to a robot, that can then learn by imitation.
The second poster argued that the use of argumentation is under-addressed in the field
of Explainable AI (XAI). The author applied Tolmin’s theory of argumentation to a machine
learning model (logistic regression), for which he showed how to extract data (specific fact)
and warrants (general) for claims (fact, specific). He presented a prototype called MindTone to
showcase this approach for an argumentative AI communicator.
The third poster presented a conceptual framework for intent recognition in human-robot
collaboration, and highlighted different aspects of this problem that need to be considered in the
context of Industry 4.0 / 5.0 -relevant settings. This framework addresses three different aspects:
the temporal sequence of actions (an intermediate intention may not be immediately observ-
able by a single action), the granularity of intentions, and deviation from the predetermined
tasks. The poster also presented the process required by the framework, namely observation
and context analysis, intention recognition, deviation detection, adaptation and reaction, and
communication.
4. Conclusion and Remarks
From the discussions held during the workshop, it appears that the study and design of AI
communicators is indeed a blooming, multi-disciplinary research field. Many aspects need to be
considered and human-AI communication encompasses human-side, technical, system-wide, and
ethical and societal issues. The World Café also highlighted that many research methodologies
applied in different fields may need to be considered and integrated, as a lot of them are usually
absent from AI research practices. Examples of such are co-creation approaches or observational
studies. The discussions also highlighted a strong interest in researchers focusing on human-AI
communication for more interdisciplinary collaboration and widening of the practices. On the
networking side, the workshop has been highly successful as all participants are involved in
follow-up collaborations.
Acknowledgments
This workshop was partially supported by:
• the Swedish Research Council, under grant numbers 2021-05409 and 2022-04676
• the European Union’s Horizon 2020 research and innovation program under grant agree-
ment No 952026
• UKRI TAS Node on Trust (EP/V026682/1)
References
[1] A. L. Guzman, Human-machine communication: Rethinking communication, technology,
and ourselves, Peter Lang Publishing, Incorporated, 2018.
[2] F. S. Melo, M. T. J. Spaan, S. J. Witwicki, QueryPOMDP: POMDP-Based Communication in
Multiagent Systems, in: M. Cossentino, M. Kaisers, K. Tuyls, G. Weiss (Eds.), Multi-Agent
Systems. EUMAS 2011. Lecture Notes in Computer Science, volume 7541, Springer Berlin
Heidelberg, Berlin, Heidelberg, 2011, pp. 189–204.
[3] G. Best, M. Forrai, R. R. Mettu, R. Fitch, Planning-aware communication for decentralised
multi-robot coordination, in: 2018 IEEE International Conference on Robotics and Au-
tomation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, IEEE, 2018, pp. 1050–1057. URL:
https://doi.org/10.1109/ICRA.2018.8460617. doi:10.1109/ICRA.2018.8460617.
[4] K. Inkpen, S. Chancellor, M. De Choudhury, M. Veale, E. P. Baumer, Where is the human?
bridging the gap between ai and hci, in: Extended abstracts of the 2019 chi conference on
human factors in computing systems, 2019, pp. 1–9.
[5] H. A. Frijns, O. Schürer, S. T. Koeszegi, Communication models in human–robot interaction:
An asymmetric model of alterity in human–robot interaction (amodal-hri), International
Journal of Social Robotics (2021).
[6] I. Maurtua, I. Fernandez, A. Tellaeche, J. Kildal, L. Susperregi, A. Ibarguren, B. Sierra,
Natural multimodal communication for human–robot collaboration, International Journal
of Advanced Robotic Systems 14 (2017) 1729881417716043.
[7] K. Fischer, K. S. Lohan, K. Foth, Levels of embodiment: Linguistic analyses of factors
influencing hri, in: Proceedings of the seventh annual ACM/IEEE international conference
on Human-Robot Interaction, 2012, pp. 463–470.
[8] C. S. De Souza, The semiotic engineering of human-computer interaction, MIT press, 2005.
[9] J. Danielsson, K. SäLjedal, V. Kaptelinin, Employing futuristic autobiographies to envi-
sion emerging human-agent interactions: The case of intelligent companions for stress
management, in: 33rd European Conference on Cognitive Ergonomics, 2022, pp. 1–7.