Artificial Intelligence (AI) has become a fundamental ally to improve the reliability, eficiency, and efectiveness of Human-Machine Interaction (HMI) systems. This is especially true considering the latest advancements of wearable and sensing technologies, which are accompanying us in every-day life, and are exploited in diferent applications spanning from continuous patient monitoring to immersive video-game experiences. While these novel applications can positively impact a user's life, work, education, health, and free-time, their development should be rigorous and consider a possible real-time configuration, the quality and quantity of available data, the portability of the employed technologies, the scalability of AI strategies, and the ethical and regulatory aspects concerning the use of personal data, just to name a few challenges. The AIxHMI workshop aims to connect researchers and practitioners from diferent fields to collect multidisciplinary contributions on topics concerning HMI and especially on the influence that AI has in the interaction between humans and machines. Thirteen papers have been submitted to the third edition of AIxHMI. Out of these, two have been accepted for oral presentation as abstracts, five as short papers, and five as regular papers. The authors and three invited speakers have presented very diverse topics and approaches bounded to the HMI ifeld, from the use of large language models to the assessment of agreement and reliance in learning tasks, from biomarkers and multimedia signals to robots used in treating autism spectrum condition, from generative art to the use of augmented reality.
be guarded from possible dual use, the resulting data are usually of lower quantity and quality in respect to their wired counterparts, the system needs to provide an instantaneous and proper feedback, and in general should follow some ergonomic rules related to its usability, such as satisfaction, eficiency, and efectiveness of the BCI.
These issues can be expanded and translated to systems employing other control and sensing devices, which can be configured in a multi sensorial and multimodal fashion, require the integration of heterogeneous data, and consider the user environment as part of the information to be used. Moreover, a key aspect is the emotional involvement of the users when dealing with HMI systems, thus giving space to the fields of emotional intelligence and afective computing. In fact, having machines that are able to adapt to the emotional states of their users may provide better communication between them. For example, being able to detect frustration could allow the re-modeling of a specific control system to the necessities of a single user.
This observation highlights the need to move towards human-centered computing and sensing, ensuring a better user experience. It is again necessary to provide a good data quality, organization and management, considering that these data come from multiple sources.
Therefore, the AIxHMI workshop wants to assemble multidisciplinary contributions that pertain but that are not limited to the fields of HMI, BCI, control systems, wearable sensing and devices, virtual and augmented reality, emotional intelligence, afective computing, human-centered sensing and computing, human factors and ergonomics, user experience, interface and sensor design, and ethics and security in AI, having that the AI is a transversal discipline that influences all these aspects.
Thirteen submissions have been sent by 58 authors to the AIxHMI workshop and twelve have been accepted in this volume with the following distribution: • Two abstracts for oral presentation; • Five short papers of which one is an experimental protocol proposal, and two a preliminary study design.
• Five regular papers of which two are pilot studies.
In particular, C. Fregosi presented the abstract co-authored with A. Campagner, C. Natali, and F. Cabitza entitled “Assessing appropriate reliance: a framework for evaluating AI influence on user decision-making”, providing some insights on the concept of appropriate reliance, i.e., on the human capability of deciding when to trust the suggestions given by a machine.
Instead, N.A. Borghese started from the abstract entitled “Co-design of scenarios for interacting with a NAO robot in treating autism spectrum condition” to present a long term project intended to leverage on social-robots as assistants to train socio-cognitive skills in children with autism spectrum conditions. Co-authors of this contribution are F. Ciardo, E. Chitti, R. Scuotto, R. Actis-Grosso, F. Cavallo, L. Fiorini, L. Pugi, B. Olivari, M.A. Tedoldi, C. Carenzi, and P. Ricciardelli.
Crocamo et al. [
Instead, Cazzaniga, Gasparini, and Saibene [
A particular attention to the users is also given by Arabi et al. [4] and Antico et al. [5]. In the first contribution, the authors develop a GPT-based chatbot (Habit Coach) to support users’ habit change leveraging on cognitive behavioral and and narrative therapy techniques. Testing their proposal, the authors found that a reduction on habit-strength has been detected on the participants to their study. Instead, Antico et al. leverage on a retrieval-augmented generation system to refine ChatGPT behavior when dealing with the needs of University of Milano-Bicocca students, searching for university-related information. The proposed Unimib Assistant passed diferent testing and usability phases that allow to propose future refinements of the tool, already evaluated as friendly and clear.
Another contribution focusing on Large Language Models (LLMs) is the one by Franch, Roberti, and Blanzieri [6], who propose a methodology to fine-tune LLMs in domain-specific contexts by enforcing specific rules to improve the model behavior on LLM self-generated training datasets. The proposed training gives the model the ability to extend the rules used in a specific context to closely-related contexts.
Considering the power of AI to generate content, Serrao et al. [7] propose a study design intended to understand the ability that generative AI has in expressing emotions through non-figurative elements. Abstract images on six target emotions, i.e., joy, sadness, fear, anger, disgust, and surprise, will be generated and future participants will be asked to assess if these emotions are efectively represented by the generated artworks.
This study can be also related to the proposal by Perez and Rabaioli [8], who use AI to generate visual representations of subjective physiological responses to music tracks, aiming at providing interactive sound environments especially in the context of music therapy. This preliminary assessment of the “Soundscapes of the soul" project provides an initial understanding on how immersive technologies leveraging on artistic sensibility and subjective data could provide new applications in music therapy. Fontana et al. [9] focus instead on other cognitive aspects that are mainly-related to people attention while driving. An experimental protocol relying on visual distractors appearing in a virtual environment is described with the aim of proposing a safe configuration to evaluate the efects of the defined distractors while driving. Drivers’ reaction time and lane keeping will be measured to assess the efect of distractors on attention.
A more playful application is instead presented by Chitti et al. [10]. The authors try to understand mechanisms and rules that should characterize an augmented reality Monopoly board-game to improve users’ experience in terms of engagement and immersion. Custom rules and designs are proposed as well as a proper game interface. The players’ assessment reveals that while the game is engaging, play for a long time induces fatigue. Future developments will take into consideration this and other user-related requirements.
Besides the oral presentation of the aforementioned contributions, three invited speakers participated in the AIxHMI workshop: • Chiara Capra, CEO of LIFE Neurotech and CPO at Sense4Care (Spain). • Francesca Gasparini, Associate Professor at the Department of Informatics, Systems, and Communication of the University of Milano-Bicocca (Italy). • Angelika Peer, Full Professor at Faculty of Engineering of the Free University of Bozen-Bolzano (Italy).
Chiara Capra speech, entitled “AI to detect Parkinson’s disease symptoms via wearables: from detection to management to treatment”, focused on the importance of early detection of Parkinson’s disease (PD) and the consequent necessity of continuously monitoring patients to provide an efective and personalized treatment. In particular, she presented successful case studies involving the use of the PD Holter monitor STAT-ON by Sense4Care, relying on AI to understand the progression on PD, suggest interventions, and follow remotely the patients.
Francesca Gasparini provided more insights on the topic of subjective data by contributing with her speech entitled “AI personalised models based on subjective data”. She highlighted the importance of considering the data quality, quantity, the inter and intra-subject variability to evaluate the reliability of personalized AI-based advanced human-system interfaces. Moreover, some points have been raised to consider the emotional engagement that people have when interacting with AI-based systems, providing diferent examples starting from the afective computing cycle.
Finally, Angelika Peer wrapped the speeches by delivering her talk on “The role of physiological signals in human-machine interaction”. She gave a complete excursus on the key aspects related to the efective and eficient use of physiological signals in building brain and body computer interfaces in real-world scenarios. Particular emphasis has been given on the importance of introducing contextual information to better understand people intentions when using such systems.
Acknowledgments The AIxHMI workshop co-chairs would like to thank all the Program Committee members for their reviewing and dissemination activities: • Gloria Beraldo, ISTC-CNR (Italy). • Cesar Caiafa, CONICET (Argentina). • Giulia Cisotto, University of Trieste (Italy). • Nicolò Dozio, Politecnico di Milano (Italy). • Francesco Ferrise, Politecnico di Milano (Italy). • Mirco Gallazzi, University of Insubria (Italy). • Ignazio Gallo, University of Insubria (Italy). • Francesca Gasparini, University of Milano-Bicocca (Italy). • Riccardo Giussani, Politecnico di Milano (Italy). • Alessandra Grossi, University of Milano-Bicocca (Italy). • Carlotta Lega, University of Pavia (Italy). • Karmele Lopez de Ipiña, University of the Basque Country (Spain). • Pere Marti-Puig, University of Vic (Spain). • Ruggero Micheletto, Yokohama City University (Japan). • Marta Molinas, Norwegian University of Science and Technology (Norway). • Evangelos Niforatos, Delft University of Technology (The Netherlands). • Claudia Rabaioli, University of Milano-Bicocca (Italy). • Agnese Sbrollini, Università Politecnica delle Marche (Italy). • Marta Maria Sosa Navarro, University of Milano-Bicocca (Italy). • Toshihisa Tanaka, Tokyo University of Agriculture and Technology (Japan).
• Sun Zhe, Juntendo University (Japan). [4] A. F. M. Arabi, C. Koyuturk, M. O’Mahony, R. Calati, D. Ognibene, Habit Coach: Customising RAG-based chatbots to support behavior change, in: Proceedings of the Third Workshop on Artificial Intelligence for Human-Machine Interaction (AIxHMI 2024) co-located with the 23rd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2024), CEUR Workshop Proceedings, CEUR-WS.org, 2024, pp. 57–70. [5] C. Antico, S. Giordano, C. Koyuturk, D. Ognibene, Unimib Assistant: designing a student-friendly RAG-based chatbot for all their needs, in: Proceedings of the Third Workshop on Artificial Intelligence for Human-Machine Interaction (AIxHMI 2024) co-located with the 23rd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2024), CEUR Workshop Proceedings, CEUR-WS.org, 2024, pp. 71–82. [6] D. Franch, P. Roberti, E. Blanzieri, Rule enforcement in LLMs: a parameter eficient fine-tuning approach with self-generated training dataset, in: Proceedings of the Third Workshop on Artificial Intelligence for Human-Machine Interaction (AIxHMI 2024) co-located with the 23rd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2024), CEUR Workshop Proceedings, CEUR-WS.org, 2024, pp. 17–32. [7] F. Serrao, A. Gallace, M. Gallucci, A. Gabbiadini, The ability of generative AI to express emotions through abstract images: a preliminary study design, in: Proceedings of the Third Workshop on Artificial Intelligence for Human-Machine Interaction (AIxHMI 2024) co-located with the 23rd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2024), CEUR Workshop Proceedings, CEUR-WS.org, 2024, pp. 51–56. [8] F. Perez, C. Rabaioli, Soundscapes of the soul: armonia vitale, in: Proceedings of the Third Workshop on Artificial Intelligence for Human-Machine Interaction (AIxHMI 2024) co-located with the 23rd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2024), CEUR Workshop Proceedings, CEUR-WS.org, 2024, pp. 93–101. [9] S. Fontana, A. Massironi, M. A. Petilli, C. Lega, E. Bricolo, A protocol to evaluate the impact of visual distractors on driving attention using a virtual reality simulator, in: Proceedings of the Third Workshop on Artificial Intelligence for Human-Machine Interaction (AIxHMI 2024) co-located with the 23rd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2024), CEUR Workshop Proceedings, CEUR-WS.org, 2024, pp. 8–16. [10] E. Chitti, A. Barbagallo, A. Delia, N. A. Borghese, ARnopoly: exploring strengths and weaknesses of AR experience enhancing board games, in: Proceedings of the Third Workshop on Artificial Intelligence for Human-Machine Interaction (AIxHMI 2024) co-located with the 23rd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2024), CEUR Workshop Proceedings, CEUR-WS.org, 2024, pp. 44–50.