In the era of digital technologies permeating numerous aspects of society, the development of intelligent artificial agents endowed with autonomy, social capabilities, reactivity, and proactivity has become a pivotal innovation. Traditionally, these agents were confined to singular forms, known as embodiments, which dictated their capabilities and the environments in which they operated. To transcend these limitations and enable agents to expand their capabilities, the concept of agent migration has emerged, allowing them to seamlessly transition between diferent embodiments. My research program delves into the realm of agent migration, uncovering its challenges and potential opportunities. The central objective is to leverage this technology to deepen our understanding of human afective and cognitive states across various contexts. Specifically, my aim is to develop a migratory agent architecture capable of recognizing and characterizing individuals' emotional experiences using diferent sets of capabilities tied to distinct embodiments. In summary, the intent of this research is to harness migratable AI to promote the technical advantages brought about by agent migration and gain insights into how to utilize the acquisition of additional capabilities resulting from migration to craft a more comprehensive and potentially enduring architecture, to be applied in the realm of afective computing.
Nowadays, an increasing number of human activities rely on digital technologies in the current
society [
The concept of an agent capable of migration allows for the separation of the agent’s mind
from the body it occupies [
The objective of my Ph.D. research program (started this year) is to examine the concept of agent migration, exploring its challenges and opportunities, with the aim of harnessing this technology to enhance the comprehension of the afective and cognitive states of individuals across various contexts. Specifically, the goal is to develop a migratory agent architecture that enables the recognition and comprehensive characterization of subject’s emotional experiences, utilizing diferent sets of capabilities based on the embodiment it assumes.
Focus on Migratable AI. Migrant agents and the migration process have been studied in
the context of social interactions and human-machine interactions, i.e., HRI (Human-Robot
Interaction) and HCI (Human-Computer Interaction). Pioneering this field, Imai et al. (1999)
addressed agent migration with the goal of enhancing human-robot interaction [
In the 2000s, the UCD (University College Dublin) team introduced the Agent Chameleons
architecture with the aim of enabling seamless transitions between physical and virtual
information environments, emphasizing the crucial role of migration as a means for expanding the
capabilities of autonomous systems [
Subsequently, research began to consider the efects of migration on users. Indeed, while
migration enhances agent capabilities and frees them from environmental constraints, the
critical issue of user recognition of the agent’s identity emerged. Studies, such as that of Gomes
et al. [
It is evident, therefore, that the agent’s identity constitutes a key feature in the migration process, fostering the establishment of stable and enduring relationships between the user and the agent.
Focus on Emotion Recognition. Numerous attempts have been made to classify emotions
in order to understand human behavior [
One of the channels used to gather emotional information is speech. In this context various approaches are available depending on the audio signal format. These data can be processed as raw audio waveforms or in 2D format, such as spectrograms [12]. When analyzing raw audio waveform data, a widely used approach involves the use of the WaveNet architecture [13, 14]. On the other hand, deep learning technologies, such as Long Short-Term Memory (LSTM) artificial neural networks integrated with Convolutional Neural Networks (CNN), have been employed for the analysis of 2D audio signals [15, 16, 17]. Another relevant methodology involves the use of transformers [18].
Facial expressions, similar to natural language, are a manifestation of human emotions. The Facial Action Coding System (FACS) [19], developed by Ekman and Friesen in 1978, is the most well-known and widely used system for facial activity analysis. More recently, with the introduction of Graphics Processing Units (GPUs) and Convolutional Neural Networks, real-time applications have been developed, such as Microsoft service Emotion API [20] and the MIT Media Lab Afectiva technology Afedex [21].
From a physiological perspective, emotions leave traces that can be studied. Signals such as electrodermal activity, also known as Galvanic Skin Response (GSR), heart and respiratory rate, brain activity, and many others are typically considered when attempting emotion classification through a dimensional approach [22].
In conclusion, it is evident that emotion detection and, more broadly, Afective Computing are fields of great interest, despite the ongoing challenges.
The literature review illustrates that agent migration is not devoid of challenges and complications. One of the primary challenges concerns the continuous recognition of the agent by the user, as the various embodiments may not be immediately attributed to a singular entity. This has prompted fundamental inquiries, which have undergone comprehensive scrutiny, regarding the nature of the agent’s identity and the mechanisms facilitating recognition when its form may undergo changes. Indeed, the quest for optimal interaction between humans and agents has constituted the central focus of recent research in the domain of migratable AI. Simultaneously, it becomes imperative to understand how to leverage migratable AI, what technical benefits agent migration can bring and how to use the acquisition of additional capabilities brought about by migration to produce a more comprehensive and potentially enduring architecture. These fundamental questions have served as the driving impetus behind my research endeavors in this field.
Another key aspect in the designing of the research project was the recognition of the significance of analyzing individuals’ emotional states. The conducted assessment revealed that emotions represent a fundamental form of non-verbal interaction, and the ability to express emotional states prompted by everyday situations is a crucial aspect of human life. Simultaneously, the capacity to recognize emotional states expressed by others plays a pivotal role in both interpersonal relationships and the psychological well-being of individuals. Afective computing constitutes the main research area dedicated to the analysis of these emotional states: it leverages information acquired through the analysis of specific signals, such as facial expressions, natural language, body gestures, and physiological cues, to infer individuals’ afective-cognitive states. However, research has underscored that information such as natural language and facial expressions, when considered in isolation, may lead to erroneous interpretations, as they can be consciously controlled by humans who may choose to conceal their emotional states. Conversely, physiological signals represent a more objective and reliable type of information, albeit more challenging to capture and, most importantly, interpret. These considerations have prompted the idea of applying migratable artificial intelligence to the field of afective computing.
The main objective of this research is to develop an intelligent agent capable of migrating between diferent embodiments in order to leverage a broader range of capabilities and assist the user across various types of tasks. The capabilities of this intelligent system to migrate will enable the consistent and accurate integration of various type of information gathered during user interaction.
The proposed project aims to harness these capabilities to enhance the recognition of emotions expressed by individuals. Specifically, the agent’s ability to migrate across diferent embodiments will enable it to acquire, analyze, and combine various types of user-related information. Consequently, the recognized emotional state will no longer rely on a single information channel but will result from a comprehensive analysis involving diverse data sources.
This work delves into the concept of agent migration, shedding light on both its advantages and the remaining challenges. Specifically, it identifies migration as the means to enhance the versatility and adaptability of intelligent agents across various contexts.
The main objective of this research is to expand the range of capabilities within the intelligent system, enabling it to assist users in diverse tasks. The focal point of this endeavor lies in the analysis of emotions and afective computing, recognizing the paramount importance of emotional state analysis in human interactions. Consequently, the aim is to utilize migratory capabilities to integrate diverse user-related information, ultimately enhancing the capacity to recognize individuals’ emotional states.
In conclusion, this research seeks to harness migratory artificial intelligence, identify the technical advantages stemming from agent migration, and acquire insights on how to leverage the acquisition of additional capabilities brought about by migration to develop more comprehensive and potentially enduring architectures in the field of afective computing.
This research is carried out as PhD student enrolled in the National PhD in Artificial Intelligence for Health and Life Sciences, XXXVIII Bis cycle, Università Campus Bio-Medico di Roma. [12] M. Zielonka, A. Piastowski, A. Czyżewski, P. Nadachowski, M. Operlejn, K. Kaczor, Recognition of emotions in speech using convolutional neural networks on diferent datasets, Electronics 11 (2022). [13] A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, K. Kavukcuoglu, Wavenet: A generative model for raw audio, arXiv preprint arXiv:1609.03499 (2016). [14] S. K. Pandey, H. S. Shekhawat, S. Prasanna, Emotion recognition from raw speech using wavenet, in: TENCON 2019-2019 IEEE Region 10 Conference (TENCON), IEEE, 2019, pp. 1292–1297. [15] J. Wang, M. Xue, R. Culhane, E. Diao, J. Ding, V. Tarokh, Speech emotion recognition with dual-sequence lstm architecture, in: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 6474–6478. [16] C.-W. Huang, S. S. Narayanan, Deep convolutional recurrent neural network with attention mechanism for robust speech emotion recognition, in: 2017 IEEE international conference on multimedia and expo (ICME), IEEE, 2017. [17] G. Trigeorgis, F. Ringeval, R. Brueckner, E. Marchi, M. A. Nicolaou, B. Schuller, S. Zafeiriou, Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network, in: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2016, pp. 5200–5204. [18] G. Tamulevičius, G. Korvel, A. B. Yayak, P. Treigys, J. Bernatavičienė, B. Kostek, A study of cross-linguistic speech emotion recognition based on 2d feature spaces, Electronics 9 (2020) 1725. [19] P. Ekman, W. V. Friesen, Facial action coding system, Environmental Psychology &
Nonverbal Behavior (1978). [20] A. Verma, D. Malla, A. K. Choudhary, V. Arora, A detailed study of azure platform & its cognitive services, in: 2019 International conference on machine learning, big data, cloud and parallel computing (COMITCon), IEEE, 2019, pp. 129–134. [21] D. McDuf, A. Mahmoud, M. Mavadati, M. Amr, J. Turcot, R. e. Kaliouby, Afdex sdk: a cross-platform real-time multi-face expression recognition toolkit, in: Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems, 2016, pp. 3723–3726. [22] L. Shu, J. Xie, M. Yang, Z. Li, Z. Li, D. Liao, X. Xu, X. Yang, A review of emotion recognition using physiological signals, Sensors 18 (2018) 2074.