Preface
The second MiGA workshop and challenge seeks to broaden the research community focused on micro-gesture analysis and its applications in emotion understanding, by introducing more modalities into the task (both RGB and skeleton modalities). The event aims to foster dialogue among researchers from academia and industry, highlighting key attributes influencing gesture-based emotion recognition and evaluating recent advancements in the field. Similar to the first MiGA, we introduce two distinct datasets (SMG and iMiGUE datasets) and corresponding benchmarks (classification and online recognition), with the goal of shaping a new trajectory for the emotion AI community.
foundations, technological advancements, and practical applications of gestures and micro-gestures in emotion understanding. Discussions encompassed vision-based approaches for gesture-based emotion recognition, including tasks such as classification, detection, and online recognition. The event also highlighted newly collected datasets designed to support emotion understanding and explored diverse applications of gestures and micro-gestures, such as emotion assessment in contexts like education. The
speakers: Prof. James Wang from Pennsylvania State University, USA, and Assistant Professor Hao
insightful and thought-provoking talks. We are equally grateful to all the participants for their invaluable contributions, which were instrumental in making MiGA 2024 a remarkable event and a dynamic forum for knowledge exchange within the community. Their engagement sparked vibrant discussions on pivotal and contemporary advancements, highlighting an exceptional program that exemplified cutting-edge work at the intersection of AI and emotion AI. Special thanks also gives to
The following full papers presenting original research works were accepted, and we divided them into three sessions based on the content of the work.
In Session1: MiGA Classification Schemes, Li et al. presented a framework the method is built on a two-stream 3D CNN backbone, dedicated to RGB and skeleton data each for micro-gesture classification, this scheme won first place in the micro-gesture classification track. Huang et al. introduced their second-place winning scheme for micro-gesture classification based on a framework named M2HEN, which constructs a heterogeneous ensemble network by combining two fundamentally different deep learning models: a 3D convolution-based model and a Transformer-based model. Wang et al. present a method rooted in the CLIP framework. Building on Froster CLIP, they introduce a token attenuation strategy within the video encoding module, which incrementally filters out less significant tokens at each layer.
In Session2: MiGA Online Recognition Schemes, Huang, et al. introduced the first-runner scheme on the online micro-gesture recognition track with a network that is primarily composed of two key elements: a 3D convolutional network (RGBPose-Conv3D) and a multi-scale Transformer encoder. The approach introduced by Li et al. is composed of a video encoder and an action decoder which incorporates a Mamba-MHSA module and a multi-level interaction module.
In Session3: Human Behaviour Analysis for Emotion Understanding, Xia, et al. reported a novel framework that uses event data to recognize micro-gestures and micro-expressions, which is a novel research entry and the features of an event camera meet the nature of those short and rapid movements of human behaviors. At last, a summary work of all the plans proposed in the competition of MiGA 2 from Chen et al. was presented where a detailed technical analysis of all the prize-winning schemes.
Artificial Emotional Intelligence and Bodily Expression In the Wild (Prof. James Wang, Pennsylvania