Multimodal human understanding and analysis is an emerging research area that cuts through several disciplines like Computer Vision (CV), Natural Language Processing (NLP), Speech Processing, Human-Computer Interaction (HCI), and Multimedia. Several multimodal learning techniques have recently shown the benefit of combining multiple modalities in image-text, audio-visual and video representation learning and various downstream multimodal tasks. At the core, these methods focus on modeling the modalities and their complex interactions by using large amounts of data, different loss functions and deep neural network architectures. However, for many Web and Social media applications, there is the need to model the human, including the understanding of human behavior and perception. For this, it becomes important to consider interdisciplinary approaches, including social sciences, semiotics and psychology. The core is understanding various cross-modal relations, quantifying bias such as social biases, and the applicability of models to real-world problems. Interdisciplinary theories such as semiotics or gestalt psychology can provide additional insights and analysis on perceptual understanding through signs and symbols via multiple modalities. In general, these theories provide a compelling view of multimodality and perception that can further expand computational research and multimedia applications on the Web and Social media. The theme of the workshop includes various interdisciplinary challenges related to social bias analyses, multimodal representation learning, detection of human impressions or sentiment, hate speech, sarcasm in multimodal data, multimodal rhetoric and semantics, and related topics. This year we have accepted four long papers. The contributions of the papers accepted at MUWS 2023 include approaches about exploring multimodal and multi-task approaches to analyze videos, tuning prompt-based approaches for multimodal tasks, investigating biases in knowledge sources and models for multimodal scene graph generation and exploring affection emotion recognition for brain-computer interfaces.
We would like to take this opportunity to sincerely thank the authors for their invaluable and inspiring contributions to the workshops. Our sincere thanks are due to the program committee members for reviewing the submissions and ensuring the high quality of our workshop program. We are also very grateful to the organizers of CIKM 2023 conference, and in particular to the Workshops Chairs, Martin Halvey and Lina Yao, for their support with the workshop organization. We also thank Tuomo Hiippala and Dorothy Zhao for the keynote and invited talks in the workshop.
Gullal S. Cheema Sherzod Hakimov
Marc A. Kastner
Noa Garcia The MUWS 2023 workshop is co-organised by members who received funding from the German Federal Ministry of Education and Research (BMBF, FakeNarratives project, no. 16KIS1517).
Organization