It is important to understand the stories from narrative multimedia contents (e.g., movies), and to exploit the stories for smart services (e.g., video summarization and personalized multimedia recommendation). In this paper, we extend CharNet (Character Network) to Affective CharNet (A ective Character Network) by annotating emotional relationships between characters. More importantly, we propose a novel method on analyzing A ective CharNet for extracting the plots from the narrative multimedia contents.
The goal of this study is to understand the stories (more properly, plots) of
narrative multimedia contents by using A ective CharNet (A ective Character
Network). To model and analyze stories and plots of narrative multimedia
contents, various studies have been conducted with social network analysis (e.g.,
RoleNet [
De nition 1 (Character Network). A character network N can be represented as
N = hCH; Ri (1) where C is a set of characters in the story, and R jCj.
However, they are not able to clearly re ect the plots or storylines. The plot
is de ned as a sequence of events which are described in contents and logically
? Corresponding author
related with each other. As an example, CoCharNet (our previous study [
Another issue on the previous studies is that they assume that all the
social relationships among characters are homogeneous. Jung et al. [
By considering emotional relationships between characters, the character network N can be extended to an a ective character network.
De nition 2 (A ective Character Network). An a ective character network Af N can be represented as
Af N = hCH; E; Ri (2) where C and R is a set of characters in the story and a set of emotions, respectively. The social relationships are represented as R jCj jEj jCj.
To annotate dynamic changes of emotional relationships and detect events between characters, we apply the following two approaches.
{ rst, segmenting contents based on co-occurrence of characters, and { second, detecting events based on radical changes of emotional relationships.
Based on these approaches, plots are exploited by following procedures. { segmenting contents and annotating emotional relationships between characters at each segment, { detecting the events described in contents by searching radical changes of emotional relationships, { and modeling plots based on the events and characters involved in the events.
Prior to annotating emotional relationships on CharNet, we have to extract emotional states of characters at each segment. Data sources to extract emotional states are di erent form kinds of contents. If they are visual contents, we can apply facial expressions. In other case, if they are textual contents, we can use emotional words. Preliminarily, we have manually annotated emotional states as a 1-dimensional value from 1 to 1. If the value is close to 1, it means negative emotional states. In opposite case, it means positive ones.
To transform emotional states into emotional relationships, we have made three assumptions that emotion of a character will be directed to characters { which have high connectivity with him/her, { which are included in same social groups with him/her, and { which have connection with him/her.
Fig 1 is an example of the A ective CharNet, where cha indicates an a-th event and Ea;b means emotional relationship of cha toward chb.
To model and visualize plots of the narrative multimedia contents, we locate detected events in order which is described in contents. Also, by annotating characters involved in each event, we make the model enable to represent logical linkages between characters. Fig. 2 is an example of the proposed computational model of the plots, where ei indicates an i-th event. 3
In this study, we propose a method to annotate emotional relationships on CharNet. Also, we extract and model plots of narrative multimedia contents. It is meaningful in terms of the rst try for computational analysis of plots.
However, this is a manifesto paper with some preliminary research with many
practical limitations. We have planned the following work in future.
{ Most importantly, we need to conduct experimental study to prove the
proposed methods.
{ More re ned method to extract emotional relationships is needed.
{ An appropriate way to evaluate accuracy and e ciency of the proposed
method is necessary.
{ Stori cation [
Acknowledgments. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (NRF2014R1A2A2A05007154). Also, this work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2015S1A5B6037297).