- Current approaches to emotion recognition do not address the fact that emotions are dynamic processes. This work concerns itself with the development of a gray-box framework for dynamic emotion intensity estimation that can incorporate findings from appraisal models, specifically Scherer's Component Process Model. It is based on Dynamic Field Theory which allows the combination of theoretical knowledge with data-driven experimental approaches. Further, we conducted an exemplary user study applying the proposed model to estimate intensity of negative emotions from physiological signals. Results show significant improvements of the proposed model to common methodology and baselines. The flexible cognitive architecture opens a wide field of experiments and directions to deepen the understanding of emotion processes as a whole.
II. MODEL
In the CPM, the subjective feeling (i.e. affective state) is characterized by the emotion intensity I of an emotion quality ϑ at time t and can generally be written as I(ϑ, t). The CPM provides detailed relations between the so-called stimulus evaluation checks (SECs) that happen in the appraisal process and their effects on physiology. For example, a novelty check can lead to an increase in skin conductance or the obstructiveness of an event changes the heart rate of a person.
1Chair of Automatic Control Engineering, Technische Universita¨t Mu¨nchen, Munich, Germany, www.lsr.ei.tum.de, E-mail: rj@tum.de 2Bristol Robotics Laboratory, University of the West of England, Bristol, UK, www.brl.ac.uk, E-mail: angelika.peer@brl.ac.uk
ϑˆ(t)
ˆI(ϑ, t) Fig. 1. The subjective feeling component is divided into consecutive estimation of emotion quality ϑˆ(t) and emotion intensity ˆI(ϑ, t).
Similar to Bailenson et al. [
For our model, we define the field over the emotion quality ϑ as shown in Fig. 2. The core part of the model is the intensity layer i(ϑ, t) together with a memory layer m(ϑ, t), which model the changes in the subjective feeling, i.e. the output ˆI(ϑˆ, t). The second part are the input layers, where we use one layer for each prediction from the SECs provided by the CPM, e.g. u(ϑ, t) in Fig. 2. For example, a change in skin response would be an input layer.
Input
ˆI(ϑˆ, t) ϑ Fig. 2. Architecture of the proposed dynamic model: three-layer field spanned over the dimension of emotion quality ϑ.
III. EXPERIMENTAL DESIGN
We control the emotion quality ϑˆ in our experimental design by fixing it through choice of emotion induction. This results in a simplified dynamic model at one location of the fields, i.e. three neurons and their governing equations.
For the dataset, we recorded the galvanic skin response
(GSR) of subjects. Additionally, we used a slider device
interface to record the emotion intensity experienced by the
subject. For emotion induction, we used standardized IAPS
pictures of a fixed emotion quality, here, negative emotions
[
The change in GSR is computed as a prediction of SECs and used as model input. The continuously recorded intensity measures of the slider served as ground truth. For training of the dynamic model, free parameters are determined by means of experimental data applying leave-one-out cross validation. In this, we minimize the error between the output of the dynamic model and the ground truth s.t. boundary conditions.
First, we compare the accuracy of our model with common static methods and baselines, i.e. linear regression and random regressors. We use the match of estimate with ground truth plus an acceptable error margin as accuracy measure. In summary, the dynamic model performs significantly better than common methodology and baselines. Limitations of the model become apparent for small error margins.
Secondly, capabilities and limitations of the model in its current version are examplified in Fig. 3. In the upper graph, we see the changes in GSR, which characterize the onset as well as the increase of intensity well. The memory layer (bottom graph) helps to stabilize the decay at an appropriate rate. However, limitations of the current model are apparent, as the third change in GSR should not have any impact on the intensity. This points towards the need to include additional input layers where appropriate interaction can avoid this behavior.
For the first time, a dynamic gray-box model framework based on DFT has been proposed for emotion recognition, which allows to include theoretical knowledge into the model and learn free parameters from experimental results. We designed and carried out an exemplary study to estimate emotion intensity from physiological signals. In this, the dynamic model performed significantly better than baselines. We also identified current limitations and ways to improve the model. Future work includes several extension to the architecture as well as carrying out experiments to further evaluate the model. 0 1 0 0 1 0