Most of emotional speech databases are recorded by actors and some of spontaneous databases are not free of charge. To progress in emotional recognition, it is necessary to carry out a big data acquisition task. The current work gives a methodology to capture spontaneous emotions through a cooperative video game. Our methodology is based on three new concepts: novelty, reproducibility and ubiquity. Moreover, we have developed an experiment to capture spontaneous speech and video recordings in a controlled environment in order to obtain high quality samples.
Capturing emotions is an arduous task, above all when we speak about
capturing and identifying spontaneous emotions in voice. Major progress has been
made in the capturing and identifying gestural or body emotions [
Typically, emotional databases have been recorded by actors simulating emotional speech. These actors read the same sentence with di erent tones. In our research, we have requested the collaboration of di erent volunteers with di erent ages and gender. Most of volunteers were students who donated their voices. First of all, they had to give their consent in order to participate in our experiment. Therefore, we show a novel way of obtaining new speech recordings. The next key feature for this task is reproducibility, where each experiment should provide spontaneity, although the exercise was repeated a lot of times. This characteristic is the most important drawback we have found in the literature. Most of the time, when it carries out an experiment, this user is discarded immediately, because he/she knows perfectly the guideline of the exercise, for this reason the spontaneity is deleted. The third feature is, ubiquity. When we speak about this concept, we refer to carrying out the exercise in every part of the world, but that does not mean that we cannot use the same location or devices. Nowadays, new technologies such as smart-phones, tablets and the like are necessary allies in this aspect.
In view of all the above, multiplayer videogames are the perfect way to achieve the last three premises. Each game session or scenario can be di erent. Moreover, we can play at home, in a laboratory or anywhere. Thanks to the Internet, we can nd di erent players or rivals around the world who speak other languages and have other cultures, etcetera. Each videogame has its own rules, thus each player knows the game system and they follow these rules if they want to participate. For this reason we nd the standardization feature intrinsic in the videogames. Therefore, we conclude videogames are the perfect tool in order to elicit spontaneous emotions.
This research has two main stages; they consist of capturing emotions through a videogame, more speci cally League of Legends (aka LoL), and identify the captured emotions through the new cooperative framework developed by our team. To assess the viability of our protocol, we have developed a controlled experiment in our laboratory. In subsequent sections, it will be explained in detail.
The contribution of this work is to establish a community to cooperate in collecting, developing and analyzing emotional speech data and de ne a standard corpus in di erent languages where the main source of samples will be emotional speeches captured through videogame rounds. In this sense, this paper is a rst step in proposing the design and development of an online cooperative framework for multilingual data acquisition of emotional speech. This paper is organized as follows. In the next section, we introduce some emotional databases of speech and foundations for modeling emotions in games. In section III, we introduce the proposed experiment. And nally, we conclude with the summary and future works.
Below, we present previous works carried out by di erent researchers who have focused their attention in emotional areas. Some of them have elaborated emotional databases, others have developed a ective models to improve the realism of NPCs (Non Player Character) or have attempted to verbalize certain situations that happen for game rounds, etc. We are going to attempt to nd a common ground between using videogames and the design of a protocol to capture emotions through voice. 2.1
According to [
Concerning elicit emotion and emotional responses to videogames, in [
On the other hand, some researchers focus their attention on
psychophysiological methods in game research. Kivikangas et al. carry out a complete
review of some works in relation with these kind of methods. They present the
most useful measurements and their e ects in di erent research areas such as
game e ects, game events, game design and elicited emotions.
Electromyography (EMG), Electrodermal activity (EDA), Heart Rate (HR), among others, are
some of these measurements [
Another initiative was developed by [
Regarding emotion modeling in game characters, Hudlicka and Broekens
present theoretical background and some practical guidelines for developing
models of emotional e ects on cognition in NPCs [
Most databases have been recorded by actors simulating emotional discourses
and there are a very few of them of spontaneous speech [
Two of the well-known emotional databases for speech are the Danish
Emotional Speech Database (DES) [
Concerning stress in speech, the SUSAS English database (Speech Under
Simulated and Actual Stress) is public and widely used [
Finally, the work closest to our approach that we have found in literature,
has been in [
As mentioned before, we used a videogame-like source of elicited emotions. The
chosen game was League of Legends (aka LoL). To carry out this task, it was
necessary to organize a little tournament, where the team with the best score at
the end of the tournament, obtained a check for the amount of 20 e per person
as well as a diploma. With the obtained samples, we attempt to nd correlates
between acoustics, glotals or biomechanics parameters and the elicited emotions.
To extract these parameters, we have used [
The subjects are students of Computer Science at the Universidad Politecnica of Madrid. Apparently, students had not got any disease in their voices and they gave their explicit consent in order to participate in the experiment. 3.2
LoL is a multiplayer online battle arena (MOBA) video game developed and
published by Riot Games. It is a free-to-play game supported by micro-transactions
and inspired by the mod Defense of the Ancients for the video game Warcraft
III: The Frozen Throne. League of Legends was generally well received at
release, and it has grown in popularity in the years since. By July 2012, League of
Legends was the most played PC game in North America and Europe in terms
of the number of hours played [
Each of the game sessions are carried out in our Laboratory, Neuromorphic
Speech Processing Laboratory, which belongs to R+D group Centro de
Tecnolog a Biomedica. In this laboratory, we possess a quasi-anechoic chamber that
is designed in order to entirely absorb the acoustic waves without echoing o
any surface of the chamber such the oor, roof, walls and the like. The chamber
has a personal computer inside, where a player remains throughout the game
round. Outside of the chamber, there will be another four personal computers
at the disposal of four members of the rest of the team. Concerning the choice
of the anechoic chamber, it is easy to understand that we are looking for ideal
conditions in order to develop of following stages such as characterization and
extraction of parameters, selection of parameters, classi cation and nally,
detection of emotions in speech [
During two weeks, we will convene ten subjects in two shift sessions. The will be one shift in the morning and another one in the afternoon. Each session will have ve players, where each player will log in to LoL's website with his/her o cial account. Four of ve players remain together in the same room, whereas the remaining player will be isolated inside of anechoic chamber. Approximately, each session will be limited to three hours and a half, because the average length of a game is 40 minutes. Once a game is over, the player who stays inside the chamber, will come out the chamber and the following mate takes his place. This continues until ve rounds have been completed. Therefore, each day of tournament, we will have recorded 10 di erent players for 40 minutes. These records will be saved in our raw speech recordings database. Each team will play a maximum of 3 games over the two weeks of tournament. The process of the experiment is depicted in the Fig. 1. This experiment and others, which although they are not the objective of this paper to analyze them deeply, have been developed in order to elicit spontaneous emotions by our team. These experiments are incorporated inside the framework Emotions Portal. 3.6
The aim of this experiment is the capturing of speech recordings through a cooperative online videogame. The idea is that our server collects spontaneous emotional voice in di erent languages, with di erent accents and origins, etc. This framework can be de ned as cooperative, scalable or modular and not subjective. According to Fig. 2, we divide our online framework into four stages: User identi cation, Start of the recording, Play the game and End of the recording and save the speech and video Recording. At the top of the picture is depicted the player who is inside of the anechoic chamber. He/she is connected to our platform which is deployed in our server. The user identi cation step consists of a sign up process through a web form. In this web form, users provide their personal data, for instance, their name, native language, country, gender, age and email. The last requirement, email address, is convenient in order to have the chance to keep in touch with the user and to give him/her information about the progress of the project. Once logged in to our platform, the user can choose
Fig. 1. Scenario of Experiment. the di erent kind of experiments which are available. The actions mentioned before are explained as follows.
First of all, users answer if they su er from any type of disease in their voice at the moment of performing the test. It is crucial to know if the user su ers from any organic or functional dysphonia, diseases that can a ect voice and prevent the use of biometric techniques. Then, the player chooses the language of procedure. When the user is ready to start playing, he/she presses the Start button. In this moment, the platform begins recording and throws a call to LoL's application. LoL's platform opens and the user carries out the process of sign in with his/her LoL's account. These steps are depicted through numbers 1 and 2 in Figure 2. Approximately 40 minutes later, the game round is over. The player signs out of the LoL's platform and he/she presses the Stop button in our platform. Finally, the speech and video recording are saved in our repository inside of our server. The last steps are depicted through numbers 3 and 4 in Figure 2. 4
The spontaneous data acquisition is a very complex topic to resolve. However, it is the key point to improve human computer interfaces, robots, video games and the like. As mentioned before, some researchers have developed new models and methodologies to improve NPCs, and others have researched new designs to evoke certain emotions during the game sessions. We have developed a methodology in order to capture spontaneous emotions through a cooperative videogame with high quality audio in a controlled environment. Afterwards, we will be able to design a well-labeled database and continue with our previous work on emotion recognition.
Acknowledgments. This work is being funded by grants TEC2012-38630-C0401 and TEC2012-38630-C04-04 from Plan Nacional de I+D+i, Ministry of Economic A airs and Competitiveness of Spain.