Linguistic and Gender Variation in Speech Emotion Recognition Using Spectral Features?

Abstract

This work explores the e ect of gender and linguistic-based vocal variations on the accuracy of emotive expression classi cation. Emotive expressions are considered from the perspective of spectral features in speech (Mel-frequency Cepstral Coe cient, Melspectrogram, Spectral Contrast). Emotions are considered from the perspective of Basic Emotion Theory. A convolutional neural network is utilised to classify emotive expressions in emotive audio datasets in English, German, and Italian. Vocal variations for spectral features assessed by (i) a comparative analysis identifying suitable spectral features, (ii) the classi cation performance for mono, multi and cross-lingual emotive data and (iii) an empirical evaluation of a machine learning model to assess the e ects of gender and linguistic variation on classi cation accuracy. The results showed that spectral features provide a potential avenue for increasing emotive expression classi cation. Additionally, the accuracy of emotive expression classi cation was high within mono and cross-lingual emotive data, but poor in multi-lingual data. Similarly, there were di erences in classi cation accuracy between gender populations. These results demonstrate the importance of accounting for population di erences to enable accurate speech emotion recognition.

Keywords

• A ective Computing
• Speech Emotion Recognition
• Machine Learning
• Prosody Analysis
• Convolutional Neural Networks

Introduction

Speech emotion recognition (SER) is the classi cation of the emotional states of a speaker from speech. These emotional states can either be discrete experiences, such as the basic emotions (Anger, Disgust, Fear, Joy, Sadness, and Surprise), or attributes of emotional states (Arousal, Valence, Dominance). The ability to accurately classify these emotional states through SER enables the provision of tailored services that can adapt to the psychological needs of user groups. SER attempts to classify these emotional states via verbal and non-verbal components of speech. Verbal components of speech are the speci c words a speaker says.

Verbal SER involves classifying emotional expressions from the word choice or associations, syntax, use of colloquialisms or sarcasm concerning the intent of the speaker [ 13 ]. Non-verbal components of speech refer to how the speaker expresses their speech. Non-verbal SER involves classifying emotional expressions from the ? This publication has emanated from research supported in part by a Grant from

Science Foundation Ireland under Grant number 18/CRT/6222 Copyright 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) speaker's acoustic features (Pitch, Tone, Timbre, Number of pauses, Loudness, Speech Rate) while speaking [ 14 ].

Research has shown that non-verbal components provide valuable information, distinct from verbal content, in accurately classifying emotional expressions.

For example, non-verbal features are capable of indicating emotional signals when the verbal content of the speech is neutral [ 29 ] [ 25 ]. Many non-verbal features have been shown to exist across multiple languages, enabling generalisable analysis [ 25 ] [ 27 ]. Additionally, non-verbal features have been shown to have a direct impact on emotion recognition processes in the brain, indicating they are vital to achieving accurate emotion classi cation [ 28 ]. Overall, the larger amount of features available for non-verbal SER, in comparison to verbal SER, enables more accurate emotion classi cation.

Traditional non-verbal SER1 approaches recruit human observers, who are asked to classify emotions from non-verbal content collected from one or several speakers. While traditional approaches are useful for relatively small-scale datasets, as they demonstrate high levels of accuracy [ 26 ] [ 27 ] [ 4 ], they are impractical for dealing with large and continuously growing datasets that characterise modern human-computer interactions. In an attempt to scale the bene ts of SER, research and industrial applications have been developed to conduct SER automatically. These applications range from open-source research tools (for example, Audeering ) to commercial tools (for example, Good Vibrations, Vokaturi, and deep a ect API ) [ 11 ] [ 9 ], all of which employ machine learning-based approaches. If SER can be accurately automated, it would facilitate increased e cacy in human-computer interactions, user modelling, and personalisation services.

Despite the potential bene ts of automated SER, it has not been widely implemented in everyday life settings [ 1 ]. This is despite reviews of the SER literature demonstrating that automated SER approaches perform similarly or even better than traditional approaches [ 1 ]. One reason for the limited uptake of automated SER approaches is that they are not as generalisable and adaptable as human classi ers, who can use contextual information (for example, the gender of the speaker and the language of the speech) when classifying speech. These two factors, gender and language, have been shown to signi cantly a ect the accuracy of automated SER approaches [ 17 ] [ 23 ].

SER models that account for gender di erences are more accurate than those that do not [ 31 ]. It is di cult, however, to accurately model for these di erences.

While there exists many features of speech that are signi cantly in uenced by gender, there exists considerable overlap between males and females on these same features. This overlap makes it di cult to distinguish between the gender and linguistic features of emotions that reliably indicate emotions and those that do not. Concerning language, SER models trained on one language signi cantly decrease in performance when tested on another language. This contrasts to human observers in traditional SER, who are capable of detecting emotional expressions in speech in languages they do not speak [ 4 ] [ 26 ] [ 27 ]. 1 Hereonafter, non-verbal SER is referred to as SER as shorthand.

One potential mechanism for improving the accuracy of SER across di erent gender and linguistic populations are spectral features. Spectral features of audio are computed by converting a time-based signal into the frequency domain. These features represent the distribution of energy across a frequency and the harmonic components of sound. Components such as pitch changes in the audio signal, can assist in gender di erentiation, and voice quality, can be identi ed through voice level as either tense, harsh, breathy, or lax [ 12 ]. Aggregating spectral features' continuous, qualitative, and spectral data aids di erentiation between genders and linguistic populations thereby enabling more generalisable and accurate emotive expression classi cations.

This study assesses the e ect of spectral features on the performance of automatic SER across di erent gender and linguistic populations by (i) conducting a comparative analysis of spectral features to identify a suitable feature set, (ii) comparing classi cation performance of SER approaches when using monolingual, multi-lingual, and cross-lingual emotive data, and (iii) empirically evaluating the performance of a machine learning (ML) model to assess the e ect of gender and linguistic variations on SER classi cation accuracy. 2

Related Work

Previous research has examined the di erences between gender populations in acoustic features and their e ect on SER performance. Several acoustic features have been described as \gender-dependent". Gender dependent in this case means that gender signi cantly in uences the expression of that feature [ 30 ]. Pitch is an example of a gender-dependent feature. Females score higher, on average, than males on pitch [ 16 ]. This information, however, is not su cient to enable generalisable SER classi cation, because there is considerable overlap between males and females on this feature [ 16 ]. This is also the case for several other acoustic features described as \gender-dependent".

Similarly, previous research has examined the impact of language in developing accurate SER models. Human observers can accurately classify emotions in their native language, and non-native languages [ 4 ]. This result has been replicated across various cultures, including indigenous tribes, demonstrating the generalisability of traditional SER [ 26 ] [ 27 ]. Similarly, automated SER models achieve a high level of accuracy in mono-lingual settings, where the models are trained and evaluated in the same language. The performance of these models, however, signi cantly drop when they are tested across languages [ 10 ] [ 21 ].

If ML models can di erentiate between acoustic features that indicate emotional expressions for each gender and linguistic population reliably, then one should expect generalisability of SER performance between both populations [ 31 ]. To achieve this, spectral features that reliably indicate emotional experiences are rst required.

A survey [ 8 ] analysed features across 17 distinctive datasets. These datasets comprise 9 languages, both male and female speakers, professional and nonprofessional acting sessions, recorded call centre conversations and speech recorded under simulated and naturally occurring stress situations.

A pool of acoustic features and SER were analysed in [ 8 ] and [ 1 ]. Which can be divided into several categories (Continuous, Qualitative, Spectral and Nonlinear Teager Energy Operated (TEO). The results of both reviews indicated that Continuous and Spectral features were related to emotional expressions in speech. A weaker relationship was found between emotional expressions and Qualitative and TEO features.

To automatically classify relevant spectral features, a ML methodology is proposed in [ 15 ]. The methodology uses a convolutional neural network (CNN) and SER analysis. The approach consists of using spectral features such as Mel-frequency Cepstral Coe cients (MFCCs), Mel-scaled spectrogram, Chromagram, Spectral contrast and Tonnetz representation extracted from emotive audio belonging to three distinct datasets. This work, however, focused on a combined gender classi cation of emotive expression, inhibits the analysis of vocal variations between gender. This study extends this work to by analysing the effect of gender and linguistic populations on the emotive expression classi cation of spectral features.

3 Methodology

To assess the e ect of spectral features for emotive expression classi cation across di erent gender and linguistic populations a series of experiments were conducted. Which required the identi cation of a suitable set of gender and/or language-dependent spectral features. The classi cation was enabled through the usage of the CNN described in 3.5, the performance of this CNN will be compared across experiments that account for linguistic and gender di erences and experiments that do not account for these di erences. The gender populations are male and female and the linguistic populations are English, German, and Italian.

Discrete emotional models identify several distinct emotions to be classi ed. This study uses Basic Emotion Theory (BET), as the discrete emotional model for SER. In BET the distinct emotions are Anger, Disgust, Fear, Joy, Sadness, and Surprise. These six emotions are considered basic as (i) they consistently correlate with psychological, behavioural, and neurophysiological activity [ 7 ], which makes their objective measurement possible, and (ii) they interact to form more cognitively and culturally mediated emotions, such as shame or guilt [ 7 ]. The BET contrasts from dimensional models of emotions that focus on attributes of emotions (for example, arousal, valence). BET models have been extensively used in SER research as they capture a wider range of emotions and are intuitive to label in comparison to dimensional models [ 1 ]. 3.1

Emotive Speech Data

Three distinct emotive audio datasets were used (see Table 1) to enable a comparative analysis of variations between the data. These variations can originate from many factors such as the gathering method, emotions exhibited, language, speaker gender, or sampling rate.

RAVDESS contains emotional speech data constituting statements and songs in English. For the proposed work, only speech samples are considered.

Datasets

RAVDESS[ 18 ] EMO-DB[ 3 ] Participants uttered two neutral statements across multiple trials. For each trial, participants were asked to utter the statement in a manner that conveyed one of the six basic emotions. The statements were controlled to ensure equal levels of syllables, word frequency, and familiarity to the speaker.

EMO-DB contains emotional speech data constituting statements in German. Participants uttered ten statements, comprised of everyday language and syntactic structure, in various lengths to simulate natural speech. Each utterance was evaluated with regards to recognisability and naturalness of emotions exhibited. The emotive reaction of surprise is not considered.

EMOVO contains emotional speech data constituting statements in Italian. The participants uttered fourteen distinct semantically neutral statements. Each conveys a basic emotion and is spoken naturally. An important consideration in the creation of this dataset was the presence of all phonemes of the Italian language and a balanced presence of voiced and unvoiced consonants. 3.2

Feature Selection

In order to identify spectral features indicative of emotive expression, feature selection is performed. Which identi es information such as pitch, energy, voice levels and energy distribution for classi cation. Spectral features were extracted, using the audio analysis library Librosa [ 19 ].

These features are as follows: Mel-Frequency Cepstral Coe cients (MFCC) which represents the shape of a signal spectrum, and is achieved by collectively representing a Mel-frequency cepstrum per frame. Chroma Energy Normalized (CENS) which represents statistics indicative of normalised values quantifying tempo, articulation, and pitch deviations. Zero-Crossing Rate (ZCR) which represents the rate of change in the audio signal from positive to negative or negative to positive, through zero. Chromagram which represents a transformed signal spectrum built using the chromatic scale capturing the harmonic and melodic characteristics. Melspectrogram which represents a spectrogram where the frequency is converted from a linear scale to a Mel-scale, resulting in a spectrogram re ective of how humans perceive frequencies, capturing the amplitude of the signal. Spectral Contrast which represents the energy contrast computed by comparing the peak energy and valley energy in each band converted from spectrogram frames. Tonnetz which represents the tonal features in a 6-dimensional format, capturing traditional harmonic relationships. (perfect fth, minor third and major third).

In order to identify a set of spectral features that captures su cient emotive data, a classi er is trained on these features individually before being trained on permutations of the features across the datasets to identify the best performing feature set.

3.3 Linguistic Variation

To evaluate the accuracy of a SER model in relation to linguistic variation three experiments are conducted across three languages (English, German, Italian) as follows: 1. Mono-Lingual - The CNN is used to classify spectral features indicative of emotive expression from each language independently. The results will be used as a baseline performance of the model's classi cation performance in a single language. 2. Multi-Lingual - This experiment consists of three permutations. For each, the CNN is rst trained on one of the chosen languages and then evaluated for SER accuracy against the remaining languages. The intent is to identify spectral features capacity to classify emotive expression independent of language. 3. Cross-Lingual - The CNN is trained on an aggregation of all training data.

This will provide insights into the performance of a CNN model when its training corpus contains multiple languages. The intent is to identify the generalisability of the chosen spectral features for emotive expression classication.

3.4 Cross-Gender Emotion

To assess the e ect of gender populations on SER, an experiment was conducted using the spectral features identi ed in Section. 3.2, comprising of two steps. In step one, the data is classi ed using the six basic emotions. This initial classi cation provided a baseline accuracy where gender is not speci ed. In step two, the same dataset is classi ed with gender-emotion labels (for example, Male-Anger/Female-Anger; Male-Joy/Female-Joy). The intent is to evaluate if the CNN can identify gender-dependent acoustic elements from the spectral features.

3.5 Classi cation Of Emotive Speech

The architecture of the CNN used for classi cation is depicted in Figure 1. A CNN was re-implemented, due to the high emotive expression classi cation performance using spectral features as exhibited in [ 6 ]. Optimal hyper-parameters were identi ed from a comparative analysis against related approaches. The model is trained on the extracted spectral features over 150 epochs using a batch size of 16 and 5-Fold validation is performed. During each epoch a portion of data is used to evaluate the model. The model performance is evaluated across the metrics precision, recall, F1-score for each of the six basic emotions.

4 Results

Feature extraction - The comparative analysis of the spectral features in isolation provides insights into the performance on a per feature basis. MFCC, Melspectrogram and Spectral Contrast were the highest performing individual spectral features across the datasets. When combined these features formed a vector of 155 data-points, and the highest performing permutation of spectral features as denoted in Table. 2

Linguistic variation - The results for emotive analysis in mono, multi and cross-lingual data are denoted in Table. 3, these highlight the considerations for SER across multiple languages. Mono-lingual and Cross-lingual approaches achieve high accuracies. A degradation in performance is experienced from multilingual approaches.

Cross-Gender Emotion - Table. 4 identi es discrepancies in the emotive classi cation performance, across both gender and language. This indicates a degree of vocal variance stemming from the population di erences, highlighting considerations for gender-speci c SER approaches.

5 Discussion

Feature extraction - There are several notable results from the feature extraction stage that are worth discussion. Firstly, the results of the comparative analysis showed that the MFCC feature enabled the highest emotion recognition accuracy within each dataset. Demonstrating the importance of modelling for phonetic properties, found within the speech signal shape, in enabling accurate emotion classi cation across languages. Secondly, the performance of the Melspectrogram feature varied across the datasets. Melspectrogram performed highly on EMO-DB and EMOVO and poorly on RAVDESS. This inconsistency was caused by the signi cantly lower amplitude within the audio data of RAVDESS compared to EMO-DB and EMOVO denoted in Table. 2. Participants in the RAVDESS dataset were given instructions to exhibit emotions

Spectral Features MFCC Melspectogram Spectral Contrast CENS STFT ZCR Tonnetz Spectral Feature Permutations MFCC, Melspectogram, Spectral Contrast Spectral Contrast, Melspectogram, MFCC MFCC, STFT, Spectral Contrast STFT, Melspectogram, Spectral Contrast, Tonnetz Additional Characteristics Average Absoulte Amplitude (dB) 0.69 0.44 0.39 0.31 0.22 0.23 0.22 RAVDESS EMO-DB EMOVO Mean NA with varying intensity normal and strong, additionally, post-processing procedures are likely the cause for the lower level of amplitude. This has signi cant consequence for classi cation when amplitude is utilised as a measure of emotive expression. Thirdly, Spectral Contrast in isolation provides greater average accuracy than the remaining individual features. Therefore, the contrast between peak and valley energy is partially indicative of emotion in emotive speech. The high performance of these features in isolation indicates their potential suitability for emotive expression classi cation. Finally, the results showed that combining these three features increased accuracies for each data set. This demonstrates the importance of including such spectral features in SER classi cations.

Linguistic variation - SER accuracy decreased signi cantly between the di erent types of analysis. The high performance on mono-lingual analysis indicates the importance of the selected spectral features for each language. The poor performance in multi-lingual analysis, however, indicates a lack of universality between spectral representations of emotions across the datasets. For example, there was signi cant variation in the amplitude and pitch range across the languages, as identi ed in [ 20 ]. Additionally, these di erences likely stemmed from di erences in data collection (equipment, volume, actor-microphone distance) across the datasets, thereby decreasing the performance of the CNN in multi and cross-lingual analysis. This highlights the need for standardised recording lassi cation accuracy per language derived from 5-fold cross-validation, and train/test data speci ed. (X represents data from the corresponding column name) Method Mono-Ling.

Train/Test

X/X Multi-Ling.-Eng. RAVDESS/X Multi-Ling.-Ger. EMO-DB/X Multi-Ling.-Ital. EMOVO/X Cross-Ling.

All/All

English

German

Italian NA NA

NA NA NA NA

NA procedures within SER research, and/or additional procedures to normalize linguistic variances.

Cross-Gender Emotion - There were substantial discrepancies in spectral representations between the gender populations. As a result, SER performance between the genders di ered signi cantly across the six basic emotions. These di erences were varied across languages, indicating that there is a generalisable e ect of gender on SER performance when using spectral features, however this is impacted by linguistic variation. For example, high energy emotions (such as Anger and Joy) were more clearly detected in German-speaking males than in other linguistic or gender groups. The German language is characterised by higher amplitude, energy and harsher speech when articulated within males [ 20 ]. This likely contributed to higher accuracy of detecting those emotions from German-speaking males. Disgust and Fear, in contrast, were more accurately classi ed from the speech of females across each language. This may have resulted from di erences in vocals, particularly pitch, between the gender groups [ 22 ]. Additionally, female voices tend to articulate in a softer manner [ 24 ] conducive to representing softer, lower amplitude emotions.

These results likely contributed to the weaker performance of combined gender classi cation in comparison to gender-speci c classi cations of emotion. Combined approaches can account for variations between genders and languages, however, in certain cases, a speci c gender may act as a limiting factor reducing the overall accuracy.

Limitations - The major limitations of this work concern sampling issues. Firstly, the sample size across the three datasets is small (N = 40). Since sample sizes can exaggerate the variances between populations, then the small sample size in this study may have exaggerated real di erences in spectral features between gender and linguistic populations. Secondly, the datasets were comparing linguistic populations across an unequal number of speakers. Di erent sample sizes will have higher or lower ranges of variability. Comparing di erent sample sizes makes it di cult to determine whether the results stem from real betweengroup di erences or are the result of a higher level of \noisy" data found in small sample sizes. These limitations damage the generalisability of the work. In a follow-up study, the sample size of the overall dataset will be increased and the number of speakers per linguistic group will be controlled.

6 Conclusion

This work was an exploratory analysis of linguistic and gender variation on the emotive classi cation of spectral features. The results showed that features using the Mel-Scale and representations of amplitude and energy are important in accurate SER across di erent gender and linguistic populations. Additionally, the results showed that higher energy emotions such as Anger, Joy, Surprise were easier to identify originating from male voices in a high amplitude, harsh language such as German. Similarly lower energy emotions such as Disgust and Fear were easier to identify from female voices in each language. These observations highlight the importance of signal amplitude and energy when analysing emotion across gender and language.

The performance of emotive expression classi cation across mono, multi and cross-lingual data provides insights into linguistic variation of emotive audio. Mono-lingual approaches are suitable as baselines in comparison to multi and cross-lingual approaches. A linguistic variance between emotive expression represented by spectral features can be identi ed. To overcome these variances, it is recommended that cross-linguistic approaches, that combine languages for training, are implemented.

Future work should explore whether classi cation accuracy is a ected by (i) model optimization in terms of structure and/or (ii) comprehensive hyperparameter tuning (epochs, batch size, optimizers) using a grid search technique. Additionally, the relationships between linguistics, gender and emotion should be explored from a vocal, psychological and technical classi cation perspective, with a larger sample size to garner further insights into improving SER generalisability.

References

1. Akcay , M.B. , Oguz , K. : Speech emotion recognition: Emotional models, databases, features, preprocessing methods, supporting modalities, and classi ers . Speech Communication 116 , 56 { 76 ( 2020 ). https://doi.org/10.1016/j.specom. 2019 . 12 .001

2. Ancilin , J. , Milton , A. : Improved speech emotion recognition with Mel frequency magnitude coe cient . Applied Acoustics 179 , 108046 ( 2021 ). https://doi.org//10.1016/j.apacoust. 2021 .108046

3. Burkhardt , F. , Paeschke , A. , Rolfes , M. , Sendlmeier , W.F. , Weiss , B. : A database of German emotional speech . In: 9th ECSCT ( 2005 )

4. Cordaro , D.T. , Keltner , D. , Tshering , S. , Wangchuk , D. , Flynn , L.M.: The voice conveys emotion in ten globalized cultures and one remote village in Bhutan . Emotion 16 ( 1 ), 117 ( 2016 )

5. Costantini , G. , Iadarola , I. , Paoloni , A. , Todisco , M.: EMOVO corpus: an Italian emotional speech database . LREC p. 4 ( 2014 )

6. Dair , Z. , Donovan , R. , O 'Reilly , R. : Classi cation of emotive expression using verbal and non verbal components of speech . In: 2021 ISSC . pp. 1 { 8 ( 2021 ). https://doi.org/10.1109/ISSC52156. 2021 .9467869

7. Ekman , P. : Emotions revealed . Bmj 328 ( Suppl S5 ) ( 2004 )

8. El Ayadi , M. , Kamel , M.S. , Karray , F. : Survey on speech emotion recognition: Features, classi cation schemes, and databases . Pattern Recongnition 44 ( 3 ), 572 { 587 ( 2011 ). https://doi.org/10.1016/j.patcog. 2010 . 09 .020

9. Eyben , F. , Huber , B. , Marchi , E. , Schuller , D. , Schuller , B. : Real-time robust recognition of speakers' emotions and characteristics on mobile platforms . In: 2015 ACII . pp. 778 { 780 . IEEE ( 2015 )

10. Feraru , S.M. , Schuller , D. , et al.: Cross-language acoustic emotion recognition: An overview and some tendencies . In: 2015 International Conference on A ective Computing and Intelligent Interaction (ACII) . pp. 125 { 131 . IEEE ( 2015 )

11. Garcia-Garcia , J.M. , Penichet , V.M. , Lozano , M.D.: Emotion detection: a technology review . In: Proceedings of the XVIII international conference on human computer interaction . pp. 1 { 8 ( 2017 )

12. Gobl , C. , Chasaide , A. : The role of voice quality in communicating emotion, mood and attitude . Speech Communication 40 , 189 { 212 (04 2003 ). https://doi.org/10.1016/S0167- 6393 ( 02 ) 00082 - 1

13. Goddard , C. : Semantic Analysis: A Practical Introduction . Oxford University Press ( 2011 )

14. Hsu , J.H. , Su , M.H. , Wu , C.H. , Chen , Y.H. : Speech emotion recognition considering nonverbal vocalization in a ective conversations . IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 , 1675 { 1686 ( 2021 )

15. Issa , D. , Fatih Demirci , M. , Yazici , A. : Speech emotion recognition with deep convolutional neural networks . BSPC 59 , 101894 ( 2020 ). https://doi.org/10.1016/j.bspc. 2020 .101894

16. Latinus , M. , Taylor , M.J.: Discriminating male and female voices: di erentiating pitch and gender . Brain topography 25(2) , 194 { 204 ( 2012 )

17. Lausen , A. , Schacht , A. : Gender di erences in the recognition of vocal emotions . Frontiers in psychology 9 , 882 ( 2018 )

18. Livingstone , S.R. , Russo , F.A. : The Ryerson audio-visual database of emotional speech and song (RAVDESS) . PloS one 13(5) , e0196391 ( 2018 )

19. Brian McFee , Colin Ra el , Dawen Liang, Daniel P.W. Ellis, Matt McVicar , Eric Battenberg , Oriol Nieto: Librosa: Audio and Music Signal Analysis in Python . In: Proceedings of the 14th Python in Science Conference . pp. 18 { 24 ( 2015 ). https://doi.org/10.25080/Majora-7b98e3ed-003

20. Mennen , I. , Schae er, F., Docherty , G.: Pitching it di erently : a comparison of the pitch ranges of German and English speakers . 16th ICPS pp. 1769 { 1772 (08 2007 ). https://doi.org/20.500.12289/42

21. Neumann , M. , et al.: Cross-lingual and multilingual speech emotion recognition on English and French . In: 2018 ICASSP. pp. 5769 { 5773 . IEEE ( 2018 )

22. Pepiot , E.: Voice, speech and gender:: Male-female acoustic di erences and cross-language variation in English and French speakers . corela ( 2015 ). https://doi.org/10.4000/corela.3783

23. Rajoo , R. , Aun , C.C. : In uences of languages in speech emotion recognition: A comparative study using malay, english and mandarin languages . In: 2016 ISCAIE . pp. 35 { 39 . IEEE ( 2016 )

24. Sandier , B.R. : Women faculty at work in the classroom, or, why it still hurts to be a woman in labor . Communication Education 40 ( 1 ), 6 { 15 ( 1991 ). https://doi.org/10.1080/03634529109378821

25. Sauter , D.A. , Eisner , F. , Calder , A.J. , Scott , S.K. : Perceptual cues in nonverbal vocal expressions of emotion . Quarterly Journal of Experimental Psychology 63 ( 11 ), 2251 { 2272 ( 2010 )

26. Sauter , D.A. , Eisner , F. , Ekman , P. , Scott , S.K. : Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations . Proceedings of the National Academy of Sciences 107 ( 6 ), 2408 { 2412 ( 2010 ), publisher: National Acad Sciences

27. Sauter , D.A. , Eisner , F. , Ekman , P. , Scott , S.K. : Emotional vocalizations are recognized across cultures regardless of the valence of distractors . Psychological science 26(3) , 354 { 356 ( 2015 )

28. Schirmer , A. , Gunter , T.C.: Temporal signatures of processing voiceness and emotion in sound . Social cognitive and a ective neuroscience 12(6) , 902 { 909 ( 2017 )

29. Soleymani , M. , Garcia , D. , Jou , B. , Schuller , B. , Chang , S.F. , Pantic , M.: A survey of multimodal sentiment analysis . Image and Vision Computing 65 , 3 { 14 ( 2017 )

30. Vogt , T. , Andre , E.: Improving automatic emotion recognition from speech via gender di erentiaion . In: LREC . pp. 1123 { 1126 ( 2006 )

31. Xu , Z. , Meyer, P., Fingscheidt , T. : On the e ects of speaker gender in emotion recognition training data . In: Speech Communication; 13th ITG-Symposium . pp. 1 { 5 . VDE ( 2018 )