Voice mimicry is the act in which imitators reproduce the vocal characteristics of another person. It can be considered to be an attack to a speaker recognition system. This work evaluates a speaker identification system under mimicry attacks: the goal is to point out how the accuracy of the system changes depending on the various real scenarios could occur. For this purpose, a GMM-UBM model and an I-Vector have been implemented and tested over dataset of Italian language imitations. Tests have been performed different audio lengths and different use cases. Use cases also take into consideration some possible countermeasures.
Voice has always been one of the most widely used biometrics as a distinctive and measurable
feature for the recognition of users in terms of biometric security [
An important topic that is crucial in these years is about the security of the biometric systems, indeed
these systems can be prone to various attacks [
2020 Copyright for this paper by its authors.
This work is focused on the task of speaker identification with a text-independent approach. More specifically, this study focuses on voice mimicry attacks to analyze the vulnerability of speaker identification systems, depending on the various scenarios that may arise, whether it is under attack or not.
The main contributions of this study are: • Identify and test a set or reals attack scenarios and possible countermeasures. • Compare two state-of-the-art speaker identification systems: a Gaussian Mixture Model Universal Background Model (GMM-UBM) recognizer and an I-vector recognizer. • The creation of an Italian language Imitation dataset for the aim of the study.
The paper is divided into the following sections. The section 2 describes the previous state-of-theart research in the field of speaker identification and mimicry attacks. The section 3 describes the methods and the approach used in this work. The section 4 explains the dataset and the experiments that have been performed and finally in the section 5 and 6 will be discussed the result obtained during the testing phase and the conclusion.
The amount of related works is not extensive highlighting that this is a relevant emerging topic. Yee
Wah Lau et al. [
More recent studies have focused on two widely used speaker recognition systems, one based on a
Gaussian Mixture Model - Universal Background Model (GMM-UBM) [
Other interesting research, [
Vestman et al. proposed another type of work based on impersonation slightly different from the
previous. They used a two ASV system one publicly available based on I-vector and PLDA, and one
closed source ASV system based on x-Vector. The aim of this research is to perform a similarity search,
in a speech corpus, between recruited attackers and potential target speaker with the first ASV system,
then test the impersonators and the most similar voice to target speaker, founded by the first system, on
the second ASV system. The research highlights the impersonators don’t affect the performance of the
ASV system, but an ASV system that attacks another ASV system can be potentially dangerous [
on the I-Vector model.
In this work two systems have been tested. The first is based on GMM-UBM models and the second Gaussian Mixture Model – Universal Background Model
Systems based on Gaussian Mixture Model – Universal Background Model are widely used in the
field of speaker recognition due to their easily implementation, the low computational cost compared
to the other technics, and the excellent result that can be achieved [
A Gaussian Mixture Model (GMM) is a “parametric probability density function represented as a
weighted sum of Gaussian component densities” [
After the generation of the GMM target speaker models the system can recognize “target speaker” or “ubm” in case of “no target speaker” this because the UBM model can acts like an impostor hypothesis model.
Finally, the verification score of the system is the log-likelihood ration between the test utterance
generated from the speaker and that generated by the UBM [
I-Vector
Another widely used approach is based on the identity vector (I-Vector). At the base of this approach there is the idea that the MAP adaptation performed only on the mean vectors will result in a supervector of concatenated means.
Given m the super-vector of UBM means, T a low-rank matrix that defines the total variability space and ϕ a standard distribution vector. The super-vector M of the segment GMM can be calculated adapting the means of the UBM, so M can be written as: = + (1)
The standard distribution ϕ is used as the extracted i-vector, while the T-matrix is calculated with an
expectation-maximization (EM) algorithm from a development dataset. Usually on the i-vector can be
applied a post-processing algorithm like radial Gaussianization, in this way the i-vector can better
follow the Gaussian assumptions used in the UBM model. Finally for measuring the similarity between
two utterances represented by their corresponding i- vectors can be used the cosine similarity [
These two systems have been selected because of their large use in many on-the-shelf application
[
The audios have been all converted to .Wav format, re-sampled to 8 kHz and switched from stereo
to mono channel [
First, a pre-emphasis filter has been applied to the signal to enhance the high frequencies of the spectrum, reduced by the speech production process,
The signal have been divided into successive 25-millisecond frames with 10-millisecond
overlaps by frame blocking and hamming windowing. This process is intended to minimize frequency
discontinuity. Since the speech signal varies slowly over time or is a quasi-stationary signal, speech must
be examined over a sufficiently short period [
A Voice Activity Detection (VAD) filter have been applied: this allowed the removal of all
superfluous parts (in most cases silence and/or background noise) from the signal, selecting only those
discriminating components and allowing the correct speaker to be identified,
4. A RASTA (Relative Spectral Filtering) filter have been applied to eliminate frequencies that
are different from the normal change in the voice signal, such as frequencies affected by background
noise recorded together with the voice signal,
5. Finally, the MFCC features have been extracted. In detail, for each audio file, a feature vector
composed of 20 MFCC coefficients have been extracted with a filter bank composed of 26 triangular
filters. The choice of using 20 coefficients was made after some considerations: withmore coefficients,
the performance worsens because they would represent rapid changes in signal energy, not
representative of the individual's vocal characteristics. On the other hand, fewer would not have enough
information to represent the voice adequately. In addition, 20 MFCC delta-features were also computed
from the 20 MFCC coefficients for a vector size of40 values. During the feature extraction phase, it
was also chosen to replace the first value of the vector, the cepstral coefficient at position zero, usually
with a null value, with the log of the entire energy component [
Two different approaches have been used for training depending on the specific system.
Regarding the GMM-UBM, Model Adaptation was chosen adapting a previously pre-trained UBM of the Italian language to the speaker's feature vector. The adaptation has been performed using the Maximum a Posteriori (MAP) algorithm, which as the name may suggest, will maximize the a posteriori probability that, given a recording, the correct speaker is selected. It consists of a first phase in which the feature vectors are mapped with probabilistic ratios, inserting them into the UBM mixtures, in this case, 512. Then, the mixtures are adapted using the new data, referring to the "relevance factor ", which quantifies the new data to be analyzed in a single mixture to balance the contribution made by the latter to perform the adaptation. Finally, the testing phase is characterized by matching the feature vector extracted from the system and the model of a speaker, calculating the LLR (Log-Likelihood Ratio).
Concerning the i-vector system, the UBM model has been used to calculate the Total Variability Matrix or TV Matrix, which represents a matrix in which the i-vectors will be extracted, then it will use a function with rank equal to 400 and number of iterations fixed at 20 to realize the actual TV Matrix. The next step will proceed to obtain the statistics for each type of audio, specifically test or training. Once the statistics are obtained, it will proceed with the actual extraction of the i-vector for each speaker and for each test segment, thus obtaining a unique and discriminating model. The resulting supervectors with reduced dimensions are then the i-vectors, representing the result of the mapping carried out in the first phases. In the last instance, it will calculate the cosine similarity or the cosine of the angle between the adjacent vectors that are compared between the vector representing the speaker enrollment, then the model, and the vector representing the speaker test.
The speech material used in this work consists of 22 Italian celebrities from the world of politics and entertainment. The audios were extracted from interviews, shows, or performances available on online platforms. These 22 identities represent the genuine set of users which will be attacked by imitators. In addition, 17 imitators were chosen and their original voices as well as imitations speech of the 22 genuine users were collected. Finally, there are 24 impersonators representing attacks in the system. As can be guessed, the "imitation" relationship between famous individuals and imitator is n-to-n since multiple imitators can imitate each famous individual, and one imitator can imitate multiple famous individuals. For each speaker, both genuine and impostors, one audio of the duration of 5 minutes have been extracted, while for the audios of the imitations, their duration varies from 40 seconds to 5 minutes. 22 users
In addition to the dataset created, it has been used another dataset for the UBM model training as
already described in the previous section, in detail is an extensive Italian repository, the Common Voice
Corpus 6.1, consisting of 5729 voices, female, male and uncleared identity (54% male and 14% female
and 23% uncleared identity) at different ages (from 18 to 79 years), 158 hours of speech and about
80.000 files [
The model has been trained on the 25% of the entire dataset this because the training phase of the UBM model has taken a large amount of time (more than 10 hours).
For the testing phase, audio files were chunked into 1-second and 5-seconds files to analyze if and how, the system behaves by varying the length of the audio.
Tests have been evaluated in terms of precision, recall, and F1 score.
Different use cases have been considered as reported in Table 2. Cases 1, 3, 6, and 7 represent situations in which the system is tested only considering speakers of which it is aware. Cases 2 represent the situation in which the imitator (unknown to the system) performs a mimicry attack. Case 4 is referred to the situation in which the imitator is known to the system as a genuine user, but he/she performs a mimicry attack on another genuine user. Case 5 refers to situations in which the imitator act as a genuine user as well as an impostor.
The last two cases are referred to the possibility to add imitation models.
Precision 94,39% 70,87% 92,09% 79,72% 76,90% 97,15% 77,56% Recall 89,52% 86,69% 86,39% 85,65% 84,70% 87,45% 73,18%
system under mimicry attack
system including imitators which does not act as imitators at
testing
system including imitators under mimicry attack The baseline system including imitators which act as imitators (mimicry attack) and genuine users at testing
explicitly aware of imitations
F1 Score 91,53% 76,79% 88,37% 80,91% 79,43% 91,78% 77,88%
Use case 1 2 3 4 5 6 7
system under mimicry attack
system including imitators which does not act as imitators at
testing
system including imitators under mimicry attack The baseline system including imitators which act as imitators (mimicry attack) and genuine users at testing
explicitly aware of imitations
F1 Score 99,42% 84,86% 99,37% 88,84% 89,10% 99,53% 88,29%
Precision 99,70% 80,50% 99,71% 86,48% 85,46% 99,60% 95,65%
The baseline system under attack shows a performance degradation of 15% in both tests (1s and 5s) if the impostor is an outsider (use case 2). A performance degradation of 8% is observed if the impostor is an insider (use case 4 vs use case 3). The situation in which the system is trained on some possible mimicry attack (use case 5) does not differ significantly in performance from the previous case.
Considering the tests performed at 1 second and at 5 seconds, it can be seen that the length of the audio segment affects the performance of the system, in general it can be stated that a wider duration strongly decreases successful attacks. However, the general performance trend along the different use cases is independent by the audio duration. The performance worsens considerably (cases 2, 5, 7) when the system is attacked and does not know speakers or knows only partially imitations (cases 6 and 7) or imitators (cases 3,4,5).
system under mimicry attack
system including imitators which does not act as imitators at
testing
system including imitators under mimicry attack The baseline system including imitators which act as imitators (mimicry attack) and genuine users at testing
explicitly aware of imitations
system under mimicry attack
system including imitators which does not act as imitators at
testing
imitators under mimicry attack The baseline system including imitators which act as imitators (mimicry attack) and genuine users at testing
explicitly aware of imitations
The system explicitly aware of imitations under mimicry attack 87.08%
Considering the result achieved with the i-vector system, it can be seen that the baseline system under attack shows a performance degradation of 30% in 1s test and 25% in 5s test if the impostor is an outsider (use case 2). A performance degradation of 14% in both tests (1s and 5s) if the impostor is an insider (use case 4 vs use case 3). The situation in which the system is trained on some possible mimicry attack (use case 5) does not differ significantly in performance from the previous case.
Finally, comparing the I-vector results, the same trend obtained in the GMM-UBM model in the various use cases is also reflected in this model. However, the I-vector system suffer from attacks of a slightly higher performance degradation.
In this work, a study of the vulnerability of speaker identification systems against voice mimicry attacks is presented. The study conducted presents a new Italian mimicking dataset, two very widespread models are implemented and tested on this dataset: the GMM-UBM, and I-vector.
The result obtained using the GMM-UBM system show that the baseline system under attack has a considerable degradation of the performance especially if the system doesn’t know or partially know the imitators or the imitations. The same trend has be observed for the I-vector system, even if with slightly increased degradation respect to the GMM-UBM. Comparing the I-vector results, the same trend obtained in the GMM-UBM model in the various use cases is also reflected in this model. However, the I-vector system suffer from attacks of a slightly higher performance degradation.
Performance degradation also depends upon the fact if impostor is another genuine user known by the system or not.
Concerning the length of the audio, longer audio files are able to report higher performances then shorter ones, however, the general performance degradation trend along the different use cases is independent by the audio duration.
In future studies, it will be possible to extends this work with other state-of-the-art technics like artificial neural networks (ANN), extend the Italian mimicry dataset with additional users and compare the result between GMM-UBM, I-vector and ANN.
This work has been supported by the Italian Ministry of Education, University and Research within the PRIN2017 - BullyBuster project - A framework for bullying and cyberbullying action detection by computer vision and artificial intelligence methods and algorithms.