This paper presents the description of our speech synthesis system for the Audio Deep Synthesis Detection Challenge (ADD 2023). We utilizes a FastPitch-based model augmented with a BERT-based prosody feature and an utterance embedding predictor to model the generated speech. We incorporate these components to improve one-to-many generation modeling. Evaluation results indicate a significant advantage over some false speech detection models, earning a second-place ranking in the competition overall.
The proliferation of deepfake technology has under- Speaker Embedding Decoder
scored the importance of reliable methods to detect and
fparkeeveDnetttehcetimonalCichioaullsenugsee o(AfDdeDep2f0a2k3e)s.isTahedeAeupdlieoaDrneienpg- [
The competition comprises four tracks, each with a
unique focus and set of challenges. In this paper, we LengPthhoRneemguelator
describe the speech synthesis system we employed in the [
The primary challenge of Track 1.1 is the develop- Speech synthesis technology, based on deep learning,
ment of efective adversarial attacks that can generate has the capability to generate counterfeit speech that
speech that is dificult for detection models to distinguish mimics a target speaker’s voice from text and the target
from genuine audio. Deepfake audio generated using ad- speaker’s voice data[
×2
F C
(a) Overview
long pauses to align punctuation with these two types
of pauses. Redundant punctuation was tagged with no
In accordance with the requirements of Track 1.1 (gener- duration. To account for the silence mark that may
ation task), the AISHELL-3[
2.2. Pitch and Energy
linear spectrogram. Subsequently, we utilized the aligned
LDP duration to average the frame-level energy sequence
and generate the LDP-level energy sequence. The
resulting sequence was quantized to aid in subsequent
processing. In the second step, we utilized the WORLD[
Wav Output HiFiGAN Vocoder
Post
Processing
3.1. Encoder
3.2. Explicit modelings
To overcome any potential pronunciation interference
that may arise due to direct superposition of the BERT 3.3. Decoder
embedding input and the aggregated phoneme input, we The Decoder architecture, akin to the Encoder, is
comemploy a Conformer block to fuse the information. This prised of four Conformer blocks. In this approach, we
is accomplished by leveraging a linear layer to adjust the fuse the Encoder output, pitch embedding, and energy
dimension of the BERT embedding input. Analogously, embedding, and pass the resultant through two
Conthe utterance embedding is similarly inserted and pro- former blocks. Subsequently, we incorporate the speaker
cessed through a Conformer block. Ultimately, we obtain embedding obtained from the lookup table into the
arthe Encoder output by passing the input through two chitecture. Finally, we pass this through two Conformer
Conformer blocks. The Conformer block utilized in our blocks and a linear layer to obtain the mel spectrum.
approach is akin to the one described in the delightful
TTS. However, we have made certain modifications, such 3.4. Vocoder and Post-processing
as removing the Depthwise convolution and
substituting the self-attention with Relative Position Multi-head
Attention[
proach are converted into speech signals using a
universal and fine-tuned HiFi-GAN vocoder[
This trick has been demonstrated to alleviate some
highfrequency noise. The processing steps are outlined below:
During the training stage, we adopt the approach
proposed in [
that requires participants to generate adversarial sam4. Results ples and enhance the anti-attack capabilities of the audio deepfake detection model from two opposing sides. The 4.1. Experimental setup generation and detection tasks of Track 1 are evaluated separately. For the generation task (Track 1.1), the decepIn our experimental setup, we downsample the AISHELL- tion success rate (DSR) is chosen as the metric. The DSR 3 audio from 44.1 kHz to 24 kHz for the TTS model and metric quantifies the extent to which the audio deepfake to 16 kHz for all evaluations. We represent the audio detection model is deceived by the generated utterances features as a sequence of 80-dimensional log-mel spec- and is defined as follows: trogram frames. These frames are computed from 40 ms wthiendCoownsfotrhmaterarbeloschkifsteind obuyr1p0rmopso.sTehdemhoidddeelnisssizeet toof = · (2) 128. Each feed-forward layer of the Conformer blocks where W denotes the count of wrong detection samcomprises of two 1-D convolution layers, each with a ker- ples by all the detection models on the condition of nel size of 3 and 1024 intermediate channels. Regarding achieving their respective equal error rate (EER) perthe acoustic model, we utilize an l1 loss function for the formance, A is the number of evaluation samples, and N mel spectrum, while the mean-squared error (MSE) loss represents the number of detection models. In the first function is applied to other components. The duration round, the DSR against the Track 1.2 submissions constiand energy predictor compute the loss in the logarithmic tutes the overall generation performance metric. In the second round, weighted consideration is also given to the DSR against the detection model that we release. Specifically, in the second round, the generation performance metric is defined as: = +
(3) where =0.4 and =0.6, and they denote the respective weights for EER and DSR in our consideration. 4.3. Evaluations
tack samples while adhering to the specified text and speaker identities. Our team id is A02. As depicted in Figures 4(a) and 4(b), our deception success rate (DSR) ranks 7th and 8th in the first and second rounds, respectively. Notwithstanding, in Figure 4(c), which displays the DSR scores of the baseline models provided by the organizers in the second round, we secured the first position and outperformed the second-ranking system by a significant margin. Ultimately, our speech synthesis system obtained the second position in the overall scoring shown in Figure 4(d).
which has yielded promising results in the adversarial process with the fake audio detection model. Specifically, our system has demonstrated significant advantages over baseline models, thereby validating its eficacy in the task of fake audio generation.