Audio deepfakes pose significant security threats to speaker recognition systems (SRSs), particularly with the growing threat of adversarial attacks. Existing black-box attack methods mostly rely on ensembling multiple datasets and models to search for adversarial examples (AEs) with good transferability, but they ignore the limitations of such search algorithms. In this paper, we comprehensively analyze diferent iterative-based adversarial attack methods and explain diferent transferability from the perspective of optimizing the search space. Furthermore, we propose a difusion-based attack method located on the boundary (DAoB for short), which takes boundary information into consideration to achieve better transferability. Specifically, DAoB starts searching for an appropriate AE from the boundary of the search space instead of the original example, then it guides the search process by difused audio and the gradients of multiple white-box models to obtain better gradient directions. To validate the efectiveness, we conducted experiments on seven state-of-the-art SRSs and DAoB outperforms others. Remarkably, even in the black-box scenario, the attack success rate of DAoB attains an impressive 97.2%, in close proximity to the rate achieved in the white-box scenario.
incorrect output [7]. This attack can create examples that deceive deep learning models without being easily deWith the rapid development of information technolo- tected by humans. This reveals the common and serious gies, identity recognition have become more intelligent vulnerabilities of deep learning models and promotes the and convenient. Typical methods such as facial recogni- further development of various intelligent technologies. tion, fingerprint recognition, iris recognition, and speaker Among them, adversarial attacks against SRSs are more recognition can quickly perform identity verification dificult than those against images and have started later with high success ratio[1, 2]. Especially with the emer- [2]. gence of deep learning technologies, identity recogni- Commonly speaking, there are three attack scenarios: tion has made significant progress. However, Deepfake white-box, gray-box, and black-box. In the white-box technology verifies the challenge of identity recognition scenario, attackers have access to all the details of the systems’ reliability [3, 4]. Compared to image-based deep- SRSs, so they often employ gradient-based attack methfake technology, speech-based deepfake technology is ods to directly generate adversarial examples. Recent a relatively new field. For example, recently emerging studies [8, 9, 10] show that adversarial attacks can fully technologies [5, 6] such as speech recording and replay, overcome almost all white-box SRSs. In the gray-box sceText-to-Speech (TTS), and Voice Conversion (VC) can nario, attackers need to continuously query the victim deceive SRSs to a certain extent. system and obtain corresponding score vectors to opti
Various deep-learning models are shown to be vul- mize adversarial examples. For example, the FAKEBOB
nerable to adversarial attacks which add small amounts method proposed in [
next section highlights the related work in the field of adversarial attacks against speaker recognition systems. Then, we demonstrate DAoB and its theoretical analysis in Section 3. We describe the setup of the experiment in Section 4 and show the experiment results and discussions in Section 5. Finally, we conclude the paper in Section 6.
Many works on enhancing transferability [12, 13, 14]
have achieved positive results, but there are still some
unresolved or undiscovered issues. First, the development 2.1. FakeBob
of transferability for adversarial examples is still insuf- FakeBob [
In this paper, we propose a difusion attack located on applications. the boundary (called DAoB for short), which adopts the boundary information to address the above-mentioned problems. The workflow of DAoB is described in Fig. 1. 2.2. Transferable Adversarial Attack Specifically, we place the search region for adversarial Zhang et al. [13] proposed an integrated attack at the examples directly on the attack boundary, which is more logits level can be used to achieve a black-box attack likely to produce high transferability at the beginning of with a higher success rate, but they did not give a reasoneach iteration. Secondly, when calculating gradient infor- able solution to the problem of inconsistent logits ranges mation, we obtain multiple gradient information that has between diferent models in the paper. The method in been difused by taking an information difusion space [12] uses spatial momentum to calculate the gradient for near the example and taking the average of these gradi- integrated attack, that is, the gradient information in the ents as guidance for this iteration. Finally, considering previous iteration process will be used in this iteration, that the logits of diferent models have large diferences, and the use of momentum can efectively improve the we adopt gradient-level model integration for the attack. aggressiveness and transferability of adversarial exam
We summarize the paper’s main contributions as fol- ples. The STA-MDCT [14] method introduces discrete lows: cosine transform into the adversarial attack of speaker • We conduct theoretical analysis and experimental recognition, it firstly converts the audio to the frequency comparisons of existing adversarial attack meth- domain and adds random adversarial noise, and then ods, propose definitions of ideal and actual search converts it back to the time domain for model integration attack. This method improves the interpretability and transferability of the attack in the black-box envi- boundary epsilon to prevent excessive noise. For multironment, but the additional transformation increases a step attacks (using the most commonly used parameter lot of computing time, and the current models are all selection, step size = / ), as the gradient direction end-to-end input, they would not pay more attention to changes, the attacker is unable to search near the set the frequency domain part of the audio. boundaries.
For transfer-based black-box attacks, many experiments have shown that the single-step attack method 3. METHODOLOGY has certain advantages in transferability. It can also be inferred from experience that the further an adversarial 3.1. DAoB example is from the original example, i.e., the closer it As shown in the Fig. 1, we divide each iteration of DAoB is to the boundary, the greater the diference in speaker into two phases: a module for computing the gradient of characteristics between the adversarial example and the the current iteration, and another for using the gradient original example. Therefore, we believe that the transto guide the search for the adversarial example in this it- ferability of adversarial examples in the search space eration. Specifically, in the gradient computation module, near the boundary is stronger than that of adversarial we input the difused example into multiple white-box examples in the search space near the original example. models to obtain their corresponding embeddings and In summary, we propose an attack method that starts calculate the gradient accordingly. In the example update from the boundary. Based on the initial gradient informamodule, we add the gradient obtained in this iteration tion, we directly add a large amount of noise to the origito the momentum gradient. Then, we extract the direc- nal example to reach the boundary. Then, we search for tion of the momentum gradient to guide the update of adversarial examples from the boundary with a smaller the current example. Specifically, we update the original step size. example starting from the search boundary with a small step size for multiple iterations, eventually obtaining an 3.1.2. Difusion attack adversarial example with strong transferability, which can cause a black-box victim model to output incorrect results. Below, we will detail the theoretical basis and specific operations of DAoB.
of image and speech generation. The process of a single difusion step is to add noise to the example to obtain +1, while the reverse difusion is the process of using and +1 as guidance to learn the generation method.
Algorithm 1 DAoB Inspired by the difusion model, we optimize the process Require: of generating adversarial examples iteratively obtained Set of several white-box models = { | = from , by difuse it to , and then using as guidance 1, · · · , }, clean input with target labels , pa- to iterate to obtain +1. Considering that the process of rameters = {, , , , } generating adversarial examples lacks support from data Ensure: volume, we perform multiple difusions to better guide
Adversarial example ; the direction of iteration. 1: 1 ← , 0 ← 0, 0 ← ;
3: for number of sub-example ← 1 to do 4: = (, ) 5: for number of model ← 1 to do
Calculate gradient of (, ) 6: end for 7: end for 8: Update by Eq. (3-4) 9: Update by Eq. (5-7) 10: end for 11: return To improve transferability, we obtain gradients from multiple white-box models. Unlike ensemble methods based on logits level, we set the same loss function for each white-box model. We believe that the size of the gradient in any dimension reflects the influence that the gradient direction can have in that dimension. Therefore, we do not set weights for the gradients of diferent white-box models in the ensemble gradient. We simply add the gradients obtained from each white-box model and then take the direction of the ensemble gradient as the direction for the current iteration.
We introduced the details of DAoB, which is described in Algorithm 1. We formalize the specific attack process as follows: 3.1.1. Take the boundary as the starting point
= =1,· ,
∑︁ =1,· , ∇ ︁( , , ︁) where (︀ , ︀) is the derivative of the loss function with respect to the input example .
= · −1 +
‖‖1 .
inal example to reach the boundary of the search space: 1 = + · sign (1) , where is the constraint of pertubation.
Afterwards, we use the momentum gradient to generate smaller perturbation, which is then used to create the adversarial audio. This process can be formalized as follows:
= · −1 , = Clip {︁−1 + · sign (+1)}︁ , where is the attack step size, and is the attenuation factor. (4) (5) (6)
consistent with the Spk10-enroll, Spk10-test, and Spk10imposter datasets published in [16].To validate the effectiveness of our method, we selected seven strong victim models: Res34-L [17], Res34-V [17], TDy_HR [18], TDy_QR [18], TDy_VGG [18], XV-plda [19], ECAPA [20]. iteration, the audio difused into a :
The input consists of a set of several white-box models = { | = 1, · · ·
, },clean input with target labels , and a series of required parameters. In each from the previous iteration is = ( , ) = + ( , ), (1) (0, 2). where (
, ) is Gaussian noise with the same shape as and follows a normal distribution
The gradients computed using and in each iteration can be formalized as:
sarial audio, we also pay attention to the imperceptibility of adversarial audio. Therefore, we refer to [12] and use
of the imperceptibility of adversarial audio.
Speech Quality (PESQ), and 2 − as the indicators
During the process of adversarial attacks, all attacks were (2) targeted attacks in the open-set recognition scenario, and the attack targets were fixed simple targets, with the specific settings being the same as in [ 21]. The number of iterations for all iterative attack methods was 10. We conducted experiments with diferent perturbation limits, but due to space constraints, the perturbation limits shown in the experimental results below are all set to (3) 0.002. The fixed step size is set to / . In particular, there is some randomness in difusion of DAoB, so all results reported for our methods are the average of 10 repeated experiments.
We systematically studied the transferability of adversarial examples generated by DAoB and reported the experimental results in Tab. 1, where the DAoB’s parameter setting is { = 3,
= 0.8}. Specifically, all attack methods adopted the strategy of the gradient-level model ensemble. The gray part in the table is the attack success rate of adversarial examples on the white-box model. We can see that for any victim model, the attack success rate of DAoB is higher than that of MI-FGSM, with the highest increase of 16.9 percentage points. Compared with the
six group black-box attack experiments has improved by 43.65% and 23.04%, respectively. At the same time, the audio quality of DAoB is between that of MI-FGSM and FGSM, which is consistent with our description of the search space in the previous text: DAoB can search the area near the boundary, while MI-FGSM cannot. Nevertheless, the adversarial audio obtained by DAoB still achieves a high signal-to-noise ratio and PESQ. In addition, we did not include xv-plda as a white-box model in the table because we found during the experiment that xv-plda would have a negative impact on the attack. We have not found a specific reason for this phenomenon and we will try to explain it in future work.
By adopting the boundary information, we propose DAoB, a new adversarial example generation strategy that efectively improves the transferability of adversarial examples. Experimental results show that DAoB can achieve a high success rate under the black-box scenario that is close to that of white-box attacks. In future work, we will try to reduce the size of adversarial perturbations and limit the area of adversarial perturbations to enhance the stealth of attacks without afecting the success rate of attacks.