In several speech applications, the requirement of high perceptual quality and intelligibility of speech is an important role in signal processing to reduce the level of noise. Therefore, improving the captured speech signal is an essential challenging task in speech applications, such as hearing aid, smart-home, mobile phone, voice - controlled device, teleconference system, smart vehicle. Microphone array (MA) technology has been commonly installed in almost all acoustic equipment to achieve better performance and noise reduction in adverse and complex recording environments. MA are typically placed at a distance from the desired speaker and provides a high directional beampattern towards the sound source while suppressing or minimizing total output noise power. MA beamforming forms directional gain at certain direction of speech source and attenuation background noise, interference. However, in the complex and adverse environment, the MA beamforming's performance is often corrupted due to the existence of transport, nondirectional noise, diffuse noise field, competing talker. Therefore, an additive spectral mask, which blocks the speech component at the observed MA signals to increase the Minimum Variance Distortionless Response (MVDR) beamformer's performance is an optimum solution. In this paper, the author proposed a new structure of artificial neural networks (ANN) for learning MA signal's characteristics to remove speech component for enhancing MVDR beamformer's evaluation. The numerical simulations show the effectiveness of the author's proposed method in improving speech quality in the term of signal-to-noise (SNR) ratio from 5.5 (dB) to 6.0 (dB) and reduce the speech distortion to 5.5 (dB). Objective scores were utilized to measure the overall performance of the conventional MVDR beamformer and the author's suggested technique.
In many applications such as mobile phones, hearing aids, teleconference system, speech acquisition, smart – home, voice – controlled device, speech enhancement is applied to improve the captured MA signals. When the target speaker is distant from the capturing microphone, speech quality and speech intelligibility are usually significantly degraded due to the reverberation, surrounding noise, interference factor. The single channel approach often leads to speech distortion due to it usually based on spectral subtraction method, which properly estimate noise power in stationary situations and not sufficient in complex, annoying and non – stationary recording scenario. Therefore, the using the MA beamforming attracted more scholars, engineering and researchers to exploit the designed spatial information to obtain better speech enhancement, noise reduction.
MA technique use the priori information about the designed MA distribution, the direction of
arrival of interest useful signal, the characteristics of background noise, the acoustical features, the
preprocessing and post – Filtering techniques to process observed MA signals. MA beamforming
algorithms can be categorized into two groups: fixed beamformer with delay-and-sum DAS [
MVDR beamformer is one of the most useful beamforming, which is applied into numerous speech applications to extract the target speech component while suppressing background noise. However, due to the microphone mismatches, the displacement of MA, the error of estimation of steering vector, the difference between microphone sensitivities, MVDR beamformer’s evaluation often corrupted. For overcoming the drawback, spectral mask is an effective solution for improving MVDR beamformer’s performance in complex environment.
Time - frequency (T-F) masking is based on the windowing - disjoint orthogonality assumption
of concentration of speech energy only to a few time - frequency points [
Beamforming is linear filtering used to MA signals to extract the desired speaker from the noisy mixture, amplify the useful signal and attenuate background noise. In this paper, the author proposed an ANN, which uses MA characteristic to train ANN for enhancing MVDR beamformer’s performance in complex, adverse recording scenario. The numerical results have confirmed the effectiveness of the author’s suggested method in increasing the speech quality from 5.5 dB to 6.0 dB and reduce speech distortion to 5.5 dB.
The rest of this article is organized as follows. The next section introduces the principal working MVDR beamformer. Section III describes the artificial neural network for training MA features. Section IV demonstrates a perspective experiment to illustrate the advantages of the author’s proposed technique. Section V concludes the promising result.
As in Figure 4, the scheme of MVDR beamformer’s implementation was depicted. In general case, the author uses dual - microphone array (DMA2) model to express the principal working of MVDR beamformer in the frequency - domain. At considered frame , frequency , two captured microphone arrays signals can be described as: !(, ) = (, )"#! + !(, ) $(, ) = (, )%"#! + $(, ) (1) (2)
Where & = '(&), & is the direction of arrival of interest useful signal relative to the axis of DMA2, ' = ⁄ is time delay, with is the range between two mounted microphones, = 343 (⁄) is the speed of propagation of sound in the air, (, ) is the clean speech component, !(, ), $(, ) is the additive noise.
We denote: (, ) = [!(, ) $(, )]( , (, ) = [!(, ) $(, )]( , &(, &) = ["#! %"#! ]( , the equations (1) – (2) can be expressed as:
(, ) = (, )&(, &) + (, ) Where //(, ) = {) (, )(, )} = G
{|!(, )|$} {$∗(, )!(, )} {!∗(, )$(, )} {|$(, )|$}
I (⬚)) is conjugate operator.
By using recursive equation, the auto – cross power spectral densities can be calculated as: /"/"(, ) = /"/" (, − 1) + (1 − )1∗(, )1(, )
/"/#(, ) = /"/#(, ) + (1 − )1∗(, )"(, ) Where is the smoothing parameter, which in the range {0 … 1}.
In several speech applications, because of the complex and annoying environment, the microphone mismatches, the different microphone sensitivities, the error of estimation of steering vector, the displacement of MA or undetermined environmental factors, the MVDR beamformer’s performance often corrupted. To overcome this drawback, the author proposed using spectral mask
The constrained criteria of MVDR beamformer assumes that minimum the total output noise power while preserving the speech data without distortion. The formulation of MVDR beamformer can be defined as:
(, )) (, )**(, )(, ) ) (, )&(, &) = 1 And the formulation of MVDR beamformer’s coefficient yields:
Unfortunately, the spectral matrix of noise is not available and still a challenging task in almost acoustic equipment. Therefore, the matrix of observed microphone array signals used instead of. The final optimum coefficient can be derived as: +,-.(, ) =
%*!*(, )&(, &) )& (, &)%*!*(, )&(, &) +,-.(, ) =
/%/! (, )&(, &) )& (, &)/%/! (, )&(, &) (3) (4) (5) (6) (7) (8) to suppress the speech component in observed MA signals to enhance beamforming’s evaluation. The author uses ANN for training to obtain an effective spectral mask.
The author’s ideal is using the spectral mask, which based on the temporal the signal - to - noise ratio Q(, ). The proposed spectral mask yields as: *(, ) =
1 1 + Q(, )
From the captured MA signals, the relation between two microphone signals can represented by the coherence /$/%(, ) = /$/%(, )VT/$/$(, ) × /%/%(, ).
In realistic recording environment, this coherence derived with the information presence of speech component and noise as the following equation: /$/%(, ) =
Q(, ) 1 + Q(, ) "$#! +
1 1 + Q(, ) * Where * = 1 in coherent noise field, and * = (')⁄' in diffuse noise field. Substituting (9), (10) can be rewritten as:
/$/%(, ) = X1 − *(, )Y"$#! + *(, )* And:
Before implementing MVDR beamformer, the observed MA signals multiplied with the spectral mask as the following way: (10) (11) (12) (13) (14) (9) *(, ) = /$/%(, ) − "$#!
* − "$#! Z!(, ) = !(, ) × *(, ) Z$(, ) = $(, ) × *(, )
In a living room, a dual - microphone system with 5 cm was used to simulate audio data with sampling frequency 16 with added coherent noise. The distance from speaker to the axis of DMA2 is = 3(). The direction of arrival of interest useful signal is & = 90(). From TIMIT database speech, the author uses 150 randomly selected sentences for recording and training data to obtain proper coherence /$/%(, ). The purpose of multilayer perceptrons (MLP) is suppress incoherent, diffuse noise field and non-directional noise to achieve microphone array signal, which contain the only coherence noise.
With the observed microphone array signal, the temporal coherence b/$/%(, ) between two mounted microphones can be computed. With = 512, b/$/%(, ) is the vector input of MLP. = [/$/%(!, ) ⋯ /$/%(233( , )]( with !, … , 233( is the frequency band. The output layer values are the promising coherence.
The value of th node of the output layer can be expressed as: 45(, ) = jk $233( "8!
233( 4($") j k "(1!)15 + "('!)m + 4($')m 18! (15) Where (. ) is the sigmoid function, "('!), 4($') are the bias weights, "(1!) mean the weight for ($) denotes the coefficient for the output of the th hidden the th input by hidden layer neutron , 4" layer neuron by the th node of the output layer, and mean the set of coefficients.
The purpose of the conducted experiment is illustrating the effectiveness of the author’s spectral mask in increasing the perceptual metric listener, the speech intelligibility and the speech quality of the MVDR beamformer’s output signal. The advantage of spectral mask, which suppresses the speech component to pass only the noise component for MVDR beamformer’s input, according to the theory of MVDR beamformer. For capturing the clean speech data, these parameters = 512, overlap 50% were set. The observed MA signals can be depicted in Figure 6.
By using the traditional MVDR beamformer, the output signal yields as:
By applying the proposed spectral mask, the processed signal can be derived in Figure 8.
An objective measurement [21] is used for calculating the speech quality of the original MA signals, the processed signals by MVDR beamformer and the author’s suggested technique. NIST (National Institute of Standards and Technology) STNR (Signal-To-Noise Ratio) [22] is an efficient method for measuring the SNR based on the estimation of sequential Gaussian mixture. WADA (Waveform Amplitude Distribution Analysis) SNR [23] based on the model of additive Gaussian noise signal and Gamma distribution of useful speech signal of talker to compute the SNR.
NIST SNR WADA SNR 6.2 4.8
Table 1 and Figure 9 compare the speech quality and energy of microphone array signal, the processed signal by MVDR beamformer and using the spectral mask. The effectiveness of the author’s suggested method has been confirmed with increasing the speech quality from 5.5 dB to 6.0 db and reducing speech distortion to 6.0 dB. Because of the original constrained criteria of MVDR beamformer is using the only noise component, therefore, the author’s proposed technique blocked the speech component at the observed MA signals, which contains the only noise component for ideally performing the MVDR beamformer to preserving the target speaker while suppressing background noise.
Due to the microphone mismatches, the different microphone sensitivities, the displacement of MA, the error of estimation of preferred steering vector or non-directional noise, MVDR beamformer’s evaluation often corrupted. Using spectral mask, which is based on training data from the ANN is an optimum solution for enhancing beamforming’s performance for recovering the clean speech data while removing interference, background noise. The author’s structure of ANN can be integrated into multi-channel system for resolving other complicated problems.
Speech enhancement plays an important role in almost acoustic speech applications to enhance perceptual listener, speech intelligibility and speech quality while removing interference, background noise or competing talker. In this article, the author suggested a structure of ANN for forming an effective spectral mask, which multiplies with the observed microphone array signals to enhance MVDR beamformer’s evaluation and extract the desired speech signal. The obtained result has confirmed the effectiveness of the author’s spectral mask, which was trained by the ANN in improvement of considered beamformer. The promising result has shown that the speech quality in the term of signal - to - noise increased from 5.5 (dB) to 6.0 (dB) and reduced speech distortion to 5.5 (dB). The proposed configuration of ANN can be integrated into a multi-channel system for solving complicated problems. [20] J. Oruh, S. Viriri and A. Adegun, "Long Short-Term Memory Recurrent Neural Network for Automatic Speech Recognition," in IEEE Access, vol. 10, pp. 30069-30079, 2022, doi: 10.1109/ACCESS.2022.3159339. [21] SNRVAD. [Online]. Available: https://labrosa.ee.columbia.edu/projects/snreval/. [22] A. W. Rix, J. G. Beerends, M. P. Hollier and A. P. Hekstra, "Perceptual evaluation of speech quality - A new method for speech quality assessment," Proc. Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2001, pp. 749–752. [23] C. Kim and R.M. Stern, "Robust signal-to-noise ratio estimation based on waveform amplitude distribution analysis," Proc. Interspeech 2008, 2598-2601, doi: 10.21437/Interspeech.2008-644.