This paper presents an algorithm for detecting a pathological voice. It is shown that the convolutional neural network efectively extracts features from the spectrograms of voice recordings and diagnoses voice disorders. The deep belief convolutional network helps to initialize weights and makes the system more reliable. The efect of the size of convolutional network filters on each layer on the system performance is also studied.
in signal processing technologies. These comprehensive tools are usually non-invasive and
non-subjective, which is generally an advantage in the medical field[
Over the past few decades, many scientific works have been carried out related to the
automatic detection of voice pathologies. Usually, these features are extracted from speech recordings
and then processed by classifiers to distinguish normal speech from pathological speech. Signs
are mainly derived from two areas of research. One of them is related to speech recognition
applications, where signal processing tools are used to automatically detect signal properties such
as Mel-frequency cepstral coeficients (MFCC), linear predictive cepstral coeficients (LPCC),
and the energy and entropy of discrete wavelet packets[
Other signs come from measuring voice quality in accordance with physiological and
etiological studies. While pitch, jitter, and flicker are used to determine the depth of speech, other
characteristics such as harmonic-to-noise ratio (HNR), normalized noise energy (NNE),
laryngealto-noise ratio (LNR), and cepstral peak prominence (CPP) represent speech hoarseness[
Modern signal processing techniques previously used in the field of speech recognition have
also made significant progress in the field of automatic detection of abnormal voice. For example,
in [
Figure 1 shows a block diagram of the proposed system for detecting abnormal voice. First, preprocessing is applied to speech recordings, which includes resampling and shape-changing methods. Then a short-time Fourier transform (STFT) is applied to obtain speech recording spectrograms as input to the CNN system. Weights in the CNN system are pre-trained using CDBN and adjusted using the back propagation method. The trained CNN system is able to automatically extract features and classify audio samples.
One of the properties of CNN is the ability to reduce the dimension of two-dimensional feature maps. Therefore, speech recordings are converted from one-dimensional signals to twodimensional spectrograms. 2.1.1. Dataset This paper uses the Saarbruecken voice database, which was registered by the Institute of phonetics of the Saarland University in Germany. This database contains 71 diferent pathologies with speech recordings from more than 2000 people. Each participant’s file contains recordings of the sustained vowels /a/, /i/, and /u/ inneutral, low, high, and low-high-low intonations, and the continuous speech sentence "Guten Morgen, wie geht as Ihnen?" ("Good morning, How are you?”). Stable vowels are used in this work because they are stationary in time and it is easier to see changes.
The following pathologies were selected as the pathological group • laryngitis • leukoplakia • Reinke’s edema • paralysis of the recurrent laryngeal nerve • carcinoma of the vocal folds • polyps of the vocal fold.
All these pathologies are organic dysphonia, which are caused by structural changes in the vocal cord. The vowel /a/ is used at a neutral height for each individual, of which 482 are healthy and 482 are diagnosed with pathologies (140 laryngitis, 41 leukoplakia, 68 Reinke’s edema, 213 recurrent laryngeal nerve paralysis, 22 vocal fold carcinoma and 45 vocal fold polyps).The data is divided into a training set and a test set containing 75% and 25% of the samples, respectively. 2.1.2. Pretraining The source speech is encoded at a frequency of 25 kHz for the pre-processing stage. The goal of this step is to reduce the amount of data in the feature map to speed up the learning process. In addition, STFT is used to convert a time domain signal to a spectral domain signal. At this stage, each file is divided into 10ms of Hamming window segments with 50% overlap between consecutive Windows. Finally, the spectrogram is changed to the same size of 60*155 points to get rid of the useless part that doesn’t contain any information. In this case, useless noise is discarded and significant signs appear.
CNN Is represented by an input layer and several hidden layers. Each individual layer consists of a convolutional layer and a merging layer . The input feature map is defined as ( = 1, ..., ), and the convolutional feature map is defined as ( = 1, ..., ). The filter weights are common to all units on the convolutional layer, calculated as,
ℎ = (∑︁ ∑︁ ,+− 1, + 0) =1 =1 (1) where , element of the m-th unit of l-th input layer , and ℎ element of m-th block of the k-th convolutional layer . is defined as the size of the filters, , is n-th unit of weight and +0 is the 0-th unit of weight.
In this procedure, objects are detected locally and automatically using shared weights across the feature map.
To reduce resolution in convolutional plys and reduce computational complexity, a union of convolutional maps is used. The maximization or averaging function is usually used to build the unifying layer. In this case, set as the size of the merging window using the maximize function, and the element on the merging layer is defined as, where is the step of the merging window moving in the convolutional layer and other variables are defined above.
The experimental network shown on figure 2 contains 10 hidden layers. In the first hidden layer, the filter size is 8*3, and the step is 1. The size of the merging window is 4*4 and step 1. After the first hidden layer, each layer was collapsed by 8 filters with the shape 8*3*8 and step 1. The size of the unifying windows is 4*4 and the RELU activation function for the entire neural network. Finally, the feature map is formed into a dense layer (a fully connected layer) to train the classification model. L2 regularization is used to solve the problem of retraining. Parameters such as pitch, size of filters in each layer, and the number of layers can be changed and should be selected depending on the signal features used. In this paper, we also studied networks with the configurations shown in table 1. The rectangular filter window is used because of the specific characteristics of the spectrograms.
Deep learning is a "black box" that requires a large amount of data and processes to adjust the weight. In turn, Bayesian methods are reliable and interpretable on small amounts of data, which is exactly what deep learning methods lack.
To combine the complementary advantages of these two methods, generative models have been developed to improve the efectiveness of deep learning on small data sets and eliminate overfitting problems. In deep learning structures, a section of the weight space is detected by a generative model, which helps the network quickly converge to a global minimum. The convolutional restricted Boltzmann machine (CRBM) is a typical generative model and is an extension of RBM with visible and hidden layers as images that is suitable for CNN settings. The model is trained to reach a state of thermal equilibrium, which is the deepest energy minimum state. In this state, hidden layers can model the structure of input data.
The CRBM consists of two layers: the visible (input) layer and the hidden (convolutional) layer . Similar to the CNN setting, the weights between the input layer and the convolutional layer are distributed among all elements in the hidden layer. Hidden elements are binary, while visible elements can be real or binary. Assume that the size of the visible layer is , and the size of the hidden layer is . There are weights and each weight is collapsed with the visible layer, and there is an ofset for each weight and an ofset for the visible layer. An energy function with a binary input is defined as,
(, ℎ) = − ∑︁ ∑︁ ∑︁ ℎ+− 1 − =1 =1 =1 ∑︁ ∑︁ ℎ − ∑︁
=1 =1 =1 An energy function with a real input is defined as (, ℎ) = 2 1 ∑︁ 2 − ∑︁ ∑︁ ∑︁ ℎ+− 1 − =1 =1 =1 ∑︁ ∑︁ ℎ − ∑︁
=1 =1 =1 Joint distribution is defined as,
Sensitivity shows the efectiveness of detecting abnormal voice files, and specificity shows the proportion of correctly detected healthy voice files. The accuracy (P) and F1-score (F1) are presented below, where the accuracy shows the proportion of the corresponding pathological voice files.
= = =
+
+
+ (6) (7) (8) 1 = 2 · (9)
+
True negative (TN) means that healthy voice recordings are correctly identified. True positive (TP) means that abnormal voice recordings are correctly identified. False-negative (FN) indicates that abnormal voice recordings are detected incorrectly and false-positive (FP) indicates that voice recordings were detected incorrectly.
There is also a diference in the operation of the CT system with and without pre-training. When using a CDN to initialize weights, the CNN setup becomes more reliable, with similar performance for the custom data set and the test data set. This shows that the CDBN can avoid overfitting problems to some extent. However, the accuracy on the test dataset is less when using pre-trained CDBN weights.
Similarly, the CRBM is trained using Gibbs block sampling[10] as an extension of the Gibbs sampling in RBM to maximize the similarity of the distribution between the construction visible layer and the input visible layer, and in this case achieve an equilibrium state. The stacks from the CRBM make up the CDBN. After the first CRBM layer is trained, activations are sent to the input of subsequent layers and the weights are " frozen”, and the remaining layers are processed in the same way. Since the visible layer in the first layer works with real data, Gaussian visible units are used for the first CRBM layer. After pre-training the weights in each layer, reverse propagation is applied to fine-tune the weights for a better classification result. Testing results are shown in tables 2 and 3. The Shared Views of Four Research Groups," IEEE Signal Processing Magazine, vol.29, no. 6, pp. 82-97, 2012. [10] H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations," presented at the Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Quebec, Canada, 2009. [11] Wu H. et al. A deep learning method for pathological voice detection using convolutional deep belief networks //Interspeech 2018. – 2018.