Automatic detection of pathological voice disorders, such as paralysis of the vocal cords or Reinke's edema, is a complex and important problem of medical classification. While deep learning methods have made significant progress in speech recognition, fewer studies have been conducted in the detection of pathological voice disorders. This paper presents a new system of pathological voice recognition using convolutional neural network (CNN) as the basic architecture. The new system uses spectrograms of normal and abnormal speech recordings as input to the network. Initially, the deep belief convolutional network (CDBN) is used to pretrain CNN weights. It acts as a generative model for studying the structure of input data using statistical methods. CNN then uses training with controlled back propagation to adjust the weights. As a result, it is clear that a small amount of data can be used to achieve good results in classification using this approach. The performance analysis of this method is performed using real data from the SaarbruckenVoice database.
Voice pathologies affect the larynx and lead to irregular fluctuations in the vocal folds. This leads to psychological and physiological problems for individuals, and also has a significant impact on the economy, taking into account the costs of medical diagnosis and treatment. The traditional method of diagnosing voice pathology relies on the experience of a doctor and on expensive devices such as a laryngoscope, endoscope, etc. However, computer-based medical systems for diagnosing voice pathologies are becoming popular due to significant advances in signal processing technologies. These comprehensive tools are usually non-invasive and non-subjective, which is generally an advantage in the medical field [1].
Over the past few decades, many scientific works have been carried out related to the automatic detection of voice pathologies. Usually, these features are extracted from speech recordings and then processed by classifiers to distinguish normal speech from pathological speech. Signs are mainly derived from two areas of research. One of them is related to speech recognition applications, where signal processing tools are used to automatically detect signal properties such as Mel-frequency cepstral coefficients (MFCC), linear predictive cepstral coefficients (LPCC), and the energy and entropy of discrete wavelet packets [2][3][4].
Other signs come from measuring voice quality in accordance with physiological and etiological studies. While pitch, jitter, and flicker are used to determine the depth of speech, other characteristics such as harmonic-to-noise ratio (HNR), normalized noise energy (NNE), laryngealto-noise ratio (LNR), and cepstral peak prominence (CPP) represent speech hoarseness [5]. Most research papers use the Massachusetts Eye and year Infirmary (MEEI) database.However, healthy voice recordings and abnormal voice recordings in this database are recorded in two different environments [6], which makes it difficult to distinguish whether these are discriminating environments or voice features.The Saarbruecken Voice Database is a downloadable database with all recordings sampled at 50 kHz and 16-bit resolution. This database is relatively new, So little research has been done on it. However, the recordings are recorded in the same environment, so it was decided to choose it for this study.
Modern signal processing techniques previously used in the field of speech recognition have also made significant progress in the field of automatic detection of abnormal voice. For example, in [7], the Russian language Gaussian Mixture Model (GMM) based on the Saarbruecken voice database is used, and 67% classification accuracy is achieved with a neutral stable vowel /a/. However, with the increasing computing capabilities of hardware and the improvement of machine learning algorithms, the Markov model hidden in the deep neural network gradually replaces the traditional GMM-HMM [8] and becomes a popular method of speech recognition. To date, deep learning methods are not commonly used in the field of pathological voice detection, mainly due to the limited amount of data, since DNN requires a large amount of data for training. In [9], a restricted Boltzmann machine (RBM) is proposed as an unsupervised method for pre-training DNN to accurately achieve global minima. As a generative model, it improves deep learning performance even on small datasets. Deep belief convolutional networks (CDBNS) were proposed in [10] as an advanced specific structure for CNN pre-training. This article considers a new deep learning method for automatic detection of abnormal voice. In this paper, we use the CNN convolutional neural network structure for automatic analysis of speech recording spectrograms. CDBN is used for pre-training weights and preventing problems with over-training. A similar approach is proposed in [11], but the influence of convolutional neural network parameters is left behind in that study.
Figure 1 shows a block diagram of the proposed system for detecting abnormal voice. First, preprocessing is applied to speech recordings, which includes resampling and shape-changing methods. Then a short-time Fourier transform (STFT) is applied to obtain speech recording spectrograms as input to the CNN system. Weights in the CNN system are pre-trained using CDBN and adjusted using the back propagation method. The trained CNN system is able to automatically extract features and classify audio samples.
One of the properties of CNN is the ability to reduce the dimension of two-dimensional feature maps. Therefore, speech recordings are converted from one-dimensional signals to twodimensional spectrograms.
This paper uses the Saarbruecken voice database, which was registered by the Institute of phonetics of the Saarland University in Germany. This database contains 71 different pathologies with speech recordings from more than 2000 people. Each participant's file contains recordings of the sustained vowels /a/, /i/, and /u/ inneutral, low, high, and low-high-low intonations, and the continuous speech sentence "Guten Morgen, wie geht as Ihnen?" ("Good morning, How are you?"). Stable vowels are used in this work because they are stationary in time and it is easier to see changes.
The following pathologies were selected as the pathological group • laryngitis • leukoplakia • Reinke's edema • paralysis of the recurrent laryngeal nerve • carcinoma of the vocal folds • polyps of the vocal fold.
All these pathologies are organic dysphonia, which are caused by structural changes in the vocal cord. The vowel /a/ is used at a neutral height for each individual, of which 482 are healthy and 482 are diagnosed with pathologies (140 laryngitis, 41 leukoplakia, 68 Reinke's edema, 213 recurrent laryngeal nerve paralysis, 22 vocal fold carcinoma and 45 vocal fold polyps).The data is divided into a training set and a test set containing 75% and 25% of the samples, respectively.
The source speech is encoded at a frequency of 25 kHz for the pre-processing stage. The goal of this step is to reduce the amount of data in the feature map to speed up the learning process. In addition, STFT is used to convert a time domain signal to a spectral domain signal. At this stage, each file is divided into 10ms of Hamming window segments with 50% overlap between consecutive Windows. Finally, the spectrogram is changed to the same size of 60*155 points to get rid of the useless part that doesn't contain any information. In this case, useless noise is discarded and significant signs appear.
CNN Is represented by an input layer and several hidden layers. Each individual layer consists of a convolutional layer 𝐻 and a merging layer 𝑉 . The input feature map is defined as 𝑉 𝑙 (𝑙 = 1, ..., 𝐿), and the convolutional feature map is defined as 𝐻 𝑘 (𝑘 = 1, ..., 𝐾). The filter weights are common to all units on the convolutional layer, calculated as,
where 𝑣 𝑙,𝑚 element of the m-th unit of l-th input layer 𝑉 , and ℎ 𝑘 𝑚 element of m-th block of the k-th convolutional layer 𝐻. 𝑁 𝑤 is defined as the size of the filters, 𝑤 𝑘 𝑙,𝑛 is n-th unit of weight and +𝑤 𝑘 0 is the 0-th unit of weight. In this procedure, objects are detected locally and automatically using shared weights across the feature map.
To reduce resolution in convolutional plys and reduce computational complexity, a union of convolutional maps is used. The maximization or averaging function is usually used to build the unifying layer. In this case, set 𝐺 as the size of the merging window using the maximize function, and the element on the merging layer is defined as,
where 𝑠 is the step of the merging window moving in the convolutional layer and other variables are defined above.
The experimental network shown on figure 2 contains 10 hidden layers. In the first hidden layer, the filter size is 8*3, and the step is 1. The size of the merging window is 4*4 and step 1. After the first hidden layer, each layer was collapsed by 8 filters with the shape 8*3*8 and step 1. The size of the unifying windows is 4*4 and the RELU activation function for the entire neural network. Finally, the feature map is formed into a dense layer (a fully connected layer) to train the classification model. L2 regularization is used to solve the problem of retraining. Parameters such as pitch, size of filters in each layer, and the number of layers can be changed and should be selected depending on the signal features used. In this paper, we also studied networks with the configurations shown in table 1. The rectangular filter window is used because of the specific characteristics of the spectrograms.
Deep learning is a "black box" that requires a large amount of data and processes to adjust the weight. In turn, Bayesian methods are reliable and interpretable on small amounts of data, which is exactly what deep learning methods lack.
To combine the complementary advantages of these two methods, generative models have been developed to improve the effectiveness of deep learning on small data sets and eliminate overfitting problems. In deep learning structures, a section of the weight space is detected by a generative model, which helps the network quickly converge to a global minimum. The convolutional restricted Boltzmann machine (CRBM) is a typical generative model and is an extension of RBM with visible and hidden layers as images that is suitable for CNN settings. The model is trained to reach a state of thermal equilibrium, which is the deepest energy minimum state. In this state, hidden layers can model the structure of input data.
The CRBM consists of two layers: the visible (input) layer 𝑉 and the hidden (convolutional) layer 𝐻. Similar to the CNN setting, the weights 𝑊 𝑘 between the input layer and the convolutional layer are distributed among all elements in the hidden layer. Hidden elements are binary, while visible elements can be real or binary. Assume that the size of the visible layer is 𝑁 𝑉 , and the size of the hidden layer is 𝑁 𝐻 . There are 𝐾 weights and each weight 𝑊 𝑘 is collapsed with the visible layer, and there is an offset 𝑏 𝑘 for each weight and an offset 𝑐 for the visible layer. An energy function with a binary input is defined as,
Sensitivity shows the effectiveness of detecting abnormal voice files, and specificity shows the proportion of correctly detected healthy voice files. The accuracy (P) and F1-score (F1) are presented below, where the accuracy shows the proportion of the corresponding pathological voice files.
𝑇 𝑃 𝑇 𝑃 + 𝐹 𝑁 (6)
True negative (TN) means that healthy voice recordings are correctly identified. True positive (TP) means that abnormal voice recordings are correctly identified. False-negative (FN) indicates that abnormal voice recordings are detected incorrectly and false-positive (FP) indicates that voice recordings were detected incorrectly.
There is also a difference in the operation of the CT system with and without pre-training. When using a CDN to initialize weights, the CNN setup becomes more reliable, with similar performance for the custom data set and the test data set. This shows that the CDBN can avoid overfitting problems to some extent. However, the accuracy on the test dataset is less when using pre-trained CDBN weights.
Similarly, the CRBM is trained using Gibbs block sampling [10] as an extension of the Gibbs sampling in RBM to maximize the similarity of the distribution between the construction visible layer and the input visible layer, and in this case achieve an equilibrium state. The stacks from the CRBM make up the CDBN. After the first CRBM layer is trained, activations are sent to the input of subsequent layers and the weights are " frozen", and the remaining layers are processed in the same way. Since the visible layer in the first layer works with real data, Gaussian visible units are used for the first CRBM layer. After pre-training the weights in each layer, reverse propagation is applied to fine-tune the weights for a better classification result. Testing results are shown in tables 2 and 3.