We propose a two-step classification framework to diagnose Parkinson's Disease (PD) using speech samples. At the first stage, multiple one-way ANalysis Of VAriance (ANOVA) is used on independent subsets of vocal features to extract the best set of features from each speech processing algorithm. These extracted feature subsets are then merged with other baseline vocal features (shimmer, jitter, pitch, harmonicity, vocal fold, and fundamental frequency parameters) to form the training feature set. In the second step, this combined training set is used to train an extreme gradient boosting (XGBoost) classification model, which is a decision tree based algorithm. The overall model performance was scored and evaluated using the Receiver Operating Characteristic Area Under Curve (ROC AUC), F-Measure, Matthews Correlation Coefficient (MCC), and accuracy. It was then compared with benchmarked statistical classifiers and other studies that use different combinations of features from this PD dataset. We apply one-way ANOVA on different speech feature sets to extract the best features without losing useful vocal information. Our classification performance outperforms state-of-theart PD classification models that use generic feature selection methods or use only one or more of the vocal feature subsets.
PD is one of the most common diseases of the motor
system degeneration that results from the loss of cells in
various parts of the brain. PD’s primary symptoms are tremor,
slow movement, speech disorder, impaired balance, and gait
problems. There are no diagnostic tests or biomarkers for PD
diagnosis because the symptoms resemble the ones observed
due to other diseases. Physicians use methods like MRI,
ultrasound, blood tests to eliminate other conditions with
similar symptoms. Research has also been done to detect PD
using various motor and non-motor symptoms
PD Diagnosis has typically involved measuring the
severity of the symptoms using non-invasive medical techniques.
Since approximately 90% of PD patients suffer from speech
disorders, analyzing speech samples to study vocal
impairment is considered as the most common technique for
PD diagnosis
Past studies have used different combinations of the aforementioned features to train classifiers without any focus on extracting useful features from different types of vocal features. This study proposes a novel two-step classification framework for PD diagnosis. The first step uses multiple one-way ANOVAs to extract vocal features from MFCCs, WTs, and TQWTs separately. Extracted feature sets are merged with other baseline vocal features to form the final training set. In the second step, a decision-tree based classifier is trained on this training set to make predictions. To the best of our knowledge, this is the first PD classification study that employs a multiple ANOVA strategy to extract the best vocal features from TQWT, MFCCs, and WTs, and combine all of them with standard baseline features like jitter, skimmer, etc., to generate an extensive training set. Our study shows that extracting features separate from each other prevents not only loss of useful vocal/ signal information but also addresses the high-dimensionality nature of the dataset. Using a decision-tree based classifier on extracted features also handles any class imbalance without the need of oversampling or under-sampling the dataset. Classification results obtained on the public dataset show that our proposed two-step framework outperforms current state-ofthe-art models that use just one or more of the vocal feature subsets without extracting the best features from individual algorithms.
There are no laboratory tests or biomarkers for the diagnosis
of PD
Analyzing voice samples and deterioration has shown
great potential in the advancement of PD diagnosis
There have been multiple studies on PD classification
techniques using vocal features.
Since then, there have been a few studies that have
proposed different classification methods using TQWT-based
features and this larger dataset built by
Solana-Lavalle, Gala´n-Herna´ndez, and Rosas-Romero (2020) proposed using a Wrapper Feature Selection method along with an SVM classifier and obtained a classification accuracy of 94.7% on the larger dataset. The feature selection method used in this study did not account for the biological and vocal features in the dataset separately and instead selected the best K features suited to the used classifier. Only 8 to 20 features are selected from 754 vocal features. This leads to loss of valuable acoustic and signal information, especially from WT and TQWT-based features – since they are extensive WT techniques that quantify frequency deviations in speech signals and contain 10+ original features each. Wrapper feature selection methods try to find the best set of features suited to a specific learning algorithm by evaluating all combinations of features against the evaluation/ performance metric, and thus, there is also a high chance of over-fitting to the training data.
Compared to the previous work, our work is one of the first studies to demonstrate an improved speech feature selection methodology and a decision-tree based robust classifier that handles class imbalance without having to modify the original dataset by over-sampling or under-sampling.
The dataset we used for the analysis was gathered at the
Department of Neurology in Cerrahpasa Faculty of Medicine,
Istanbul University
MFCCs of a sound signal separate the impact of the vocal
cords (source) and vocal tract (filter) in the signal
WT is used to analyze signals in terms of wavelets, time,
and frequency domain limited functions to detect regional
fluctuations. WT features of the basic frequency of speech
signal (F0) have been used for PD diagnosis
TQWT is a discrete-time wave transform, like WT.
TQWT uses 3 tunable parameters (Q, J , and r) to tune
it based on the behavior of the speech signal
PD classification is treated as a binary classification task in which the framework takes an input of extracted speech features and predicts a class (PD/ No PD). Figure 1 illustrates the end-to-end classification framework for PD diagnosis. The dataset contains 752 features in 4 feature sets: baseline features, MFCCs, WT, and TQWT. The drawback of using MFCCs, WT, and TQWT together is the ‘curse of the dimensionality’ problem. High-dimensional datasets lead to overfitting, hinders useful vocal information in the dataset, and leads to computational instability. Extracting a meaningful set of features from each feature set is important to reduce the dimensionality of the feature set while still ensuring that all useful vocal features are retained. This will also reduce the computational complexity of the classifier. We propose using the one-way ANOVA selection schemes to extract the best performing training features from MFCCs, WT, and TQWT feature-sets. The selected features from each method are merged with the baseline features. This merged feature set serves as the training data for the classifier. We then train an optimized XGBoost classifier on the training data and evaluate its performance against past studies and benchmarked statistical classification models.
ANOVA Feature Selection ANOVA is a statistical hypothesis test used to determine whether the means from two or more samples of data come from the same distribution or not. It is usually used in problems involving numerical inputs and a classification target variable. There are two types of ANOVA: one-way ANOVA and two-way ANOVA. One-way ANOVA only involves one independent variable, while two-way ANOVA compares two independent variables.
To find how well each speech feature discriminates between the two output classes, we use a one-way ANOVA F-test. F-tests are a class of statistical tests that calculate the ratio between variances values. ANOVA tests the following null hypothesis (H0): there is no difference between features, and the features have the same mean value. The alternate hypothesis (H1) is that there is a difference between the means and the groups (feature variances are not equal). The ANOVA F-test produces an F-score based on the variance ratio calculated among the means to the variance within the group. Group means drawn from features with the same or highly similar mean values will have lower variance between the group and have a lower F-score. A high F-score implies that features have different mean values and can discriminate between the dependent variable categories better. The results of this test can be used for feature selection where those features that are independent of the target variable can be removed from the training set. The F-score for each speech feature is calculated as follows:
F =
K BGV = X ni(Y i:
i=1 W GV = XK Xni (Yij i=1 j=1
K
N
Y )2 1 Y i:)2 K Where K is the number of groups, N is the overall sample size, ni is the number of observations in the ith group. Yij is the jth observation in the ith out of K groups. Y is the overall mean of the variable set, and Y i:is the sample mean of the ith group. K 1 is also defined as the degrees of freedom in some studies, referring to the maximum number of logically independent features with the freedom to vary.
The scikit-learn machine learning library provides a
native implementation of a one-way ANOVA F-test
(f classif) and a SelectKBest class to pick
features with the highest F-scores. The F-test score function
returns an array of F-scores, one for each speech feature.
SelectKBest class then picks the first k features with the
highest scores
Using ANOVA feature selection on the entire dataset leads to loss of vital vocal information. Each of the 54 baseline features provides fundamental and distinct speech information. Removing any of these baseline features leads to lost information, which is not available in any of the other vocal feature sets. Just selecting the best k features from the entire dataset using the highest F-scores leads to many crucial original and derived features being left out. This is especially observed in the highly dimensional WT and TQWT feature subsets. This can also lead to overfitting to certain derived features or a classification model that relies primarily on features that perform well for that specific model instead of features that represent the disease. To conserve vital information obtained from each feature subset while also addressing the broader dimensionality problem, we extract features from each feature set separately. This ensures that the original signals are retained and focuses on finding the best performing derived features. All baseline features are used, and the best ki features are extracted from MFCCs, WTs, and TQWTs, respectively. ki is obtained for each subset using grid-search cross-validation. The grid-search crossvalidation evaluated a different combination of ki features from each subset to find the optimal classification performance. Forty features from MFCCs, 75 from WT, and 100 from TQWT were selected with the highest F-scores in their category, and these were used along with baseline features as the training set.
Learning Rate Number of Estimators Max Depth Min Child Weight Gamma Subsample Col. Sample by Tree Num. Thread Scale POS Weight XGBoost is a robust gradient boosting library based on ensemble tree-boosting. Its fundamental function predicts a new classification membership after each iteration. Predictions are made from weak classifiers and are iteratively improved. Incorrect classifications from the previous iteration receive higher weights, forcing the model to focus on their performance improvement. The final classification combines the improvement of all the previously modeled trees. XGBoost is not susceptible to overfitting because of its more robust regularization framework that constrains overfitting. An XGBoost classifier was trained on the training dataset that was extracted after ANOVA. XGBoost’s
MFCC WT TQWT Baseline + MFCC Baseline + WT Baseline + TQWT MFCC + WT MFCC + TQWT WT + TQWT
All features
multi-ANOVA
+ XGBoost (proposed framework)
Combined ANOVA
+ XGBoost
Gunduz (2019): All features + CNN
Gunduz (2019): All features + SVM
Sakar et al. (2019): Top-50 features using mRMR + SVM (RBF)
Polat (2019): RFC
AUC 0.91 0.89 n/a n/a n/a n/a built-in cross-validation was used at each iteration to get the optimal boosting iterations in a single run. Grid-search cross-validation was used to optimize the model parameters. The final hyper-parameters obtained are shown in Table 2. The optimized model achieved the highest classification accuracy of 94.78%. In the following section, we evaluate our framework’s performance with benchmarked statistical models and other studies on this dataset.
Evaluation metrics are needed to assess the predictive performance of the proposed framework. Although accuracy is a common metric, it may yield misleading results in case of unbalanced class distribution. Evaluation metrics such as F-measure, MCC, and ROC AUC can measure how well a classifier performs, even in class imbalance cases. We use ROC AUC, F-Measure, MCC, and accuracy to evaluate the performance of the proposed framework against statistical classifiers and other studies using this dataset. While using individual feature sets, TQWT-based features perform better than other feature subsets. Significant improvement in Performance Metrics
F1 Acc. MCC classification performance is observed when one feature set (baseline, MFCC, or WT) is complemented with TQWT features. Using ANOVA to extract the best features and then using them to train an XGBoost model performs better than other state-of-the-art techniques proposed on this dataset. Polat’s (2019) proposal to use SMOTE to over-sample the minority class and train an RFC leads to a slightly better classification accuracy (0.001). However, AUC, F-measure, and MCC metrics of Polat’s model are unknown. The performance of benchmarked classifiers, including SVM, RFC, and Gradient Boosting Classifier (GBC), using different feature combinations is shown in Table 3. The performance metrics of our proposed framework, compared to other studies, are presented in Table 4. We also demonstrate that using a multi-ANOVA strategy performs better than one ANOVA on the entire feature set.
This paper presents a two-step classification framework to diagnose PD using a set of 753 vocal features. We propose a novel vocal-feature selection technique for PD classification using multiple one-way ANOVA on the MFCCs, WT and TQWT. The selected features are merged with baseline vocal and biological features to form the training set. We propose an XGBoost classifier trained on the extracted data for PD classification. The proposed framework achieves a classification accuracy of 94.71% with an F-1 of 0.965 and an MCC of 0.86. We show that the proposed framework performs better than the state of the art without altering the dataset by over or under-sampling. We demonstrate that separately extracting features from different algorithms reduces the dimensionality without the loss of any vital speech information and performs better than a generic feature selection technique. We also show that the proposed framework performs better than benchmarked statistical classifiers. Most literature on PD diagnosis relies on a very small sample size collected from 20-30 persons. High levels of accuracy in predictions of models based on a significantly larger data set (i.e., 252 persons) have been demonstrated in this paper. Thereby, the generalization capabilities of the model are validated. Using the proposed framework, clinical diagnosis of early-onset of PD will be consistent across physicians, thereby eliminating the chances of misdiagnosis. Specifically, the high levels of accuracy, F1, MCC, and ROC AUC indicate that there is a very negligible chance of missing a diagnosis. We have open-sourced the code used in this study in a public GitHub repository (https://github.com/ Gaurangprasad/parkinson disease ANOVA classifier).