A model designed for automatic prediction of diseases based on multimedia data collected in hospitals is introduced in this working notes paper. In order to perform the automatic diseases prediction eficiently, while using as few data as possible for training, we develop a two-stage learning strategy, which first performs the weighted discriminant embedding (WDE) to project the original data to a low-dimensional feature subspace and then utilizes the cost-sensitive nearest neighbor (CS-NN) method in the learned subspace for disease prediction. The proposed approach is evaluated on the MediaEval 2018 Medico Multimedia Task.
Aiming at improving the eficiency of detecting medical abnormalities in the machine intelligence assisted medical diagnosis, and using as little information as possible, the MediaEval 2018 Medico Multimedia Task [3] seeks to design an integrated approach to assist the medical experts’ decision-making process using a combination of video and image information, as well as other sensory information. In this paper, a two-stage learning strategy is introduced to facilitate eficient detection of diseases using multimedia and sensory information. The first stage consists of a dimensionality reduction process which projects the original data to a low-dimensional feature representation using weighted discriminant embedding (WDE), which improves the eficiency of the learning process while also preserving the key discriminant information of the original data. Then, the cost-sensitive nearest neighbor (CS-NN) method is employed to make the prediction in the learned subspace.
Let be the training set: = {(x1, 1), · · · , (x, )}, where x ∈ R ( = 1, ..., ) denotes the feature representation of the -th sample, ∈ {1, · · · , } denotes the label of x, denotes the number of data samples in the set, denotes the number of classes, and denotes the original dimension of data. Given the training set, weighted discriminant embedding (WDE) aims to learn a transformation matrix W ∈ R× ( ≤ ), which is capable of projecting the original high-dimensional data to a low-dimensional subspace = R, where the weighted discriminant information could be preserved.
In this year’s Medico task, the sample numbers in diferent classes are highly imbalanced. To enhance the algorithm’s power in making correct detection on rarer classes, we expect that data samples belonging to the same class, especially for the rarer class, should be close to each other as much as possible in the learned subspace, while nearby data samples from diferent classes, again, especially for rarer classes, should be separated from each other as much as possible in the learned subspace.
To minimize the weighted intra-class scatter, we present the following objective:
W = arg min ︁( ∑︁ W (x − x )(x − x ) W︁) , (
W ,=1
where = ( + )/2 if = ; and 0 otherwise. Here
indicates the importance of class and is defined using the
entropy-based formulation [2]:
= −
(1 − )2
log(),
where denotes the proportion of class in the dataset. In
Eq. (
W = arg min (W LW),
W where L is a Laplacian matrix [1] defined as L = D − A, with D being a diagonal matrix defined as ( ) = ∑︀=1() ( = 1, · · · , ).
Similarly, we define the following objective function to maximize the weighted inter-class scatter:
W = arg max ︁( ∑︁ W (x − x )(x − x ) W︁) , (4)
W ,=1 where = ( + )/2 if ̸= ; and 0 otherwise. Here = (−‖ x − x ‖2/2 2) is utilized to measure the closeness between two data samples. Eq. (4) could be rewritten as:
W = arg max (W LW), (5)
W
where L = D − B, with D being a diagonal matrix
defined as ( ) = ∑︀
=1() ( = 1, · · · , ).
(
We integrate Eqs. (
W = arg max
W ︂(
W LW )︂ W LW .
Then the optimal W that maximizes the objective function in Eq. (6) is composed of the normalized eigenvectors corresponding to the largest eigenvalues of the following eigen-decomposition problem:
Lw = Lw. (6) (7) 3
To evaluate our approach, we test its performance on the MediaEval 2018 Medico Multimedia Task. The task contains both development set (with 5, 293 samples) and test set (with 8, 740 samples). For each sample, we use six types of features: the 168-D JCD feature; the 18-D Tamura feature; the 33-D ColorLayout feature; the 80-D EdgeHistogram feature; the 256-D AutoColorCorrelogram feature; and the 630-D PHOG feature. The totally dimension is 1, 185.
We participate in two subtasks: 1) Classification of diseases and findings; and 2) Fast and eficient classification. For both tasks, we submit 5 runs.
from the development set for training; ∙ For Run 1 (on both subtasks), we use all the data ∙ For Run 2 (on both subtasks), we randomly select 50% data for each class from the development set for training; ∙ For Run 3 (on both subtasks), we randomly select 50% data for each class from the development set, together with the remaining data in the “out-ofpatient” and “instruments” classes, for training; ∙ For Run 4 (on both subtasks), we randomly select 25% data for each class from the development set for training; ∙ For Run 5 (on both subtasks), we randomly select 25% data for each class from the development set, together with the remaining data in the “out-ofpatient” and “instruments” classes, for training.
In the training stage, we use the training data to learn the transformation matrix W via WDE. We set = 1 and the subspace dimension = 50. In the test stage, we use the obtained W to map both training and test data to the 50-D subspace, and then use the cost-sensitive nearest neighbor (CS-NN) method for the nfial classification in the learned subspace, where the cost of misclassifying the data of class ( = 1, · · ·
, ) to other classes is defined as = /, with and being the total number of the training data and the number of data in class , respectively. task 1 and subtask 2, respectively. Although the accuracy looks good, the overall performance is far from satisfactory as the results on other four important criteria are relatively low. 4) and that on Run 3 (Run 5), we observe that even we use all the data from the minority classes (i.e., the “out-of-patient” and “instruments” classes), the performance is not improved. The reason might be that the number of data in these two classes are too small to represent the “real” distribution of the classes. On possible solution is to employ the oversampling technology to reasonably and faithfully generate samples for minority classes. 4
In this paper, we propose a subspace learning method called weighted discriminant embedding (WDE), aiming at discovering the discriminant subspace for imbalanced dataset. After dimensionality reduction, the cost-sensitive nearest neighbor is utilized for classification. We plan to extend our work from two aspects. First, we will generalize our approach to nonlinear case to enhance its data representation ability. Second, we will incorporate some oversampling methods into our approach to make it stronger for imbalanced learning problem.
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 61503317, in part by the General Research Fund (GRF) from the Research Grant Council (RGC) of Hong Kong SAR under Project HKBU12202417, and in part by the SZSTI Grant with the Projct Code JCYJ20170307161544087.