An endoscopy is a procedure in which a doctor uses specialized instruments to view and operate on the internal organs and vessels of the body. This paper aims to detect the diseases and abnormalities in the Gastro-Intestinal Tract using multimedia data. It difers from other projects in the medical domain because it does not use medical imaging like X-rays, CT scan etc. The dataset, which comprises of 5293 images, is provided by MediaEval Benchmarking Initiative for Multimedia Evaluation. The data is collected during traditional colonoscopy procedures. Techniques from the fields of multimedia content analysis (to extract information from the visual data) and machine learning (for classification) have been used. On testing data, 98% accuracy, 0.76 F1 and an MCC of 0.75 is achieved using majority voting of logistic regression, random forest, and extra trees classifiers.
Medical image diagnosis is one of the most challenging tasks
pertinent to the industry of computer vision. Most of the work in the
recent times has been done on CT-Scans, X-Rays, and MRI etc. The
Medico Task of 2018 [
Feature Engineering is one of the most challenging and key part
of any Machine Learning problem. Figure 1 shows the proposed
model. Discriminating features are the requirement for the function
approximation. The task organizers provided 6 pre-computed visual
features for every image. These include JCD, Tamura, Color Layout,
Edge Histogram, Auto Color Correlogram and PHOG. Alongside
these pre-computed visual features, deep learning features are also
used to extract meaningful information for classification. There are
some visual features those can be extracted by using deep Networks.
As the training dataset is of 5293 images is very low for the training
of a deep learning model, a pre-trained model VGG19 is used [
Classifiers are trained on the logistic regression [
The interesting characteristics of this competition included the limited data to train the models and the class imbalance. The technique of resampling is used to generate more data for each class. The resampling generated some more features of each class and resulted in the same number of instance for each of the available 16 classes. The resampling also increased the training dataset and the increased dataset is used to train and validate diferent models. 3
The linear regression, extremely randomized trees and random forest models have been implemented using Python’s scikit-learn package. We trained logistic regression, random forest, and extremely random trees on both deep and global features. The results are first evaluated on training data using 10 Fold cross validation. By applying the proposed model, we obtained the accuracy of 97%, F1 score of 0.90 and MCC of 0.81 on the 10 fold cross validation of the training data. Based on this initial investigation, the following runs are submitted to evaluate the performance of classifiers independently. The runs are submitted with the focus of 3 runs for speed results generation and 3 runs for accuracy.
• Run1 Ensemble of 7 features [JCD, Tamura, Color Layout, Edge Histogram, Auto Color Correlogram, PHOG and VGG features] trained on 60 images, using voting of the logistic regression, random forest and extremely random trees classification algorithms. • Run2 Same as Run1 but trained on 300 images. • Run3 Same as Run1 but trained on all 5293 images. • Run4 Ensemble of 6 features [JCD, Tamura, Color Layout, Edge Histogram, Auto Color Correlogram, and PHOG] trained on 60 images, using voting of the logistic regression, random forest and extremely random trees classification algorithms. • Run5 Same as Run4 but trained on 300 images.
• Run6 Same as Run4 but trained on all 5293 images.
Table 1 shows the summary of some evaluation criterions on best run. Accuracy of 97.9% is observed with F-score of 0.75 and MCC of 0.76. It is interesting to see that the best run is obtained by using just global features without using any deep learning features. We will investigate in future why deep features perform poorly. Initial investigation has indicated that a lot of samples that should belong to class “ulcerative-colitis" are misclassified as class “esophagitis" by using deep features. The best run is obtained using Run6 in which all 5293 images are used and this approach is basically ensemble of 6 features (JCD, Tamara, Edge Histograms, Color Layout, Auto Color Correlogram and PHOG). Logistic regression, random forest and extremely random trees is being used as a classifier with weighted majority voting. Table 2 is the confusion matrix of various classes. It is observed that total of around 1469 samples are misclassified. Two categories are mainly responsible for the misclassification which are “dyed-lifted-polyps" and “dyed-resection-margins". Around 500 samples are misclassified in these 2 categories (Tables 3 and 4). Table 5 shows the confusion matrix for polyps versus non-polyps Z. Khan, A. Tahir class. Overall, performance is satisfactory but still there is a need to investigate state of the art texture and local features to further improve the performance.
It has been observed that results produced for many classes are quite accurate. However, there are some classes that are confusing the system. Future work aims to target these classes hierarchically and improve the performance using local features.
A model to classify gastro-intestinal abnormalities using endoscopic images is presented. Training (5293 samples) and Testing (8740 samples) data was provided by MediaEval Benchmarking Initiative for Multimedia Evaluation. As mentioned earlier in the introduction, the study used multimedia content analysis, machine learning and ensemble learning techniques for classification. The best of the results were found on majority voting of three models including logistic regression, random forest and extremely random trees classifier on 6 diferent features (including JCD, Tamura, Color Layout, Edge Histogram, Auto Color Correlogram and PHOG) which resulted in an accuracy of 97% with F1-score of 0.75 and MCC of 0.76 on testing data.