A major challenge for Medical Image Retrieval (MIR) is the discovery of relationships between low-level image features (intensity, gradient, texture, etc.) and high-level semantics such as modality, anatomy or pathology. Convolutional Neural Networks (CNNs) have been shown to have an inherent ability to automatically extract hierarchical representations from raw data. Their successful application in a variety of generalised imaging tasks suggests great potential for MIR. However, a major hurdle to their deployment in the medical domain is the relative lack of robust training corpora when compared to general imaging benchmarks such as ImageNET and CIFAR. In this paper, we present the adaptation of CNNs to the medical clustering task at ImageCLEF 2015.
This paper documents the Biomedical Engineering and Technology (BMET)
team from the University of Sydney's submissions for the ImageCLEF 2015 [
The objective of our experiments was to evaluate the e ectiveness of
Convolutional Neural Networks (CNNs) for this task. In particular, we propose a deep
learning framework that learns high-level representations of anatomical elements
contained in each image and uses these to cluster the images.
since 2012 [3{5]. CNNs share the common features of all deep learning
algorithms: stacked layers of neuronal subunits that learn hierarchical
representations (allowing the data to be understood at various levels of abstraction, in
isolation or combination [
A more subtle implication of deep learning is that it can automatically extract
features from raw data [3{5]. Typically, a key factor in the success of typical
machine learning algorithms is extracting salient features from the raw data.
Taking image recognition as an example, a feature set such as edges or SIFT [
Deep learning algorithms, however, are able to directly utilise raw data
instead of hand-crafted features. By feeding the data sequentially through many
successive layers of subunits, the higher levels of the system are able to
understand the data in terms of successively abstract representations [
Medical Image Retrieval (MIR) tasks, such as the tests devised for
ImageCLEF, require learning precisely these kinds of highly abstract representations,
i.e. image modality or the anatomical semantics of the image. However, to the
best of our knowledge it is not currently a well established method in this domain.
This is due to not only the inherent challenges of medical images[
Large training sets are a current necessity of very deep systems because they contain many millions of internal parameters that must be estimated from the data. Too little data can result in the the higher-level neurons' activation being the result of salient features of the training set and not re ecting the high-level representations. If this 'over tting' occurs then the system's ability to generalise on new data is severely impaired [13].
In addition to the issues regarding the volume of data required, it must be mentioned that while deep learning can automatically perform excellent feature extraction, this comes at the signi cant cost of the larger number of hyperparameters that must be evaluated in order to nd an optimal system [14]. For 4 Krizhevsky et. al. [8] used approximately 1.2 million labelled examples for their breakthrough result in ImageNET in 2012. example, compared to a commonly used machine learning algorithm such as the Support Vector Machine (SVM) that has a basic hyperparameter search space with dimensions of choice of kernel, regularization constant and kernel hyperparameter, even the simplest implementation of a CNN requires fundamental choices about the number and type of layers, lter size and number of lters per layer, and the learning rate. More advanced implementations include factors such as unit activation function and the use of dropout. While there are guidelines for these choices in the literature [14], the di culty of even a small parameter search is compounded by the increased computational requirements of training the system. 3
A requirement of our classi ers was uniformly sized input vectors, however, there was some variation in the training data size. The sizes of the images were at least 1600px in one dimension and then between 1600 and 2348px in the other. In order to use consistently sized images and not lose any crucial information, we created a new square image with the dimensions of the largest dimension of the original image, lling any empty space with black pixels.
Even prior to training the CNN, we were aware that the computational requirements were quite demanding and this would be exacerbated by using large images. With that in mind, we resized the images to 256x256px to reduce computational overhead. Good results have been reported in the literature for complex tasks with 48x48px images [15] and Krizhevsky et. al. [8] achieved state-of-the art general object recognition with 256x256px images (technically, the system had an input of 224x224px, but these were subimages of the original 256x256px images).
After resizing the images were 256x256x3px, the third dimension describing the three colour channels. The images supplied were in actual fact gray scale, i.e. all colour channels were equal-valued, so we simply sliced the array preserving only the 'red' channel.
We chose to train a single run of four models using 100% of the training data with no parameter optimization and use the ImageCLEF results as the test. 3.2
The task requires multi-label classi cation across four anatomical classes, with a null set indicating that the data is a true-negative image taken with the same camera, but not of the human body. To facilitate this output from our experiments we constructed 4x1 vs. All CNN models.
The architecture for the CNN used for our experimentation was based on a simpli ed version of Yann LeCun et. al.'s [16] LeNet-55. This basic CNN is capable of correctly classifying the MNIST handwritten digit database with 1.7% 5 http://deeplearning.net/tutorial/lenet.html test error. We modi ed the input to account for larger images and output a di erent number of classi cations. The network consists of two convolutional pooling layers, with one fully connected hidden layer. The features that are output by the hidden layer are used for binary classi cation by a logistic regression classi er. The architecture of the system is shown in Figure 1. The speci cations of the convolutional-pooling layers are detailed in Table 1. Other hyperparameters for the CNN are detailed in Table 2.
As mentioned earlier the CNN requires a great deal of computational resource to run. We initially began training the four models on a CPU-only solution and despite it being a very powerful machine6, it took approximately 90 minutes to train a model for a single epoch (albeit, training four models simultaneously). Fortunately, we were given an opportunity to run these models on a system with two Nvidia K20 GPUs. Even training the four models simultaneously (two per GPU), it only took approximately 11 minutes to train a model for a single epoch { an 8-fold speedup. We planned to submit a single run, having trained each of the four classi ers for 100 epochs. This process would have taken over a week on a CPU-only system, instead it took less than a day on the GPU server. 4
The test results for our submission as supplied by ImageCLEF are displayed in Table 4.
We believe that that these results can be signi cantly improved upon by making use of a variety of techniques. Primarily we would want to continue to explore training the CNNs using GPUs, because as we have demonstrated, the performance increase is non-trival and allows us to expand our hyperparamter and architecture search. Recti ed Linear Units (ReLUs), as opposed the Tanh units used in our network are also known to improve training performance [17, 18].
Although this network is very capable of learning quality representations of the MNIST dataset, it is both less deep and less dense than networks used to achieve state-of-the-art results in more sophisticated tasks [8]. For instance, Krizhevsky et. al. [8] used a network with 2 convolutional-max pooling layers, 3 convolutional layers and 3 fully connected layers, all of which were more neurondense that ours, to achieve their result in ImageNET 2012. Improved training performance will allow us to implement a larger and deeper network along these lines.
Larger and deeper networks introduce issues with over tting, but we believe this can be controlled using well-tried techniques such as dropout [8, 13, 19], data augmentation [8, 20] and unsupervised pretraining [21, 22].
This work was supported in part by a Microsoft Azure for Research grant, which provided the cloud infrastructure to conduct our experiments. 8. A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet Classi cation with Deep Convolutional Neural Networks, in Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 10971105. 9. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, ImageNet: A largescale hierarchical image database, in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, 2009, pp. 248255. 10. J. Kalpathy-Cramer, A. G. S. de Herrera, D. Demner-Fushman, S. Antani, S.
Bedrick, and H. Mller, Evaluating performance of biomedical image retrieval systemsAn overview of the medical image retrieval task at ImageCLEF 20042013, Comput.
Med. Imaging Graph., vol. 39, pp. 5561, 2015. 11. H. Mller, N. Michoux, and D. Bandon, A review of content-based image retrieval systems in medical applicationsclinical bene ts and future directions, International journal of, 2004. 12. M. S. S. Faruque, M. S. Shahriar Faruque, B. Shourav, M. Kazi Mohammed, H. Mahady, and M. Ashraful Amin, Teaching & Learning System for Diagnostic Imaging Phase I: X-Ray Image Analysis & Retrieval, in Proceedings of the 7th International Conference on Computer Supported Education, 2015. 13. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A Simple Way to Prevent Neural Networks from Over tting, J. Mach.
Learn. Res., vol. 15, no. 1, pp. 19291958, Jan. 2014. 14. Y. Bengio, Practical recommendations for gradient-based training of deep architectures, arXiv [cs.LG], 24-Jun-2012. 15. D. Ciresan, U. Meier, J. Masci, and J. Schmidhuber, A committee of neural networks for tra c sign classi cation, in Neural Networks (IJCNN), The 2011 International Joint Conference on, 2011, pp. 19181921. 16. Y. Lecun, L. Bottou, Y. Bengio, and P. Ha ner, Gradient-based learning applied to document recognition, Proc. IEEE, vol. 86, no. 11, pp. 22782324, Nov. 1998. 17. V. Nair and G. E. Hinton, Recti ed linear units improve restricted boltzmann machines, in Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, pp. 807814. 18. A. L. Maas, A. Y. Hannun, and A. Y. Ng, Recti er Nonlinearities Improve Neural
Network Acoustic Models, W|&CP, vol. 28, 2013. 19. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, Improving neural networks by preventing co-adaptation of feature detectors, arXiv [cs.NE], 03-Jul-2012. 20. Classifying plankton with deep neural networks, Sander Dieleman. [Online]. Available: http://benanne.github.io/2015/03/17/plankton.html. [Accessed: 30-May2015]. 21. X. Glorot, A. Bordes, and Y. Bengio, Domain adaptation for large-scale sentiment classi cation: A deep learning approach, in Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, pp. 513520. 22. Y. Bar, I. Diamant, L. Wolf, and H. Greenspan, Deep learning with non-medical training used for chest pathology identi cation, in SPIE Medical Imaging, 2015, p. 94140V94140V7.