Deep Learning in Medical Imaging Survey

Abstract

Since years ago and currently, the world has witnessed great development and interest in the fields of Machine learning, Deep learning, which provides solutions at all levels, especially in medical image analysis. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in Magnetic Resonance Imaging (MRI). First, a brief introduction of deep learning and imaging modalities of MRI images is given. Then, common deep learning architectures are introduced. Next, deep learning applications of MRI images, such as image detection, image registration, image segmentation, and image classification are discussed. Subsequently, the deep learning tools in the applications of MRI images are presented. Finally; the limitation and future of Deep learning and a small conclusion.

Keywords

• First Keyword
• Second Keyword
• Third Keyword

-

Artificial intelligence is the branch of computer science devoted to creating systems toper form tasks that ordinarily require human intelligence. Artificial intelligence [ 13 ] is not only a field of computer science that was created in the 1950s but also a thriving field with many practical applications and research hotspots. Artificial intelligence attempts to simulate human intelligence and produce a new intelligent machine that would be able to process information with human consciousness, behavior, and thinking. Its ultimate goal is to develop brain-like robots. Artificial intelligence has been applied to many fields, such as image analysis, natural language processing, robotics, and expert systems .Machine learning [ 4-6 ] is the subfield of artificial intelligence in which algorithms are trained to perform tasks by learning patterns from data rather than by explicit programming [ 7 ]. Machine learning involves a number of disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. Machine learning mainly uses induction and synthesis to make computers acquire new knowledge by simulating human learning behavior and then reorganizes the existing knowledge to continually improve computer performance. Machine learning has also been widely applied in many fields, such as computer-aided disease diagnosis[ 8-10 ] ,bioinformatics[ 11-13 ], and computer vision[ 14-16 ]. Medical image analysis and interpretation are fundamental cognitive tasks of a diagnostic radiologist. Effective computer automation of these tasks has historically been difficult despite technical advances in computer vision, a discipline dedicated to the problem of imparting visual understanding to a computer system. Recently, however, computer science researchers using a technique called deep learning have demonstrated breakthrough performance improvements in a variety of complex tasks, including image classification, object detection, speech recognition, language translation, natural language processing, and playing games [ 17,18 ]. However, deep learning [ 19 ] has overcome this obstacle by incorporating the feature engineering step into a learning step. That is, instead of extracting features manually, deep learning requires only a set of data with minor preprocessing, if necessary, and then discovers the informative representations in a self-taught manner [ 20,21 ].

With the deepening of artificial neural networks [ 22 ], the concept of deep learning [ 23,24 ] has been proposed. Deep learning is not only an improvement in artificial neural networks, but also a new field in machine learning research [ 25-30 ]. The successful application of deep learning brings machine learning closer to artificial intelligence. The idea of the artificial neural networks arises from our understanding of the human brain, which comprises interconnections between neurons. The difference between artificial neural networks and the human brain is that any neuron in the human brain is connected to other neurons via a specific physical path, whereas neural networks contain discrete layers, connections, and data propagation directions. Since deep learning consists of more hidden layers in comparison to artificial neural networks, a more abstract high-level feature representation for different classes is formed by using multiple hidden layers to combine low-level features. Similar to artificial intelligence, deep learning also attempts to build and simulate the human brain to analyze the learning process of the neural network, which simulates the learning mechanism of the human brain when it attempts to understand unknown concepts. The deep learning system has been widely deployed in Google’s commercial products, such as Google Photos, Google Search, and Google Street View. Feature representation plays an important role in medical image processing and analysis. As a technology, deep learning methods have two obvious advantages in feature representation, as follows:  Deep learning can be used to automatically find features from a given dataset for each specific application. In general, traditional feature extraction methods are based on some prior knowledge to extract features in a particular application. Thus, these methods are semi- automatic learning methods.  Deep learning can find new features that are suitable to specific applications, but have never been previously discovered by researchers. Traditional feature extraction methods are often limited by some a priori knowledge, which can only extract some features associated with a particular application.

Additionally, the two elements that affect the results of medical image processing and analysis, are image acquisition and image interpretation, as follows:  Image acquisition: As we all know, the better the image quality, the better the results obtained in image processing and analysis. However, the quality of the image depends on image acquisition; therefore, the better the image acquisition, the higher the image quality. Magnetic Resonance Imaging (MRI) does not only have the characteristics of non-invasive and good soft tissue contrast, but also does not expose subjects to high ionizing radiation. Since MRI can provide a lot of invaluable information about tissue structures, such as shape, size, and localization, it is attracting more and more attention for clinical routine and computer-aided diagnosis [ 31-33 ]. Therefore, in this article, we focus on MRI images.  Image interpretation: In clinical practice, most medical image interpretations are basically performed by clinicians to determine whether the subjects are abnormal. However, due to limitations with regard to the clinician’s personal skills, subjectivity, energy, and other factors, the medical image interpretations by clinicians often differ significantly. To obtain accurate image interpretation results, it is imperative to develop an automatic image interpretation system that includes many functions, such as image detection, image registration, image segmentation, and image classification. To realize this system, many machine learning methods have been widely applied. However, due to the fact that deep learning architectures can obtain highlevel latent features, many researchers have applied deep learning architectures to the development of this automatic image interpretation system. Therefore, in this survey, we focus on deep learning the subjects are abnormal. However, due to limitations with regard to the clinician’s personal skills, subjectivity, energy, and other factors, the medical image interpretations by clinicians often differ significantly. To obtain accurate image interpretation results, it is imperative to develop an automatic image interpretation system that includes many functions, such as image detection, image registration, image segmentation, and image classification. To realize this system, many machine learning methods have been widely applied. However, due to the fact that deep learning architectures can obtain high-level latent features, many researchers have applied deep learning architectures to the development of this automatic image interpretation system. therefore, in this survey, we focus on deep learning.

Deep Learning Architectures

Deep learning systems encode features by using an architecture of artificial neural networks, an approach consisting of connected nodes inspired by biologic neural networks. Systematic methods to train neural networks on the basis of a process called back-propagation were developed in the 1980s [ 34 ]. However, success in training the deep multilayer neural networks needed for hierarchical representations was limited by the difficulty
of the underlying optimization problem as well as the limits of the computing hardware of that early era. Consequently, research attention in machine learning for the next few decades drifted toward other techniques such as kernel methods and decision trees. Deep learning is a type of representation learning in which the algorithm learns a composition of features that reflect a hierarchy of structures in the data. Complex representations are expressed in terms of simpler representations [ 35 ].These deep learning systems propose an end-to-end approach by learning simple features (such as signal intensity, edges, and textures) as components of more complex features such as shapes, lesions, or organs, therefore leveraging the compositional nature of images(see Fig. 1).

Artificial neural network

Artificial neural networks (ANNs) is one of the most famous machine learning models, introduced already in the 1950s, and actively studied since[36,Chapter 1.2]. Roughly, a neural network consists of a number of connected computational units, called neurons, arranged in layers. There’s an input layer where data enters the network, followed by one or more hidden layers transforming the data as it flows through, before ending at an output layer that produces the neural network’s predictions. The network is trained to output useful predictions by identifying patterns in a set of labeled training data, fed through the network while the outputs are compared with the actual labels by an objective function. During training the network’s parameters – the strength of each neuron – is tuned until the patterns identified by the network result in good predictions for the training data. Once the pat- terns are learned, the network can be used to make predictions on new, unseen data, i.e. generalize to new data.it has long been known that ANNs are very flexible, able to model and solve complicated problems, but also that they are difficult and very computationally expensive to train. This has lowered their practical utility and led people to, until recently, focus on other machine learning models. But by now, artificial neural networks form one of the dominant methods in machine learning, and the most intensively studied.

In the brain, neurons exchange information via chemical and electrical synapses. Electrochemical signals are propagated from the synaptic area through the dendrites toward the soma, the body of the cell (see Fig. 2, Fig. 3). When a certain excitation threshold is reached, the cell releases an activation signal through its axon toward synapses with neighboring neurons. Complex signals can be encoded by networks of neurons on the basis of this paradigm; for instance, a hierarchy of neurons in the visual cortex is able to detect edges by combining signals from independent visual receptors.

Conceptual analogy between components of biologic neurons (Fig. 2) and artificial neurons (Fig. 3). The concept of neural networks stems from biologic inspiration. (Fig. 2) In the visual cortex, there is a neural network able to detect edges from what is seen by the retina (gray circles = receptive areas of the retina). When the inner parts (smaller circles) of the three receptors are activated simultaneously, the simple cell neuron integrates the three signals and transmits an edge detection signal. (Fig. 3) An artificial neural network is composed of interconnected artificial neurons. Each artificial neuron implements a simple classifier model, which outputs a decision signal based on a weighted sum of evidences, and an activation function, which integrates signals from previous neurons. Hundreds of these basic computing units are assembled together to build an artificial neural network computing device. The weights of the network are trained via a learning algorithm where pairs of input signals and desired output decisions are presented, much like the brain, which relies on external sensory stimuli to learn to achieve specific tasks.

Artificial neural networks are inspired by this biologic process.

The “deep” aspect of deep learning refers to the multilayer architecture of multilayer perceptron’s (see Fig 4). In machine learning, artificial neural networks are a family of models that mimic the structural elegance of the neural system and learn patterns inherent in observations. The perceptron [ 37 ] is the earliest trainable neural network with a single-layer architecture, composed of an input layer and an output layer. 2.3

Stacked auto encoders

An auto encoder [ 38-40 ] is a simple deep feed forward network, which includes an input layer, a hidden layer, and an output layer. An auto-encoder or auto-associate [41] is a special type of two- layer neural network that learns a latent or compressed representation of the input by minimizing the reconstruction error between the input and output values of the network, namely the reconstruction of the input from the learned representations. Because of its simple, shallow structure, a single-layer auto encoder’s representational power is very limited. 2.4

Deep belief networks

A restricted Boltzmann machine (RBM) [ 42 ] is a single-layer undirected graphical model with a visible layer and a hidden layer. It assumes symmetric connectivity between visible and hidden layers, but no connections among units within the same layer. Because of the symmetry of the connectivity’s, it can generate input observations from hidden representations. Therefore, an RBM naturally becomes an autoencoder [ 42, 43 ], and its parameters are usually trained by use of a contrastive divergence algorithm [ 44 ] so as to maximize the log likely hood of observations. 2.5

Convolutional neural networks

Convolutional neural networks [ 45-48 ] are also deep feed forward networks, and have been widely used in recognition tasks, such as document recognition [ 49 ], handwriting recognition [ 50 ], and image classification[ 51-54 ]. The only difference between the fully connected feed forward neural networks and the convolution neural networks is that the two adjacent layers of the two neural networks are connected in different ways. The former only has some nodes connected between the adjacent two layers, while the latter has all nodes connected between the adjacent two layers. The biggest problem of using a fully connected feed forward neural network is that there are too many parameters for the network. In general, increasing the parameters will not only lead to slower calculation speed, but will also lead to over fitting problems. To effectively reduce the number of parameters in the neural networks, more reasonable neural network architectures are required. Therefore, convolutional neural networks were proposed to achieve this goal. Convolutional neural networks include two kernel layers, namely, the convolutional and pooling layers, as follows:  Convolutional Layer: Only a small patch of the previous layer is used as the input of each node in the convolutional layer, and the size of the small patch is often 3 × 3 or 5 × 5. The convolutional layer attempts to analyze each small patch of the neural network in depth, which results in the higher abstraction of feature representation.  Pooling Layer: There is often a pooling layer followed by the convolutional layer.

The pooling layer can effectively reduce the size of the matrix from the previous convolutional layer; thus, it can reduce the number of parameters in the neural network. Therefore, the use of pooling layers can not only speed up the calculation, but can also prevent the problem of over fitting. In recent years, many deep learning methods have been proposed for application in the field of MRI image processing and analysis, such as image detection, image registration, image segmentation, and image classification. All of these can be formulated as feature representation problems, and can thus be solved effectively by using deep learning methods to find an effective set of features. In this section, we review the recent progress of applying deep learning architectures in the image detection, image registration, image segmentation, and image classification of MRI images. Image detection plays an important role in computer- aided detection routines. Its main purpose is to find the tissues of interest, and then measure and analyze whether these tissues produce lesions. Localization and interpolation of anatomical structures in medical images are key steps in the radiological workflow. Radiologists usually accomplish these tasks by identifying certain anatomical signatures, namely image features that can distinguish one anatomical structure from others. Is it possible for a computer to automatically learn such anatomical signatures? The success of such methods essentially depends on how many anatomy signatures can be extracted by computational operations.

Image registration

Image registration is the process of matching and superimposing two or more images at different times, different sensors (such as imaging equipment) or different conditions (such as illumination, position, and angle) [ 56 ].

Image registration has been widely applied in medical image processing. Its main purpose is to combine various medical images, which display their information in the same image, and thereby provide multiple information for clinical diagnosis. 3.3

Image segmentation

Automatic tissue segmentation in MRI images is of great importance in modern medical research and clinical routines. Many medical image segmentation challenges have been held to encourage the development of automatic segmentation techniques, such as Ischemic Stroke Lesion Segmentation, Multimodal Brain Tumor Image Segmentation, MR Brain Image Segmentation, and cardiac MR Left Ventricle (LV) segmentation. Since most brain tumors can affect a patient’s health, and even shorten their life expectancy, automatic and reliable segmentation techniques for removing brain tumors are required. However, most brain tumors have large spatial and structural variability, which makes them difficult to segment. Thus, automatic and reliable segmentation has become a challenging problem. To address the problem, many deep learning-based brain tumor segmentation methods have been proposed [ 57-62 ]. 3.4

Image classification

Image classification plays an important role in automatic disease diagnosis and cognitive recognition, such as the classification of different severity diseases and the recognition of different brain activities. Many deep learning methods have also been proposed for performing image classification tasks in MRI images [ 63-65 ]. 4

Deep Learning Tools

Deep learning is a complex technology. To achieve the abovementioned deep learning architectures, researchers need to spend a lot of time and energy. Fortunately, in recent years, many deep learning tools have been developed as shown in Table 1. These tools are convenient for researchers; thus, they promote the application of deep learning architectures. Some common and widely used deep learning tools are shown in Table 1.

Tools

Deep

LearnToolbox

Torch

Torch

Theano

Pylearn2

Keras

CNTK

MXNet

Chainer

SINGA

Links http://torch.ch/
 [ 66 ] [ 67 ] [ 68 ] [ 69 ] [ 70 ] [ 71 ] [ 72 ] [ 74 ] [ 75 ] [ 76 ] [ 77 ] [ 78 ] [ 79 ] 5

Limitations of Deep Learning

Despite the variety of recent successes of deep learning, there are limitations in the application of the technique. First, deep learning is not the optimal machine learning technique for all data analysis problems.

For problems in which data are well structured or optimal features are well- defined, other simpler machine learning methods such as logistic regression, support

TensorFlow https://www.tensorflow.org/

Deeplearning4j https://deeplearning4j.org/

MatConvNet

http://www.vlfeat.org/matconvnet/ maxDNN https://github.com/eBay/maxDNN https://www.microsoft.com/enus/research/product/cognitive- [ 73 ] toolkit/ https://github.com/dmlc/mxnet http://chainer.org/ http://www.comp.nus.edu.sg/ dbsystem/singa/̃ vector ma- chines, and random forests are typically easier to apply and more effective [ 80 ].

Even in computer vision, where CNNs have become a dominant method, there are important limitations for deep learning. The most prominent limitation is that deep learning is an intensely data- hungry technology; learning weights for a large network from scratch requires a very large number of labeled examples to achieve accurate classification. However, unlike traditional approaches to computer vision and machine learning, which do not scale well with dataset size, deep learning does scale well with large datasets.

Deep learning systems currently excel in emulating the kind of human judgment that is based purely on pattern recognition, where the most informative patterns can be discerned from previous training. However, no finite training set can fully represent the variety of cases that might be seen in clinical practice. More complex radiology interpretation problems typically require deductive reasoning using knowledge of pathologic processes and selective integration of information from prior examinations or the patient’s health record. It is presently not clear how to train a deep learning system to emulate these more complex thought processes. 6

Future Directions

The role of deep learning and its application to the practice of radiology must still be defined. Deep learning systems may be conceived as a new form of diagnostic test with various clinical usage scenarios [ 81 ]. A triage approach would run these automated image analysis systems in the background to detect life-threatening conditions or search through large amounts of clinical, genomic, or imaging data [ 82 ]. A replacement approach would use these systems for generating figure captions [ 83 ] or even fully automated interpretation of imaging examinations. An add-on approach would support the radiologist by performing time- consuming tasks such as lesion segmentation to assess total tumor burden.

Conclusion

In summary, the aim of this survey was to provide valuable insights for researchers, with regard to applying deep learning architectures in the field of MRI- based research.

Deep learning is a powerful and generic artificial intelligence technique that can solve image detection, recognition, and classification tasks that previously required human intelligence. The introduction of deep learning techniques in radiology will likely assist radiologists in a variety of diagnostic tasks.

References

1. R. A. Brooks , Intelligence without representation, Artif. Intell. , vol. 47 , nos . 1- 3 , pp. 139 - 159 , 1991 .

2. N. J. Nilsson , Principles of Artificial Intelligence . Los Altos, CA, USA: Morgan Kaufmann, 2014 .

3. P. R. Cohen and E. A. Feigenbaum , The Handbook of Artificial Intelligence . Los Altos, CA, USA: Butterworth- Heinemann, 2014 .

4. P. Baldi and S. Brunak , Bioinformatics: The Machine Learning Approach . Cambridge, MT, USA: MIT Press, 2001 .

5. C. Bishop , Pattern Recognition and Machine Learning . Springer-Verlag, 2006 .

6. R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Machine Learning: An Artificial Intelligence Approach . Berlin, Germany: Springer Science & Business Media , 2013 .

7. EricksonBJ,KorfiatisP,AkkusZ,KlineTL. Machinelearning for medical imaging . RadioGraphics 2017 ; 37 ( 2 ): 505 - 515 .

8. J. Liu , M. Li , W. Lan , F.X. Wu , Y. Pan ,andJ.X. Wang, Classification of Alzheimer's disease using whole brain hierarchical network , IEEE/ACM Trans. Comput. Biol. Bioinform ., doi: 10.1109/TCBB. 2016 . 2635144 .

9. J. Liu , M. Li , Y. Pan , F.X. Wu , X.G. Chen ,andJ.X. Wang, Classification of schizophrenia based on individual hierarchical brain networks constructed from structural MRI images , IEEE Trans. NanoBiosci. , vol. 16 , no. 7 , pp. 600 - 608 , 2017 .

10. J.Liu , J.X. Wang , B. Hu , F.X. Wu ,andY.Pan, Alzheimer's disease classification based on individual hierarchical networks constructed with 3-D texture features , IEEE Trans. NanoBiosci. , vol. 16 , no. 6 , pp. 428 - 437 , 2017 .

11. W.Lan, M. Li , K.J. Zhao , J. Liu , F.X. Wu , Y. Pan ,and J. X. Wang , LDAP: A web server for LncRNA-disease association prediction , Bioinformatics , vol. 33 , no. 3 , pp. 458 - 460 , 2017 .

12. W. Lan , J.X. Wang , M. Li , J. Liu , Y.H. Li , F.X. Wu ,and Y. Pan , Predicting drug-target interaction using positive- unlabeled learning , Neurocomputing , vol. 206 , pp. 50 - 57 , 2016 .

13. W. Lan , J. X. Wang , M. Li , J. Liu , F. X. Wu , and Y. Pan , Predicting microRNA-disease associations based on improved microRNA and disease similarities , IEEE/ACM Trans. Comput. Biol. Bioinform ., doi: 10.1109/TCBB. 2016 . 2586190 .

14. G. R. Bradski and A. Kaehler , Learning OpenCV: Computer Vision with the OpenCV Library . Sebastopol, CA, USA: O 'Reilly Media , Inc., 2008 .

15. A. Vedaldi and B. Fulkerson , Vlfeat: An open and portable library of computer vision algorithms , in Proc.18th ACM Int. Conf. Multimedia , Firenze, Italy, 2010 , pp. 1469 - 1472 .

16. R. Cipolla , S. Battiato , and G. M. Farinella , Machine Learning for Computer Vision . Springer, 2013 .

17. LeCun Y , Bengio Y , Hinton G . Deep learning . Nature 2015 ; 521 ( 7553 ): 436 - 444 .

18. Silver D , Huang A , Maddison CJ , et al. Mastering the game of Go with deep neural networks and tree search . Nature 2016 ; 529 ( 7587 ): 484 - 489 .

19. Schmidhuber J. 2015 . Deep learning in neural networks: an overview . Neural Netw . 61 : 85 - 117 .

21. LeCun Y , Bengio Y , Hinton G. 2015 . Deep learning . Nature 521 : 436 - 44 .

22. B. Yegnanarayana , Artificial Neural Networks. New Delhi , India: PHI Learning Pvt . Ltd., 2009 .

24. Y. Bengio , Learning deep architectures for AI, Found . Trends Mach. Learn., vol. 2 , no. 1 , pp. 1 - 127 , 2009 .

26. X. Glorot , A. Bordes , and Y. Bengio , Domain adaptation for large-scale sentiment classification: A deep learning approach , in Proc. 28th Int. Conf. Machine Learning , Bellevue, WA, USA, 2011 , pp. 513 - 520 .

27. Q. V. Le , J. Ngiam , A. Coates , A. Lahiri , B. Prochnow , and A. Y. Ng , On optimization methods for deep learning , in Proc. 28th Int. Conf. Machine Learning , Bellevue, WA, USA, 2011 , pp. 265 - 272 .

28. Y. Bengio , A. Courville , and P. Vincent , Representation learning: A review and new perspectives , IEEE Trans. Pattern Anal. Mach . Intell., vol. 35 , no. 8 , pp. 1798 - 1828 , 2013 .

29. Y. LeCun, Y. Bengio , and G. Hinton , Deep learning , Nature , vol. 521 , no. 7553 , pp. 436 - 444 , 2015 .

30. Z. Yu , N. Yu , Y. Pan , and T. R. Li , A novel deep learning network architecture with crosslayer neurons , in Proc. 2016 IEEE Int. Conf. Big Data and Cloud Computing (BDCloud) , Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom)(BDCloud-SocialCom- SustainCom) , Atlanta, GA , USA, 2016 , pp. 111 - 117 .

31. J.Liu , M. Li , J.X. Wang , F.X. Wu ,T.M.Liu,andY.Pan, A survey of MRI-based brain tumor segmentation methods , Tsinghua Sci. Technol ., vol. 19 , no. 6 , pp. 578 - 595 , 2014 .

32. J.Liu , J.X. Wang , Z.J. Tang , B. Hu , F.X. Wu ,and Y. Pan , Improving Alzheimeres disease classification by combining multiple measures , IEEE/ACM Trans. Comput. Biol. Bioinform ., doi: 10.1109/TCBB. 2017 . 2731849 .

33. Rumelhart DE , Hinton GE , Williams RJ . Learning repre- sentations by back-propagating errors . Cognitive Modeling 1988 ; 5 ( 3 ): 1 .

34. Goodfellow I , Bengio Y , Courville A . Deep learning . Cam- bridge, Mass: MIT Press, 2016 ; 1 - 26 .

35. Goodfellow I , Bengio Y , Courville A . Deep learning . MIT Press; 2016 . www.deeplearningbook.org.

36. Rosenblatt F. 1958 . The perceptron: a probabilistic model for information storage and organization in the brain . Psychol. Rev . 1958 : 65 - 386 .

37. H. Bourlard and Y. Kamp , Auto-association by multilayer perceptrons and singular value decomposition , Biol. Cybern. , vol. 59 , nos . 4 & 5 , pp. 291 - 294 , 1988 .

38. G. E. Hinton and R. S. Zemel , Autoencoders, minimum description length, and Helmholtz free energy , in Proc. 6th Int. Conf. Neural Information Processing Systems , Denver, CO, USA, 1994 , p. 3 .

39. G. Alain and Y. Bengio , What regularized auto-encoders learn from the data-generating distribution , J. Mach. Learn. Res. , vol. 15 , no. 1 , pp. 3563 - 3593 , 2014 .

40. Bourlard H , Kamp Y. 1988 . Auto-association by multilayer perceptrons and singular value decomposition . Biol. Cybern . 59 : 291 - 94

42. HitonGE,SalakhutdinovRR. 2006 .Reducingthedimensionalityofdatawithneuralnetworks.Sci ence 313 : 504 - 7

43. Hinton GE . 2002 . Training products of experts by minimizing contrastive divergence . Neural Comput . 14 : 1771 - 800

44. Y. LeCun and Y. Bengio , Convolutional networks for images, speech, and time-series, in The Handbook of Brain Theory and Neural Networks, M. A . Arbib, ed. Cambridge, MA, USA: MIT Press, 1995 ,.

45. H. Lee , R. Grosse , R. Ranganath , and A. Y. Ng , Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , in Proc. 26th Annual Int. Conf. Machine Learning , Montreal, Canada, 2009 , pp. 609 - 616 .

46. Y. LeCun, K. Kavukcuoglu, and C. Farabet , Convolutional networks and applications in vision . in Proc. 2010 IEEE Int. Symposium on Circuits and Systems (ISCAS) , Paris, France, 2010 , pp. 253 - 256 .

47. J. T. Springenberg , A. Dosovitskiy , T. Brox , and M. Riedmiller , Striving for simplicity: The all convolutional net , arXiv preprint arXiv: 1412.6806 , 2014 .

48. Y. LeCun, L. Bottou, Y. Bengio , and P. Haffner , Gradient- based learning applied to document recognition , Proc. IEEE , vol. 86 , no. 11 , pp. 2278 - 2324 , 1998 .

49. P. Y. Simard , D. Steinkraus , and J. C. Platt , Best practices for convolutional neural networks applied to visual document analysis . in Proc. 7th Int. Conf. Document Analysis and Recognition , Edinburgh, UK, 2003 , pp. 958 - 962 .

50. A. Krizhevsky , I. Sutskever , and G. E. Hinton , ImageNet classification with deep convolutional neural networks , in Proc. 25th Int. Conf. Neural Information Processing Systems , Lake Tahoe, NV , USA, 2012 , pp. 1097 - 1105 .

51. K. Simonyan and A. Zisserman , Very deep convolutional networks for large-scale image recognition , arXiv preprint arXiv: 1409.1556 , 2014

52. K. M. He , X. Y. Zhang , S. Q. Ren , and J. Sun , Deep residual learning for image recognition , arXiv preprint arXiv: 1512.03385 , 2015 .

53. K. M. He , X. Y. Zhang , S. Q. Ren , and J. Sun , Identity mappings in deep residual networks , arXiv preprint arXiv: 1603.05027 , 2016 .

55. B. Zitova ́ and J. Flusser , Image registration methods: A survey, Image Vis . Comput. , vol. 21 , no. 11 , pp. 977 - 1000 , 2003 .

56. D. Zikic , Y. Ioannou , A. Criminisi , and M. Brown , Segmentation of brain tumor tissues with convolutional neural networks , Proceedings MICCAI Workshop on Multimodal Brain Tumor Segmentation Challenge , Boston, MA, USA, 2014 , pp. 36 - 39 .

57. G. Urban, M. Bendszus , F. Hamprecht , and J. Kleesiek , Multi-modal brain tumor segmentation using deep convolutional neural networks , in Proc. MICCAI BraTS (Brain Tumor Segmentation) Challenge , 2014 , pp. 31 - 35 .

58. M. Lyksborg , O. Puonti , M. Agn , and R. Larsen , An ensemble of 2D convolutional neural networks for tumor segmentation , in Proc. 19th Scandinavian Conf. Image Analysis , Copenhagen, Denmark, 2015 , pp. 201 - 211 .

59. P. Dvorak and B. H. Menze , Structured prediction with convolutional neural networks for multimodal brain tumor segmentation , in Proc. Multimodal Brain Tumor Image Segmentation Challenge , Munich, Germany, 2015 , pp. 13 - 24

60. S. Pereira , A. Pinto , V. Alves , and C. A. Silva , Brain tumor segmentation using convolutional neural networks in mri images , IEEE Trans. Med . Imaging, vol. 35 , no. 5 , pp. 1240 - 1251 , 2016 .

61. M. Havaei , A. Davy , D. Warde-Farley , A. Biard , A. Courville , Y. Bengio , C. Pal , P. M. Jodoin , and H. Larochelle , Brain tumor segmentation with deep neural networks , Med . Image Anal., vol. 35 , pp. 18 - 31 , 2017 .

62. H. I. Suk , S. W. Lee , and D. G. Shen , Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis , NeuroImage , vol. 101 , pp. 569 - 582 , 2014

63. S. Koyamada , Y. Shikauchi , K. Nakae , M. Koyama , and S. Ishii , Deep learning of fMRI big data: A novel approach to subject-transfer decoding , arXiv preprint arXiv: 1502.00093 , 2015 .

64. 64. W. H. Pinaya , A. Gadelha , O. M. Doyle , C. Noto , A. Zugman , Q. Cordeiro , A. P. Jackowski , R. A. Bressan , and J. R. Sato , Using deep belief network modelling to characterize differences in brain morphometry in schizophrenia , Sci. Rep ., vol. 6 , p. 38897 , 2016 .

65. R. B. Palm , Prediction as a candidate for learning deep hierarchical models of data, Master's thesis , Technical University of Denmark, Denmark, 2012 .

66. Y. Jia , E. Shelhamer , J. Donahue , S. Karayev , J. Long , R. Girshick , S. Guadarrama , and T. Darrell , Caffe: Convolutional architecture for fast feature embedding , in Proc. 22nd ACM Int. Conf. Multimedia , Orlando, FL, USA, 2014 , pp. 675 - 678 .

67. R. Collobert , K. Kavukcuoglu , and C. Farabet , Torch7: A matlab-like environment for machine learning , in BigLearn, NIPS Workshop, No. EPFL-CONF-192376 , 2011 .

68. R. Al-Rfou , G. Alain , A. Almahairi , C. Angermueller , D. Bahdanau , N. Ballas , F. Bastien , J. Bayer , A. Belikov , A. Belopolsky , et al., Theano: A python framework for fast computation of mathematical expressions , arXiv preprint arXiv: 1605.02688 , 2016 .

69. I. J. Goodfellow , D. Warde-Farley , P. Lamblin , V. Dumoulin , M. Mirza , R. Pascanu , J. Bergstra , F. Bastien , and Y. Bengio , Pylearn2: A machine learning research library , arXiv preprint arXiv: 1308.4214 , 2013 .

70. F. Chollet , Keras: Theano-based deep learning library , Code, https://github.com/fchollet.Documentation:http: //keras.io, 2015 .

71. M. Abadi , A. Agarwal , P. Barham , E. Brevdo , Z. F. Chen , C. Citro , G. S. Corrado , A. Davis , J. Dean , M. Devin , et al., TensorFlow: Large-scale machine learning on heterogeneous distributed systems , arXiv preprint arXiv: 1603.04467 , 2016 .

72. D. Yu , A. Eversole , M. Seltzer , K. S. Yao , O. Kuchaiev , Y. Zhang , F. Seide , Z. H. Huang , B. Guenter , H. M. Wang , et al., An introduction to computational networks and the computational network toolkit , Technical report, Tech. Rep. MSR-TR-2014-112 , 2014 .

73. T. Q. Chen , M. Li , Y. T. Li , M. Lin , N. Y. Wang , M. J. Wang , T. J. Xiao , B. Xu , C. Y. Zhang , and Z. Zhang, MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems , arXiv preprint arXiv: 1512.01274 , 2015 .

74. S. Tokui , K. Oono , S. Hido , and J. Clayton , Chainer: A next-generation open source framework for deep learning , in Proc. Workshop on Machine Learning Systems (LearningSys) in the Twenty-Ninth Annual Conf. Neural Information Processing Systems (NIPS) , 2015 .

75. Deeplearning4j Development Team, Deeplearning4j: Open-source distributed deep learning for the JVM , Apache Software Foundation License 2 .0, http://deeplearning4j.org, 2017 .

76. B. C. Ooi , K. L. Tan , S. Wang , W. Wang , Q. C. Cai , G. Chen, J. Y. Gao , Z. J. Luo , A. K. H. Tung , Y. Wang , et al., SINGA: A distributed deep learning platform , in Proc. 23rd ACM Int. Conf. Multimedia , Brisbane, Australia, 2015 , pp. 685 - 688 .

77. A. Vedaldi and K. Lenc , MatConvNet: Convolutional neural networks for MATLAB , in Proc. 23rd ACM Int. Conf. Multimedia , Brisbane, Australia, 2015 , pp. 689 - 692 .

78. A. Lavin, MaxDNN: An efficient convolution kernel for deep learning with maxwell GPUs , arXiv preprint arXiv: 1501.06633 , 2015 .

79. Domingos P. Plenary panel: is deep learning the new 42? https ://youtu.be/furfdqtdAvc?t=21m35s. Published Septem- ber 1 , 2016 . Accessed March 27, 2017 .

80. Bossuyt PM , Irwig L , Craig J , Glasziou P . Comparative accuracy: assessing new tests against existing diagnostic pathways . BMJ 2006 ; 332 ( 7549 ): 1089 - 1092 .

81. Summers RM . Progress in fully automated abdominal CT interpretation . AJR Am J Roentgenol 2016 ; 207 ( 1 ): 67 - 79 .

82. Vinyals O , Toshev A , Bengio S , Erhan D . Show and tell: a neural image caption generator . arXiv:14114555v1 . https:// arxiv.org/abs/1411.4555. Published November 17, 2014 . Accessed October 2 , 2017 .

83. Christ PF , Ettlinger F , Grün F , et al. Automatic liver and tumor segmentation of CT and MRI volumes using cas- caded fully convolutional neural networks . arXiv preprint arXiv:1702 .05970. https://arxiv.org/abs/1702.05970. Pub- lished February 20 , 2017 . Accessed October 2 , 2017 .