=Paper=
{{Paper
|id=Vol-2348/paper15
|storemode=property
|title=Machine Learning for Melanoma Management in a Clinical Setting
|pdfUrl=https://ceur-ws.org/Vol-2348/paper15.pdf
|volume=Vol-2348
|authors=Paul Walsh,Jennifer Lynch,Brian Kelly,Timothy Manning
|dblpUrl=https://dblp.org/rec/conf/cerc/WalshLKM19
}}
==Machine Learning for Melanoma Management in a Clinical Setting==
https://ceur-ws.org/Vol-2348/paper15.pdf
Smart Healthcare and Safety Systems
Machine Learning for Melanoma Management in a
Clinical Setting
Paul Walsh1,2, Jennifer Lynch2, Brian Kelly2, Timothy Manning2
1
Cork Institute of Technology, Bishopstown, Cork, Ireland
2 NSilico Life Science, UCD, Dublin, Ireland
paul.walsh@nsilico.com
Abstract. This paper describes work in progress on a melanoma management
platform known as Simplicity MDT, which is in use in major hospitals for the
collection, management and analysis of clinical data. It includes a facility for
uploading and managing high resolution digital colour photographs of melanoma
lesions. As the data managed by this platform is structured and annotated, it is
proposed that this could serve as a basis for supervised training datasets for ma-
chine learning. Machine learning models trained using this data can serve the
wider community for screening, diagnostic and prognostic purposes. The pro-
posed machine learning architecture includes integration with a model zoo, which
provides networks that are pre-trained on publicly available datasets. An over-
view of the current system is presented and a roadmap for future developments
is outlined.
Keywords: Machine Learning, Melanoma, Connected Health.
1 Introduction
The motivation for the development of a machine learning driven melanoma man-
agement platform stems from engagement with clinical end users, doctors, oncologists
and other clinical specialists who are searching for innovative solutions to impact the
treatment of melanoma. Melanoma is a malignant tumour of melanocytes with about
160,000 new cases diagnosed annually, with high prevalence among Europeans. This
is a serious health issue and the aim of this research is to impact the diagnosis of this
disease by the proposed integration of machine vision-based classification, through
deep learning technology, with the myriad of information sources in electronic health
records captured through multi-disciplinary team discussions.
Multidisciplinary care is currently accepted as best practice in the delivery of high-
quality cancer management in Ireland and internationally. Multidisciplinary Team
Meetings (MDTMs) take place at regular intervals, where a team comprised of medical
experts across different relevant disciplines come together to discuss patient cases, re-
view treatment, and plan treatments [1]. Care delivered to patients through a multidis-
ciplinary approach results in positive outcomes especially in terms of diagnosis and
treatment planning, patient satisfaction and improved survival rates. Participating in
205
Smart Healthcare and Safety Systems
2
MDTM’s also has positive outcomes for clinicians – centred around the opportunity for
education, improved communication and working relationships. While MDTMs en-
hance healthcare, they are mostly reliant on antiquated paper-based systems and inte-
grated software management tools that are sorely lacking. Moreover, there is a lost op-
portunity for allowing patients to participate in the care pathway. The melanoma man-
agement platform Simplicity MDT provides significant opportunity for gathering clin-
ically labelled digital images of melanoma lesions (see screenshot of system in Figure
1). The system has already managed over 8,000 cancer cases, including over 1,000
melanoma cases with digital imaging. This rich data set provides a basis for machine-
based assessment by allowing health professionals to gather valuable supervised train-
ing data that can be used to augment existing deep learning models to enhance clinical
assessment.
50
45
40 Data A
35 Data B
30
25
20
15
10
5
0
0 5 10 15 20 25 30
Fig. 1. Screenshot of Simplicity MDT with clinically labeled images.
2 Machine Learning for Melanoma Applications
The term machine learning is used to refer to the field of computer science concerned
with developing algorithms that can mine datasets to build statistical models [2]. Ma-
chine learning systems rely on training data that can either be labelled or unlabelled [3].
Labelled data allows researchers to apply “supervised” learning algorithms, which can
progressively train models to find associations between input patterns and their ex-
pected labels [4]. Labelling data can be extremely expensive and time consuming, so
any framework that facilitates the collection and labelling of data can serve as a useful
basis for building supervised machine learning models [5]. Unsupervised machine
learning models on the other hand do not require labelled data, but in turn can’t be used
206
Smart Healthcare and Safety Systems
3
to generalize label mappings for new data, instead finding use in more niche applica-
tions, such as anomaly detection.
Deep neural networks are a class of ML model that come in a variety of architectures,
but all are characterized as having a large number of layers, which are used to detect
low-level features and map these two increasingly complex models in the higher layers
of the network architecture [6]. Deep neural networks have had significant impact on
the classification of skin cancer as outlined in the Nature publication from Estava et al.
[7], where they note that automated classification of skin lesions using images is a chal-
lenging task owing to the fine-grained variability in the appearance of skin lesions. A
class of deep learning networks, known as convolutional neural networks (CNNs), have
demonstrated impact in challenging visual classification tasks in a variety of domains
[8] [9] [10] [11] [12] [13]. Estava et al. developed a CNN for the classification of skin
lesions using digital images, using only raw pixels and disease labels as inputs and
tested its performance against 21 qualified dermatologists on clinically assessed images
with the following binary classification use cases:
• Keratinocyte carcinomas (the most common skin cancer) versus benign sebor-
rheic keratoses.
• Malignant melanomas (the most fatal skin cancer) versus benign nevi.
Their system performed as well as the experts across both tasks, indicating that ma-
chine learning can achieve a level of accuracy comparable to a dermatologist for skin
cancer classification. Estava et al. proposed that deep neural networks deployed to de-
vices such as smart phones, could bring important dermatology assessment to a wider
audience and can possibly deliver universal access to diagnostic care.
We plan to take up this challenge, by designing an integrated cloud based mobile
solution that can enhance automated skin cancer classification by leveraging mobile
edge computing (MEC), which provides cloud computing capabilities at the edge of the
network [14] and optimises performance by bringing computing resources to the data.
Furthermore, recent advances in edge computing will have profound impacts on
healthcare systems [15] [16].
Connected healthcare has already been enhanced with the proliferation of technolo-
gies [17] and mobile hardware is having a positive impact on healthcare, e.g. the mon-
itoring of hypertensive patients with connected blood pressure monitors. Similarly,
there have been a number of smart-phone apps developed for the detection of melanoma
[18] and while they provide an array of features including information, education, clas-
sification, risk assessment and change monitoring, they can only provide limited pro-
cessing power due to the current processing limitations of mobile devices.
Smart mobile networks offer increased processing through mobile edge computing,
which offers low latency, high bandwidth and localized cloud computing capabilities
[19]. A major benefit of 5G technology is the provision of ad hoc local cloud instances
via a mobile network [20], which allows participating institutions to “securely share
patient genomic, imaging and clinical data for potentially lifesaving discoveries. It will
enable large amounts of data from sites all around the world to be analyzed in a dis-
tributed way, while preserving the privacy and security of patient data at each site”
[21].
207
Smart Healthcare and Safety Systems
4
3 Future Work
We aim to develop Simplicity MDT as a system for gathering high quality clinically
labelled digital images and integrate this system with the advances in both mobile edge
computing and deep learning to provide universal platform for assessing suspect skin
cancer lesions. The system will have the following features:
• Use of state-of-the-art pre-trained melanoma machine learning models, based on
evaluations of results using ImageNet, VGG19, ResNet-50 [22], Inception [11]
and newer architectures, including PolyNet [23], ResNeXt [24], Densenet [25],
SENets [26] and DualPathNet [27] [22].
• Machine learning implemented via mobile edge technology in a secure and fed-
erated environment [28]. The use of mobile edge local cloud computing will allow
clinicians in distributed locations, to securely review, edit and discuss cancer
cases in real time within multi-disciplinary teams.
• Ongoing experts “human-in-the-loop” to constantly label, validate and update
models [29] [30]. Our Simplicity MDT software platform serves as a basis for
collecting numerous multivariate labels which can be used to help build predic-
tion and classification models for melanoma images and other datasets.
In addition, a vast range of techniques can be used to enhance classification and
object detection. Having a network of partners, collaborators and consumers can pro-
vide a wealth of access to resources for creating quality labeled data sets.
Acknowledgements
PW is supported by funding through Science Foundation Ireland (SFI) Multi-Gene As-
say Cloud Computing Platform - (16/IFA/4342), BK, JL, PW are funded under H2020
RISE project SemAntically integrating Genomics with Electronic health records for
Cancer CARE (SageCare), grant number 644186.
References
[1] B. L. S. O. D. a. M. R. Kane, " Multidisciplinary team meetings and their impact
on workflow in radiology and pathology departments.," BMC medicine, vol. 5,
no. 1, p. 15, 2007.
[2] A. L. Samuel, "Some Studies in Machine Learning Using the Game of Checkers,"
IBM Journal of Research and Development, vol. 3, no. 3, pp. 210-229, 1959.
[3] B. Christopher, Pattern Recognition and Machine Learning, Springer, 2006.
[4] L. Deng and D. Yu, "Deep Learning: Methods and Applications," Foundations
and Trends® in Signal Processing, vol. 7, no. 3-4, pp. 197-387, 2014.
[5] H. Greenspan, B. v. Ginneken and R. M. Summers, "Guest Editorial Deep
Learning in Medical Imaging: Overview and Future Promise of an Exciting New
208
Smart Healthcare and Safety Systems
5
Technique," IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1153 -
1159, 2016.
[6] Y. LeCun, Y. Bengio and G. Hinton, "Deep learning," Nature , vol. 521, p. 436–
444, 2015.
[7] A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau and S.
Thrun, "Dermatologist-level classification of skin cancer with deep neural
networks," Nature , vol. 542, p. 115–118, 2017.
[8] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A.
Karpathy, A. Khosla, M. Bernstein, A. C. Berg and L. Fei-Fei, "ImageNet Large
Scale Visual Recognition Challenge," International Journal of Computer Vision,
vol. 115, no. 3, p. 2015, 2015.
[9] A. Krizhevsky, I. Sutskever and G. E. Hinton, "ImageNet Classification with
Deep Convolutional Neural Networks," in Neural Information Processing
Systems 2012, 2012.
[10] S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network
Training by Reducing Internal Covariate Shift," arXiv, 2015.
[11] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, "Rethinking the
Inception Architecture for Computer Vision," in The IEEE Conference on
Computer Vision and Pattern Recognition(CVPR), 2016.
[12] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V.
Vanhoucke and A. Rabinovich, "Going Deeper With Convolutions," in The IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
[13] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image
Recognition," in The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2016.
[14] M. Patel, B. Naughton, C. Chan, N. Sprecher, S. Abeta and A. Neal, "Mobile-
Edge Computing – Introductory Technical White Paper," etsi, 2014.
[15] A. H. Sodhro, Z. Luo, A. K. Sangaiah and S. W. Baik, "Mobile edge computing
based QoS optimization in medical healthcare applications," International
Journal of Information Management, 2018.
[16] O. Salman, I. Elhajj, A. Kayssi and A. Chehab, "Edge computing enabling the
Internet of Things," in IEEE 2nd World Forum on Internet of Things (WF-IoT),
Milan, 2015.
[17] L. Miller, "The Journey to 5G," 2 3 2018. [Online]. Available:
https://www.healthcareitnews.com/news/journey-5g. [Accessed 2 2019].
[18] A. Kassianos, J. Emery, P. Murchie and F. Walter, "Smartphone applications for
melanoma detection by community, patient and generalist clinician users: a
review," British Journal of Dermatology, vol. 172, no. 6, pp. 1507-1518, 2015.
[19] Y. C. Hu, M. Patel, D. Sabella, N. Sprecher and V. Young, "Mobile Edge
Computing A key technology towards 5G," 9 2015. [Online]. Available:
209
Smart Healthcare and Safety Systems
6
https://www.etsi.org/images/files/ETSIWhitePapers/etsi_wp11_mec_a_key_tec
hnology_towards_5g.pdf. [Accessed 2 2019].
[20] D. M. West, "How 5G technology enables the health internet of things.,"
Brookings Center for Technology Innovation, 2016.
[21] J. VANIAN, "Intel's Cancer Cloud Gets New Recruits," 31 3 2016. [Online].
Available: http://fortune.com/2016/03/31/intel-cancer-cloud-research/.
[Accessed 2 2019].
[22] D. B. Mendes and N. C. d. Silva, "Skin Lesions Classification Using
Convolutional Neural Networks in Clinical Images," arXiv, 2018.
[23] X. Zhang, Z. Li, C. C. Loy and D. Lin, "PolyNet: A Pursuit of Structural
Diversity in Very Deep Networks," in he IEEE Conference on Computer Vision
and Pattern Recognition (CVPR), 2017.
[24] S. Xie, R. Girshick, P. Dollar, Z. Tu and K. He, "Aggregated Residual
Transformations for Deep Neural Networks," in The IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 2017.
[25] G. Huang, Z. Liu, L. v. d. Maaten and K. Q. Weinberger, "Densely Connected
Convolutional Networks," in The IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), 2017.
[26] J. Hu, L. Shen and G. Sun, "Squeeze-and-Excitation Networks," in The IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[27] D. P. Networks, J. Li, H. Xiao, X. Jin, S. Yan and J. Feng, "Dual Path Networks,"
in Advances in Neural Information Processing Systems 30, 2017.
[28] K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C.
Kiddon, J. Konecny, S. Mazzocchi, H. B. McMahan, T. V. Overveldt, D. Petrou,
D. Ramage and J. Roselander, "Towards Federated Learning at Scale: System
Design," arXiv, 2019.
[29] B. G. Humm and P. Walsh, "Personalised Clinical Decision Support for Cancer
Care," Semantic Applications , pp. 125-143, 2018.
[30] G. Neville, S. Considine, P. Walsh, M. O’Meara, L. Feeley, T. J. Browne, F.
O’Connell and Michael, "Concordance in the Cloud –“Simplicity MDT”- Use of
a Novel Multidisciplinary Integrated Quality Assurance Software Tool for Breast
Biopsy Risk Categorization, Correlation and Outcome Recording in over 5000
Patient Discussions," Modern Pathology, vol. 31, pp. 784-785, 2018.
[31] P. Tschandl, C. Rosendahl and H. Kittler, "The HAM10000 dataset, a large
collection of multi-source dermatoscopic images of common pigmented skin
lesions," arXiv, 2018.
[32] J. McCarthy, "Generality in artificial intelligence," Communications of the ACM,
vol. 30, no. 12, pp. 1030-1035 , 1987.
[33] N. Gessert, T. Sentker, F. Madesta, R. Schmitz, H. Kniep, I. Baltruschat, R.
Werner and A. Schlaefer, "Skin Lesion Diagnosis using Ensembles, Unscaled
Multi-Crop Evaluation and Loss Weighting," arXiv, 2018.
210