<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Arti cial Intelligence in Mobile Applications of Dermatology: A Systematic Mapping Study</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Veronica Tint n</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>, Jose Caiza</institution>
          ,
          <addr-line>Hebert Atencio, and Fernando Caicedo</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Universidad de las Fuerzas Armadas ESPE, Quijano Ordon~ez y Hermanas Paez, Latacunga - Ecuador https://espe-el.espe.edu.ec/</institution>
        </aff>
      </contrib-group>
      <fpage>12</fpage>
      <lpage>25</lpage>
      <abstract>
        <p>Skin diseases are considered the fourth most common cause of human disease and are more common than it is thought. With the advancement of technology, new concepts have been incorporated into the eld of mobile health and, speci cally, into dermatology. On the other hand, some factors make access to medical care di cult which is why the use of mobile applications for medical care and diagnosis is very common today, but are these applications suitable? In this study, a systematic literature mapping is carried out of mobile applications that use techniques of arti cial intelligence for the diagnosis of skin diseases. Its focus is mainly on the level of sensitivity, speci city and overall accuracy of said diagnoses in comparison to the accuracy of a dermatologist. Several applications of care and diagnosis of skin diseases were found. However, only 9 studies describe the techniques that use these applications for the classi cation of the disease; several of them with high levels of precision, and mainly those that utilize arti cial neural networks and vector-support machine algorithms. Despite the research conducted in this eld, these techniques are still in development and are available only for a number of limited diseases. Thus, we believe it is necessary to direct the research towards the eld of dermatology and contribute to the minimization of the gap between doctor and patient.</p>
      </abstract>
      <kwd-group>
        <kwd>mobile applications</kwd>
        <kwd>dermatology apps</kwd>
        <kwd>skin care</kwd>
        <kwd>derma- tology</kwd>
        <kwd>smartphone</kwd>
        <kwd>arti cial intelligence</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>A report by the World Health Organization mentions that although
considerable progress has been made in the treatment of cutaneous diseases, they are
still frequent in many rural communities of developing countries, with serious
socio-economic and sanitary repercussions, and directly or indirectly, causing
disability.</p>
      <p>The implementation of new technologies in healthcare and the advancement
of telecommunications have spurred a rapid growth of telemedicine in the
different health systems. New information and communication technologies (ICT)
have allowed for innumerable possibilities in the exchange of health information,
as well as new forms of healthcare assistance such as assistance given remotely
from the healthcare professional to the patient, with teledermatology being one
of the elds of telemedicine application. Currently, many technologies are used
in the eld of telemedicine. Research in this area focuses on the development of
smartphone applications for the detection of skin diseases, mainly of skin cancer.</p>
      <p>
        On the other hand, Arti cial Intelligence (AI) has resurfaced recently in
scienti c and public awareness. As technology companies and scientists announce
recent advances and technologies at a dizzying pace, AI tries to understand and
build intelligent agents, often instantiated as software programs[
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. As a
result, the use of mobile phones with arti cial intelligence apps for dermatology
is spreading. It has been determined that at the end of 2018 there were an
estimated 5,000 million subscriptions worldwide with a projection of 7,200 million
smartphones subscriptions for the year 2024 [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ].
      </p>
      <p>
        It is important to mention that a smartphone camera's function is an
innovative development to which arti cial intelligence systems have been incorporated
at a low price. This addition is made so that, through images, people from
isolated communities and poor countries may receive assistance in scanning,
analyzing and performing regular dermatological exams at any given time and
place [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ][
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. This helps prevent or detect any anomaly on the skin's surface
and classi es the image as benign or malignant.
      </p>
      <p>This study was created due to the interest in investigating the applicability
of arti cial intelligence to the eld of dermatology through the development of
mobile applications. Said applications demonstrate an acceptable overall level
of precision of diagnosis as to the precision of dermatologists. The applications
take into account the sensitivity and speci city of the tests, in order to be consist
with in-person medical services. In this manner, people with limited access to
medical assistance due to distance, physical disability, employment, costs or
schedules may receive quality health services and thus contribute to bridge the
doctor-patient gap.</p>
      <p>For a better description of what was found in the studies, this document has
been organized as follows: Section II presents a general description of the main
techniques of arti cial intelligence and its applicability in the development of
mobile applications, section III describes the elements of the research protocol,
section IV shows the results of the investigation, section V discusses the results
found and answers the posed research questions, and nally section VI presents
the conclusions of the work done.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Background</title>
      <p>
        Even though the use of arti cial intelligence in dermatology is not new, the high
degree of precision that is now possible using convolutional neural networks
(CNN) raises questions about the future role of dermatologists in the diagnosis
and management of melanoma and other skin cancers. Recently, the
convolutional neural networks captured in 129,450 clinical images reached the level of
dermatologist precision in the diagnosis of skin malignancy [
        <xref ref-type="bibr" rid="ref1 ref19">1,19</xref>
        ].
      </p>
      <p>Convolutional neural networks have proven crucial to the success of image
analysis and are also responsible for the subsequent evolution in medical imaging.
Along with CNNs other techniques are also used for diagnosis, such as Support
Vector Machines (SVMs).
2.1</p>
      <sec id="sec-2-1">
        <title>How a CNN functions in image analysis</title>
        <p>
          A CNN uses a special type of layer, called a convolutional layer, to summarize
and transform groups of pixels into images and extract high-level features. They
can operate on the raw image and learn useful features from the training sets.
This simpli es the formation process and facilitates the identi cation of image
patterns. Although the formation phase of the deep learning model can be
expensive from a computational point of view, the nalized diagnostic model may
be displayed on mobile devices, potentially improving levels of sensitivity and
speci city [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ].
        </p>
        <p>
          Sensibility It is de ned as the probability of correctly classifying an
individual as sick, as in, the probability of a sick subject receiving a positive result
on the test. In other words, sensitivity can be de ned as the ability of a test to
detect disease [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ].
        </p>
        <p>
          Therefore, it represents the fraction of true positives. When expressed as a
percentage, it represents the percentage of positive results regarding the total
number of patients, as in, the percentage of true positives that will be obtained
when applying the diagnostic test to the patients [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ].
        </p>
        <p>
          Speci city It is de ned as the probability of correctly classifying an
individual as healthy, as in, the probability of a healthy subject receiving a negative
result. In other words, you can say that speci city is the ability to detect
healthiness [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ].
        </p>
        <p>
          Therefore, it represents the fraction of true negatives. When expressed as a
percentage, it represents the percentage of negative results regarding the total
of healthy people, in other words, the percentage of true negatives obtained by
applying the test to healthy people [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ].
2.2
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>How a Support Vector Machine Functions (SVM)</title>
        <p>
          They are a set of supervised learning algorithms, one of the most widely used
methods to classify data. The basic concept is that an SVM maps the input
data to an-dimensional space, where it tries to nd the optimal hyperplane to
separate data sets. Its popularity lies in its exibility to be used in a wide range
of areas and pattern-recognition problems. Its main characteristics are: the use
of the nuclei, the absence of local minimums, the solution and capacity control
obtained through margin optimization [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ].
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Systematic Mapping Study { Method</title>
      <p>
        An SMS (Systematic Mapping Study), is a method that consists of a literary
investigation about an area of interest to determine the nature, scope and
quantity of published primary studies, as well as give an overview of the researched
area through the classi cation and counting of literary contributions [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
      </p>
      <p>This study follows the guidelines established in Petersen's Guidelines for
conducting systematic mapping studies in software engineering. The purpose of
this SMS is to answer the following research questions RQ (Research Question).
{ RQ1 Which mobile applications for skin disease care and diagnosis use
arti cial intelligence?
{ RQ2 What type of dermatological diseases do the identi ed mobile
applications target?
{ RQ3 What arti cial intelligence technique is applied in the development of
the mobile applications?
{ RQ4 What are the overall levels of sensitivity, speci city and accuracy of
the implemented diagnostic tests?
Several systematic reviews have been found in the literature where topics related
to arti cial intelligence techniques used by mobile applications for the diagnosis
of skin diseases are addressed. Most of them focus solely on skin cancer, however,
we must consider that there are other types of dermatological diseases which can
be problematic due to their side e ects. The aforementioned diseases lead to very
serious, risky and/or expensive therapies which considerably reduce the patient's
quality of life. Therefore, for the purpose of this study, research questions are
posed in the context of mobile applications for the care and diagnosis of
dermatological diseases in general.
3.1</p>
      <sec id="sec-3-1">
        <title>Search Strategy</title>
        <p>
          In order to identify keywords and formulate the search chain, the PICO strategy,
(Population, Intervention, Comparison and Results) suggested in Guidelines for
performing Systematic Literature Reviews in Software Engineering by
Kitchenham [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] was used. The aim was to extract studies that help answer the research
questions.
        </p>
        <p>{ Population Within the context of this study, population corresponds to
the set of mobile applications for the care and diagnosis of dermatological
diseases for which the following keywords were selected: "apps" OR "mobile
applications" OR "dermatology apps AND "smartphone".
{ Intervention Management or intervention of interest for this study to
constitute the arti cial intelligence technique used for the mobile applications,
keyword: \arti cial intelligence".
{ Comparison No empirical comparison is made, there is no alternative
intervention to compare, although the same is not always available, so this
component is omitted and the PICO strategy becomes PIO.
{ Outcome It is the relevant consequence of interest. The expected result is
skin care, keywords: \skin care" OR \dermatology".</p>
        <p>
          The sources used for the selection of the primary studies were the following
digital bases: IEEE Xplore, Scopus, SpringerLink, Web of Science and
MEDLINE, MEDLINE can be accessed via PubMed Central. The period of time
considered for searching in the digital bases is from 2012 to 2019. 2012 was
chosen because an article had been published in that speci c year describing a
mobile hardware / software system (DERMA / Care) which helps detect skin
cancer [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. For each base the following search string was de ned:
{ IEEE Xplore (("apps" OR "mobile applications" OR "dermatology apps")
AND ("skin care" OR "dermatology") AND ("smartphone") AND ("arti
cial intelligence"))
{ Scopus ALL(("apps" OR "mobile applications" OR "dermatology apps")
AND ("skin care" OR "dermatology") AND ("smartphone") AND ("arti
cial intelligence" ))
{ SpringerLink ALL(("apps" OR "mobile applications" OR "dermatology
apps") AND ("skin care" OR "dermatology") AND ("smartphone") AND
("arti cial intelligence" ))
{ Web of Science TS=("apps" OR "mobile applications" OR "dermatology
apps") AND TS=("skin care" OR "dermatology") AND TS=("smartphone")
AND TS=("arti cial intelligence")
{ MEDLINE via PUBMED Central (("apps" OR "mobile applications"
OR "dermatology apps") AND ("skin care" OR "dermatology") AND
("smartphone") AND ("arti cial intelligence"))
In Table 1 you can see the studies found according to the Library Catalog.
The snowball method was used for the selection of the studies in the digital
bases. Only studies corresponding to journal articles and conference articles were
extracted, thus ensuring that they have been reviewed by peers. Inclusion criteria
(IC) and exclusion criteria (EC) were de ned to select relevant articles in the
literature that are relevant to the research questions [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ].
        </p>
        <p>The following inclusion criteria were applied to titles and abstracts:
{ IC1 Studies whose content refers to mobile applications for the care and
diagnosis of skin diseases.
{ IC2 Studies in which the application of arti cial intelligence techniques in
mobile applications has been identi ed.
{ IC3 Studies in which dermatological diseases that can be treated or
diagnosed with the use of a mobile application are mentioned.
{ IC4 Studies in which reference is made to the levels of sensitivity and
specicity of the diagnostic test, or the level of con dence and accuracy of the
classi er.</p>
        <p>The following criteria state when a study was excluded:
{ EC1 Studies in which the use of a mobile application is not identi ed.
{ EC2 Studies that do not provide information about the arti cial intelligence
technique used.
{ EC3 Studies that do not focus on dermatological diseases.
{ EC4 Studies that do not determine any level of sensitivity, speci city or
accuracy of the diagnostic test.</p>
        <p>The preliminary investigation resulted in 143 relevant results. To reach this
result, the search focused on nding keywords the in the titles, abstracts and
sections of the introduction respectively. Of the 143 studies found, 105 studies
were downloaded, and 38 articles were discarded, of which 11 were duplicated,
1 was in another language and 26 corresponded to conference summaries. To
carry out this activity, an open-source bibliographic manager was used, which
permitted the collection, administration and citation of research. This software
automatically detects bibliographic sources and imports data directly from web
pages. The details of the studies found and downloaded are shown in Table 2.
From the total amount of publications extracted from the digital bases (105
preselected), it was necessary to extract the relevant information to determine the
number of studies that met the inclusion criteria and could answer the research
questions. The bibliographic manager allowed the extraction of the relevant
information from each publication and generated a report with the data shown in
the Table 3:
After reading and analyzing the abstracts of the pre-selected articles from the
previous phase, those whose content were relevant to the research questions
were selected. The Table 4 shows the number of articles selected according to
the digital base:
The studies which could answer the research questions were selected and veri ed
using the following quality veri cation criteria (VC):
{ VC1 Does the selected study contribute to answering the research questions?
{ VC2 Does the selected study contain references to studies published in
journals, conferences or congresses?
{ VC3 Is the study based on research?
{ VC4 Is there a clear statement of the research's objectives?
{ VC5 Is there a clear statement of the results?
{ VC6 Is the selected study written in the English language?</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Results</title>
      <p>Fig. 3 demonstrates that 2019 and 2018 are the years when more publications
were made about mobile applications which use arti cial intelligence techniques.</p>
      <p>
        Item Type Reference
Conference Paper [
        <xref ref-type="bibr" rid="ref10 ref2 ref3 ref6">2,3,6,10</xref>
        ]
      </p>
      <p>
        Journal Article [
        <xref ref-type="bibr" rid="ref15 ref16 ref4 ref5 ref9">4,5,9,15,16</xref>
        ]
44% ( n = 4) of these articles were published in 2019, 33% (n = 3) of these
articles were published in 2018, 11% (n = 1) in 2017 and 11% (n = 1 ) in 2012
where the rst mobile application with arti cial intelligence has been registered.
Fig. 4 shows the distribution of publications according to the digital base.
      </p>
      <p>The most used research protocol is design science, 78% (n=7) of primary
studies refer to the development of dermatological mobile applications; in cases
of studies, 11% (n=1) corresponds to the applicability of an existing application;
and 11% (n=1) is a systematic review of literature.
This section analyzes the information of the articles extracted in the previous
phase to answer the research questions, based on full text reading. The following
applications have been found.</p>
      <p>
        A mobile application called DERMA /care was developed in 2012 for melanoma
detection. It uses image processing through a Support Vector Machine. No
sensitivity levels or speci city are detailed, but it is mentioned in this study due to
being the rst study found which uses arti cial intelligence techniques [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>
        A recent systematic review found very little information based on the
evidence of the e ectiveness of mobile applications for dermatology. Two studies
were identi ed, both at high risk of bias, which tested four diagnostic
applications of melanoma. The sensitivity ranges from 7% to 73% and the speci city
from 37% to 94%. The authors concluded that these applications for
smartphones have not shown su cient proof of their accuracy and the existing data
[
        <xref ref-type="bibr" rid="ref15 ref4">15,4</xref>
        ].
      </p>
      <p>
        Another study found that an all-inclusive mobile app was developed for the
diagnosis of melanoma. This app uses a Support Vector Machine (SVM) with
a public database of 200 images. Using the Synthetic Minorities Oversampling
Technique of Ethnicity (SMOTE) obtained 80% sensitivity, 90% speci city, 88%
accuracy and 85% area under curve (AUC) and without SMOTE it obtained
55% sensitivity, 95% speci city, 90% accuracy and 75% AUC. The performance
metrics and calculation times evaluated are comparable to or better than the
previous methods [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>
        In another study, a mobile application is implemented for the early detection
of melanoma employing real-time image processing algorithms along with a
normal Bayesian classi er. The lesions may be classi ed by using a Support Vector
Machine (SVM) as well, using a base of 150 images (90 for training and 60 for
testing). This method obtains results for accuracy, sensitivity and speci city of
95%, 98%, and 92.19% on average, respectively. It should be taken into account
that these results are more reliable when the lesions are geometrically di erent
[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        Another study implements an application with a set of image data provided
by the National Skin Center - Singapore (117 benign and 67 malignant). In
this study the e ectiveness of the proposed system for detection of melanoma
is con rmed with 89.09% sensitivity in comparison to 90% speci city of a SVM
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        m-Skin Doctor is an application that uses VSM for classi cation with 84
images of melanoma and non-melanoma that were used in equal amounts to
train the SVM and another 36 images in equal proportions of melanoma and
non-melanoma images were used for the test. This resulted in a sensitivity and
speci city of 80% and 75% respectively [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
      </p>
      <p>
        Another mobile application based on a deep learning algorithm of a
convolutional neural network was found. It uses a data set of 300 images of which 80%
of the images are for training and 20% of the images are for testing. 3 types of
model were used in the development process, with 2, 3 and 5 layers convoluted
respectively. These resulted in an overall pressure of 87% for model 1, 93% for
model 2 and 86% for model 3 which is the most commonly used CNN [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        Lastly, an application for the diagnosis of skin cancer was found. Said
application uses a data set of 48,373 dermoscopic images which were collected from
three di erent les and labeled and validated by dermatologist experts. A
convolutional neural network model called MobileNetV2 is trained manually using
learning transfer in order to classify skin lesions as benign or malignant. After
singing the 32-lot size, the trained model obtained results with a global accuracy
of 91.33%. No sensitivity and speci city levels are reported [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
5.1
      </p>
      <sec id="sec-4-1">
        <title>Conclusions</title>
        <p>Smartphone applications in the eld of dermatology are becoming increasingly
important. They can be particularly e ective for reaching a large number of
people and improving their daily habits, but it is crucial to know if these
applications represent a risk if they aren't able to make adequate diagnoses especially
when it comes to serious diseases. On the other hand, it is necessary to mention
that improvements have been made in the development of these applications
thanks to techniques such as arti cial intelligence, deep learning, and vector
support machine. These tools have raised the level of con dence in classi ers,
provided faster calculation speeds and improved the results. In conclusion, it
can be said that these arti cial intelligence techniques in mobile applications
help considerably in making diagnostic decisions with a high degree of precision.
However, most of these applications are still under development and available
only for a limited number of diseases. Likewise, it has been proven that advances
in smartphone applications have improved health sciences, created awareness of
the importance of a healthy lifestyle and greatly contributed to shorten the gap
between doctor and patient.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Abbott</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Smartphone apps for skin cancer diagnosis: Implications for patients and practitioners</article-title>
          .
          <source>Australasian Journal of Dermatology</source>
          <volume>59</volume>
          (
          <issue>3</issue>
          ),
          <volume>168</volume>
          {
          <fpage>170</fpage>
          (
          <year>2018</year>
          ). https://doi.org/10.1111/ajd.12758
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Alizadeh</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mahloojifar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>A Mobile Application for Early Detection of Melanoma by Image Processing Algorithms</article-title>
          .
          <source>In: 2018 25th National and 3rd International Iranian Conference on Biomedical Engineering (ICBME)</source>
          (
          <year>2019</year>
          ). https://doi.org/10.1109/ICBME.
          <year>2018</year>
          .8703575
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Chin</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tsai</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          :
          <article-title>Facial skin image classi cation system using Convolutional Neural Networks deep learning algorithm</article-title>
          .
          <source>In: 2018 9th International Conference on Awareness Science and Technology (iCAST)</source>
          . pp.
          <volume>51</volume>
          {
          <issue>55</issue>
          (Sep
          <year>2018</year>
          ). https://doi.org/10.1109/ICAwST.
          <year>2018</year>
          .8517246
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Chuchu</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Takwoingi</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dinnes</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Matin</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bassett</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Moreau</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bayliss</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Davenport</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Godfrey</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>O'connell</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jain</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Walter</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Deeks</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Williams</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          :
          <article-title>Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma</article-title>
          .
          <source>Cochrane Database of Systematic Reviews</source>
          <year>2018</year>
          (
          <volume>12</volume>
          ) (
          <year>2018</year>
          ). https://doi.org/10.1002/14651858.CD013192
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Do</surname>
          </string-name>
          , T.T.,
          <string-name>
            <surname>Hoang</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pomponiu</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhou</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cheung</surname>
            ,
            <given-names>N.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koh</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tan</surname>
            ,
            <given-names>S.H.</given-names>
          </string-name>
          :
          <article-title>Accessible Melanoma Detection Using Smartphones and Mobile Image Analysis</article-title>
          .
          <source>IEEE Transactions on Multimedia</source>
          <volume>20</volume>
          (
          <issue>10</issue>
          ),
          <volume>2849</volume>
          {
          <fpage>2864</fpage>
          (
          <year>2018</year>
          ). https://doi.org/10.1109/TMM.
          <year>2018</year>
          .2814346
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Ech-Cherif</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Misbhauddin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ech-Cherif</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Deep Neural Network Based Mobile Dermoscopy Application for Triaging Skin Cancer Detection</article-title>
          .
          <source>In: 2019 2nd International Conference on Computer Applications Information Security (ICCAIS)</source>
          . pp.
          <volume>1</volume>
          {
          <issue>6</issue>
          (May
          <year>2019</year>
          ). https://doi.org/10.1109/CAIS.
          <year>2019</year>
          .8769517
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Egea</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , Jose, J.:
          <article-title>Sensibilidad y especi cidad de los metodos diagnosticos convencionales de la caries oclusal segun la evidencia cient ca disponible</article-title>
          .
          <source>RCOE</source>
          <volume>7</volume>
          (
          <issue>5</issue>
          ),
          <volume>491</volume>
          {501 (Oct
          <year>2002</year>
          ), http://scielo.isciii.es/scielo.php
          <article-title>?script=sci_ abstract&amp;pid=S1138-123X2002000600004&amp;lng=es&amp;nrm=iso&amp;tlng=es</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Florez</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sanchez</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Villalobos</surname>
            ,
            <given-names>J.:</given-names>
          </string-name>
          <article-title>A catalog of automated analysis methods for enterprise models</article-title>
          .
          <source>SpringerPlus</source>
          <volume>5</volume>
          (
          <issue>1</issue>
          ),
          <volume>406</volume>
          (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Kalwa</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Legner</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kong</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pandey</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Skin cancer diagnostics with an all-inclusive smartphone application</article-title>
          .
          <source>Symmetry</source>
          <volume>11</volume>
          (
          <issue>6</issue>
          ) (
          <year>2019</year>
          ). https://doi.org/10.3390/sym11060790
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Karargyris</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Karargyris</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pantelopoulos</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>DERMA/Care: An advanced image-processing mobile application for monitoring skin cancer</article-title>
          .
          <source>In: 2012 IEEE 24th International Conference on Tools with Arti cial Intelligence</source>
          . vol.
          <volume>2</volume>
          , pp.
          <volume>1</volume>
          {
          <issue>7</issue>
          (
          <year>2012</year>
          ). https://doi.org/10.1109/ICTAI.
          <year>2012</year>
          .180
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Kitchenham</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>Guidelines for performing Systematic Literature Reviews in Software Engineering j Kitchenham BA</article-title>
          j
          <string-name>
            <surname>Request</surname>
            <given-names>PDF</given-names>
          </string-name>
          , https://s3.amazonaws.com/academia.edu.documents/35830450/2_ 143465389588742151.
          <article-title>pdf?response-content-disposition=inline%3B% 20filename%3DSystematic_Literature_Reviews_SLR.pdf&amp;X-Amz-</article-title>
          <string-name>
            <surname>Algorithm= AWS4-HMAC-SHA256</surname>
          </string-name>
          &amp;
          <article-title>X-Amz-Credential=AKIAIWOWYYGZ2Y53UL3A%2F20190824% 2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20190824T142757Z&amp; X-Amz-Expires=3600&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature= c88d4ff2e5c06172352d98eb386da982e2fd0d228ff895351f81ddd1bb249893</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Liberati</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Altman</surname>
            ,
            <given-names>D.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tetzla</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mulrow</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , G tzsche, P.C.,
          <string-name>
            <surname>Ioannidis</surname>
            ,
            <given-names>J.P.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Clarke</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Devereaux</surname>
            ,
            <given-names>P.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kleijnen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Moher</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration</article-title>
          .
          <source>PLOS Medicine</source>
          <volume>6</volume>
          (
          <issue>7</issue>
          ),
          <source>e1000100 (Jul</source>
          <year>2009</year>
          ). https://doi.org/10.1371/journal.pmed.
          <volume>1000100</volume>
          , https://journals.plos.org/plosmedicine/article?id=
          <volume>10</volume>
          .1371/journal. pmed.1000100
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Petersen</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vakkalanka</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kuzniarz</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Guidelines for conducting systematic mapping studies in software engineering: An update</article-title>
          .
          <source>Information and Software Technology 64, 1{18 (Aug</source>
          <year>2015</year>
          ). https://doi.org/10.1016/j.infsof.
          <year>2015</year>
          .
          <volume>03</volume>
          .007, http://www.sciencedirect.com/science/article/pii/S0950584915000646
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <given-names>Pita</given-names>
            <surname>Fernandez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Pertegas</surname>
          </string-name>
          <string-name>
            <surname>D</surname>
          </string-name>
          az, S.:
          <article-title>Pruebas diagnosticas: Sensibilidad y especi cidad</article-title>
          .
          <source>(Jul</source>
          <year>2010</year>
          ), https://www.fisterra.com/mbe/investiga/pruebas_ diagnosticas/pruebas_diagnosticas.asp#sensibilidad, unidad de Epidemiolog a
          <article-title>Cl nica y Bioestad stica</article-title>
          . Complexo
          <string-name>
            <surname>Hospitalario Universitario de A Corun~</surname>
          </string-name>
          <article-title>a (Espan~a)</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Reiter</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rotemberg</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kose</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Halpern</surname>
            ,
            <given-names>A.C.</given-names>
          </string-name>
          :
          <article-title>Arti cial Intelligence in Skin Cancer</article-title>
          .
          <source>Current Dermatology Reports</source>
          <volume>8</volume>
          (
          <issue>3</issue>
          ),
          <volume>133</volume>
          {140 (Sep
          <year>2019</year>
          ). https://doi.org/10.1007/s13671-019-00267-0, https://doi.org/10.1007/ s13671-019-00267-0
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Tau</surname>
            <given-names>q</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Hameed</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            ,
            <surname>Anjum</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Hameed</surname>
          </string-name>
          ,
          <string-name>
            <surname>F.:</surname>
          </string-name>
          <article-title>m-skin doctor: A mobile enabled system for early melanoma skin cancer detection using support vector machine</article-title>
          .
          <source>Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST 181 LNICST</source>
          ,
          <volume>468</volume>
          {
          <fpage>475</fpage>
          (
          <year>2017</year>
          ). https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -49655-9 57
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Xiong</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pfau</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Young</surname>
            ,
            <given-names>A.T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wei</surname>
            ,
            <given-names>M.L.</given-names>
          </string-name>
          :
          <article-title>Arti cial Intelligence in Teledermatology</article-title>
          .
          <source>Current Dermatology Reports</source>
          <volume>8</volume>
          (
          <issue>3</issue>
          ),
          <volume>85</volume>
          {90 (Sep
          <year>2019</year>
          ). https://doi.org/10.1007/s13671-019-0259-8, https://doi.org/10.1007/ s13671-019-0259-8
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Yas</surname>
            ,
            <given-names>Q.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zaidan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zaidan</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hashim</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lim</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>A Systematic Review on Smartphone Skin Cancer Apps: Coherent Taxonomy</article-title>
          , Motivations, Open Challenges and Recommendations, and New Research Direction.
          <source>Journal of Circuits, Systems and Computers</source>
          <volume>27</volume>
          (
          <issue>5</issue>
          ) (
          <year>2018</year>
          ). https://doi.org/10.1142/S0218126618300039
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Yu</surname>
            ,
            <given-names>K.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Beam</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kohane</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Arti cial intelligence in healthcare</article-title>
          .
          <source>Nature Biomedical Engineering</source>
          <volume>2</volume>
          (
          <issue>10</issue>
          ),
          <volume>719</volume>
          {
          <fpage>731</fpage>
          (
          <year>2018</year>
          ). https://doi.org/10.1038/s41551- 018-0305-z
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Zaidan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zaidan</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Albahri</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Alsalem</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Albahri</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yas</surname>
            ,
            <given-names>Q.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hashim</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>A review on smartphone skin cancer diagnosis apps in evaluation and benchmarking: coherent taxonomy, open issues and recommendation pathway solution</article-title>
          .
          <source>Health and Technology</source>
          <volume>8</volume>
          (
          <issue>4</issue>
          ),
          <volume>223</volume>
          {
          <fpage>238</fpage>
          (
          <year>2018</year>
          ). https://doi.org/10.1007/s12553-018-0223-9
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>