<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Deep learning: concepts and implementation tools</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Eddy Sanchez-DelaCruz</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>David Lara-Alabazares</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Postgraduate Department, Technological Institute of Misantla</institution>
          ,
          <addr-line>Veracruz</addr-line>
          <country country="MX">Mexico</country>
        </aff>
      </contrib-group>
      <fpage>142</fpage>
      <lpage>149</lpage>
      <abstract>
        <p>In this paper, the concepts and tool available to use Deep learning in scholar projects are given. We carry out experiments by combining meta-classi ers with a deep arti cial neural network in four binary datasets. The results show optimal percentages of correct classi cation in some cases. The sample criteria that prevalence in this study was a representative sample over traditional criteria.</p>
      </abstract>
      <kwd-group>
        <kwd>Arti cial Intelligence</kwd>
        <kwd>Network</kwd>
        <kwd>Meta-classi ers</kwd>
        <kwd>Arti cial Neural</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        At the beginning of the 21st century, Arti cial Intelligence (AI) in its various
disciplines that integrate it, has started in a surprising way, to emulate
faithfully human behavior and reasoning. As results of this, remarkable progress have
emerged in di erent elds such as the computer-assisted medical diagnosis,
classi cation of DNA sequences, data mining, arti cial vision, voice recognition,
analysis of written language, virtual games, robotics, and any others where the
reasoning is the main element [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>Among the di erent disciplines of AI, the deep learning is a novel alternative
(from a few decades ago) that has as main objective to make that an intelligent
agent can be capable to make its own decisions, something that currently is only
possible in science ction. In this sense, di erent approaches of Arti cial Neural
Networks (ANN) are used in deep learning, having as goal provides to an agent
of personality comparable to a human. Among these approaches, we have Deep
Neural Networks, Convolutional Neural Networks, and Deep Belief Networks.</p>
      <p>Therefore, below is exposed brie y the topic of ANN that will be helpful to
understand how deep learning works.</p>
      <p>In this study, we described basic concepts of ANN, we mentioned some ANN
tools and we experimented with various datasets.</p>
      <p>The rest of article is divided as follow: in the section 2 the ANN, deep learning
concepts are described and the tools for implement deep learning algorithms are
Copsyhroigwhetd©,2019 for this paper by its authors. U
mentioned, in the sections 3 and 4 the experiments and results are
nally, the conclusions are described in section 5. Attribution 4.0 International (CC BY 4.0)</p>
    </sec>
    <sec id="sec-2">
      <title>Arti cial Neural Network</title>
      <p>
        ANN is inspired by the way the biological nervous system works. One of the
pioneering work can be found in [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] where Warren McCulloch and Walter Pitts
proposed the rst mathematical model for a biological neuron.
      </p>
      <p>
        Formally, an ANN can be de ned as a processing element that receives a set
of input elements, denoted as X = x1; x2; : : : xn, which are modi ed respectively
for a series of weights W = w1; w2; : : : ; wn. The di erent values that are modi ed
by the weights are added together in what is called the net input (The net input is
the result of the addition of the products of each input value by its corresponding
weight adding a bias value or threshold of the neuron, denoted by b, which is
determined when it is activated). The activation of the neuron depends on the
activation function that acts on the net input which also regulates the output
of the neuron. As it can be observed in the gure 1, that both the sum and the
activation function represent the cell body of the neuron, in this place are realized
the corresponding mathematical calculations and the results are transmitted to
the output y [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. In general, an ANN is divided into three layers: the input,
hidden and output (see Figure 1), where the hidden layer has three neurons for
this example.
2.1
      </p>
      <sec id="sec-2-1">
        <title>Deep Learning</title>
        <p>
          Deep Learning is a discipline for the search of patterns through deep
abstractions that are achieved with multiple hidden layers of an ANN [
          <xref ref-type="bibr" rid="ref1 ref2">2,1</xref>
          ]. To achieve
abstraction, in a hidden layer, for example, the border of the area of interest in
an image is selected (Figure 2); depth is achieved by repeating abstraction in
as many hidden layers as desired. Both the number of hidden layers and that of
neurons range from 1 to n, since there is no metric that establishes how many
to use, rather this tries to resolve the agreement to the problem, the dimensions
and properties of the data set, and based on the experience of who implements
the ANN. The output of our ANN will be the answer that we are looking for,
which corresponds to a classi cation that tells us that in an input image such
as the one in gure 2, there will be a representation of a \gira e". Also, in the
same gure, we can see how the input and output images would be represented
for Deep learning.
2.2
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>Tools for implementing deep learning</title>
        <p>There is a wide variety of tools in which Deep learning can be implemented. Table
1 lists the ones that to the authors' knowledge are the most representative. These
tools can be found in the web as open source and are oriented to any passionate
person with not too much experience in programming or to self-taught persons
with a little patience, in such way that they can explore the Deep learning with
some tool of their choice.
We carry out four experiments using datasets from public access, which are listed
in Table 2.</p>
        <p>
          { Car-evaluation dataset was created from a simple hierarchical decision model,
originally developed for DEX demonstration, which is a system based on a
methodology that combines multi-attribute decision making with expert
systems [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ].
{ Daily-deals is a set of information about products supply and demand in
an establishment (mall), the dataset is available, for download using
RapidMiner, a data mining tool.
{ Pap-smear dataset corresponds to microscopic images of pap-smears, which
are categorized in seven states where cervical cancer disease could be [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ].
{ Blood-transfusion dataset consists of blood information of subjects to know
if they are eligible (1) or not (0) to be donors [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ].
        </p>
        <p>
          Each dataset was used with two classes; datasets with more than two classes
were adapted to binary classes with the One-vs-All approach [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ].
4
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Results and analysis</title>
      <p>
        The experiments were carried out using the WEKA v.3.8 tool (Waikato
Enviroment Knowledge Analysis) [
        <xref ref-type="bibr" rid="ref11 ref13">13,11</xref>
        ], using a Lenovo G470 laptop, having an
Intel Celeron @ 1.50GHz processor, with 2.00 GB RAM and 32 Bit professional
windows 7 OS. The WEKA tool has several categories of algorithms, each
category contains a variety of algorithms that can be implemented separately or by
combining them. In this case, for the tests of each of the experiments, each
classier of the Meta-classi ers (assembled classi er) category was combined with the
deep learning algorithm Dl4JMlp Classi er, that is, a classi er assembled with a
deep ANN. The implementations were made using three sampling criteria: cross
validation, 2=3 1=3 and representative sample (RS). In Table 3 we can observe
the highest results.
      </p>
      <p>These results are notable due to the percentage of correctly classi ed
instances, which in some cases have reached the optimum value of 100%. However,
it worth to mention that binary classi cation is the easiest to solve as compared
with the treatment of multi-class attributes. Then, a future study will be to
observe the behaviour of multi-class classi cation applying Deep Learning with
these datasets.</p>
      <p>
        It is important to highlight that both the cross-validation and the 2=3
1=3 sampling criteria are well established in AI for the classi cation of large
volumes of data. Therefore, being these four datasets of reasonable dimensions,
the prevalent criteria was statistical for RS, which was justi ed in reason that it
represents the equilibrium point for a given frequency or the data concentration
gave a distribution [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>
        According to [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], in many applications, it can be advantageous to combine
the best of two approaches, in this case: assembled classi ers and deep
learning. Assembled classi ers is an approach based on successive results re nements
obtained with standard classi cation algorithms and Dl4JMlpClassi er is an
algorithm based on advantages of ANN multilayer perceptron.
5
      </p>
    </sec>
    <sec id="sec-4">
      <title>Conclusion and future works</title>
      <p>In these previous results can be appreciated that the binary classi cation by
combining assembled algorithms with a Deep Learning algorithm give
satisfactory results. Then, it can be speculated that the results in multi-class classi
ca</p>
      <p>Dataset</p>
      <p>instances attributes classes
car-evaluation
Daily-days
pap-smear
Blood-transfusion
1728
1500
917
748
6
4
20
5
4
2
7
2
tions may be acceptable, i.e., in a range not less than 80% of correctly classi ed
instances. As future work we propose:
{ To make experiments with the algorithm combinations of Table 3, but with
multi-class datasets.</p>
      <p>Any person, with basic knowledge of programming and computer skills, can
take their rst steps and experiment in Deep Learning with the data
processing tools such as those listed in Table 1; there is also a variety of public domain
datasets to carry out experiments such as those presented in the previous section;
In addition to the above, there is information in books, scienti c and
dissemination journals, tutorials, and videos that indicate step by step how to implement
a deep ANN.</p>
      <p>Finally, as an additional comment: we would like to mention that someone can
also experience with Deep Learning from a digital artistic expression perspective
and make surrealist image designs such as those shown in Figure 3, because as
long as AI continues on its way to make an intelligent agent able to think and
feel like a human, a variety of algorithms have been developed to explore and
exploit the bene ts o ered by Deep Learning, for example, generate artistic
representations in digital format. A tool that allows you to create this type of
art is \Deep Dream Generator", available as shareware for free on the website:
https://deepdreamgenerator.com/</p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgement</title>
      <p>Authors acknowledge to the National Technological of Mexico, TECNM, and
to the council of science and techonology, CONACYT, for the support given
to realise this research. Authors acknowledge Editors and Reviewers for their
constructive comments to produce the nal version of this manuscript.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Mart</surname>
          </string-name>
          n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis,
          <source>Je rey Dean</source>
          ,
          <string-name>
            <surname>Matthieu Devin</surname>
          </string-name>
          , et al. Tensorow:
          <article-title>Large-scale machine learning on heterogeneous distributed systems</article-title>
          .
          <source>arXiv preprint arXiv:1603.04467</source>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>Itamar</given-names>
            <surname>Arel</surname>
          </string-name>
          , Derek C Rose,
          <string-name>
            <surname>Thomas P Karnowski</surname>
          </string-name>
          , et al.
          <article-title>Deep machine learninga new frontier in arti cial intelligence research</article-title>
          .
          <source>IEEE computational intelligence magazine</source>
          ,
          <volume>5</volume>
          (
          <issue>4</issue>
          ):
          <volume>13</volume>
          {
          <fpage>18</fpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>Arthur</given-names>
            <surname>Asuncion</surname>
          </string-name>
          and
          <string-name>
            <given-names>David</given-names>
            <surname>Newman</surname>
          </string-name>
          .
          <source>Uci machine learning repository</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>DC</given-names>
            <surname>Baird</surname>
          </string-name>
          .
          <article-title>Experimentacion. una introduccion a la teor a de mediciones y al disen</article-title>
          ~o de experimentos,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>Alejandro</given-names>
            <surname>Cardenas</surname>
          </string-name>
          <article-title>Cardona</article-title>
          .
          <article-title>Inteligencia arti cial, metodos bio-inspirados: un enfoque funcional para las ciencias de la computacion</article-title>
          .
          <source>PhD thesis</source>
          , Universidad Tecnologica de Pereira. Facultad de Ingenier as Electrica ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>Jan</given-names>
            <surname>Jantzen</surname>
          </string-name>
          and
          <string-name>
            <given-names>George</given-names>
            <surname>Dounias</surname>
          </string-name>
          .
          <article-title>Analysis of pap-smear image data</article-title>
          .
          <source>In NatureInspired Smart Information Systems 2nd Annual Symposium. NiSIS</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Yann</surname>
            <given-names>LeCun</given-names>
          </string-name>
          , Yoshua Bengio, and Geo rey Hinton.
          <article-title>Deep learning</article-title>
          .
          <source>nature</source>
          ,
          <volume>521</volume>
          (
          <issue>7553</issue>
          ):
          <fpage>436</fpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Warren S McCulloch and Walter Pitts</surname>
          </string-name>
          .
          <article-title>A logical calculus of the ideas immanent in nervous activity</article-title>
          .
          <source>The bulletin of mathematical biophysics</source>
          ,
          <volume>5</volume>
          (
          <issue>4</issue>
          ):
          <volume>115</volume>
          {
          <fpage>133</fpage>
          ,
          <year>1943</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Nils</surname>
          </string-name>
          J Nilsson.
          <article-title>Principles of arti cial intelligence</article-title>
          . Morgan Kaufmann,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <article-title>Anderson Rocha and Siome Klein Goldenstein. Multiclass from binary: Expanding one-versus-all, one-versus-one and ecoc-based approaches</article-title>
          .
          <source>IEEE Transactions on Neural Networks and Learning Systems</source>
          ,
          <volume>25</volume>
          (
          <issue>2</issue>
          ):
          <volume>289</volume>
          {
          <fpage>302</fpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>Ingrid</given-names>
            <surname>Russell</surname>
          </string-name>
          and
          <string-name>
            <given-names>Zdravko</given-names>
            <surname>Markov</surname>
          </string-name>
          .
          <article-title>An introduction to the weka data mining system</article-title>
          .
          <source>In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education</source>
          , pages
          <volume>742</volume>
          {
          <fpage>742</fpage>
          . ACM,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Ju</surname>
          </string-name>
          <article-title>rgen Schmidhuber</article-title>
          .
          <article-title>Deep learning in neural networks: An overview</article-title>
          .
          <source>Neural networks</source>
          ,
          <volume>61</volume>
          :
          <fpage>85</fpage>
          {
          <fpage>117</fpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Ian H Witten</surname>
          </string-name>
          ,
          <article-title>Eibe Frank, Mark A Hall, and Christopher J Pal. Data Mining: Practical machine learning tools and techniques</article-title>
          . Morgan Kaufmann,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>I-Cheng</surname>
            <given-names>Yeh</given-names>
          </string-name>
          ,
          <string-name>
            <surname>King-Jang Yang</surname>
          </string-name>
          , and
          <string-name>
            <surname>Tao-Ming Ting</surname>
          </string-name>
          .
          <article-title>Knowledge discovery on rfm model using bernoulli sequence</article-title>
          .
          <source>Expert Systems with Applications</source>
          ,
          <volume>36</volume>
          (
          <issue>3</issue>
          ):
          <volume>5866</volume>
          {
          <fpage>5871</fpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>