<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Application of neural networks to the analysis of the resistance of the human immunodeficiency virus to HIV reverse transcriptase inhibitors</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Anastasia V. Demidova</string-name>
          <email>demidova_av@rudn.university</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Olga A. Tarasova</string-name>
          <email>olga.a.tarasova@gmail.com</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Applied Probability and Informatics Peoples' Friendship University of Russia Miklukho-Maklaya str.</institution>
          <addr-line>6, Moscow, 117198</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Institute of Biomedical Chemistry Pogodinskaya str.</institution>
          <addr-line>10, Moscow, 119121</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2019</year>
      </pub-date>
      <fpage>47</fpage>
      <lpage>52</lpage>
      <abstract>
        <p>AIDS and the opportunistic infections and some other complications, associated with this syndrome lead to more than one million of human deaths per year. Human Immunodeficiency Virus is a cause of AIDS. The drugs, targeted proteins of HIV, can only lead to decrease of HIV copies in the human organism but still do not eliminate HIV from an organism. The main cause of antiretroviral drug therapy failure is HIV resistance to main drug classes: inhibitors of HIV structural proteins, protease and reverse transcriptase. One of the approaches to the classification of HIV variants into the resistant and susceptible ones is the use of machine learning methods. The aim of this work is the classification of the HIV variants into susceptible and resistant based on the nucleotide sequence of the HIV protease using neural networks. In our work, we used two topologies of neural networks: multilayer perceptron and convolutional neural network. Neural Networks were built using Python Tensor Flow and Keras libraries, where optimization of the neural networks can be performed. The training and test sets include experimental data on the nucleotide sequence of HIV protease and their resistance. Sensitivity, specificity, balanced accuracy were used as the main parameters, reflected the quality of classification. Those parameters were calculated for the test set, collected in the later period comparing to the sequences of the data set.</p>
      </abstract>
      <kwd-group>
        <kwd>and phrases</kwd>
        <kwd>neural networks</kwd>
        <kwd>HIV/AIDS</kwd>
        <kwd>inhibitors</kwd>
        <kwd>resistance</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted for private and
academic purposes. This volume is published and copyrighted by its editors.</p>
      <p>In: K. E. Samouylov, L. A. Sevastianov, D. S. Kulyabov (eds.): Selected Papers of the IX Conference
“Information and Telecommunication Technologies and Mathematical Modeling of High-Tech Systems”,
Moscow, Russia, 19-Apr-2019, published at http://ceur-ws.org</p>
    </sec>
    <sec id="sec-2">
      <title>1. Introduction</title>
      <p>Human immunodeficiency virus type 1 (HIV-1) is a retrovirus that causes the
Acquired Immunodeficiency Syndrome (AIDS) leading to the failure of immune system
and resulting in death. There are more than 36 HIV-infected people globally and
more than one million of them live in Russian Federation. Protocols for HIV/AIDS
treatment typically include use various combinations of antiretroviral drugs (Highly
Active Antiretroviral Treatment (HAART)). There are two main classes on antiretroviral
drugs included in HAART: inhibitors of structural HIV proteins, protease and reverse
transcriptase, encoded by single HIV gene named pol. Marketed antiretroviral drugs
are incapable of eliminating of virus from human organism. Mutations in the genes of
HIV cause its resistance to the drugs. HIV characterizes by a high velocity of mutations
occurrence, therefore it is able to develop resistance in short terms. HIV resistance
results in loss of drugs efects and necessity to use diferent drug combinations to prevent
viral replication. HIV resistance to the main antiretroviral classes of drugs is typically
estimated using two types of experimental tests: phenotypic tests and genotypic ones.
A genotypic test produces nucleotide sequences of the pol gene, while a phenotypic
test represents the data on the genotype of HIV resistant variant together with the
data on its resistance. The results of phenotypic and genotypic tests can be used to
develop on their basis the computational method aimed at predicting HIV resistance
to the particular antiretroviral drugs. There have been developed many of them and
an accuracy of prediction is over 90% (for some of them is more than 95%), for details,
please, see review by [1–4].</p>
      <p>
        There are several machine learning approaches for the prediction of HIV drug
resistance [
        <xref ref-type="bibr" rid="ref11 ref13 ref16">1, 2, 5–11</xref>
        ]. There were demonstrated using these approaches that an accuracy of
prediction of HIV resistance to a certain drug may vary depending on the type of
descriptor chosen, a drug, to which the resistance should be predicted. Application of neural
networks have become widely used in biology and chemistry for a past decade [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], [13].
The representation of a nucleotide sequence of HIV reverse transcriptase and protease as
a set of nucleotide fragments can be used for successful prediction of HIV drug resistance,
as was shown in a study by [2]. In the current work, we demonstrate the application of
the neural networks to the classification of HIV protease sequences into resistant and
susceptible groups based on the descriptors generated from the nucleotide sequences, as
we previously developed.
      </p>
      <p>2.</p>
    </sec>
    <sec id="sec-3">
      <title>The description of the data</title>
      <p>Nucleotide sequences of HIV protease were use as the training and test sets with the
data on their resistance produced using Phenosense test system for phenotypic tests.
In our study we used data on the HIV resistance to six marketed inhibitors of HIV
protease:
– fosamprenavir (FPV);
– azatanavir (ATV);
– indinavir (IDV);
– lopinavir (LPV);
– nelfinavir (NFV).</p>
      <p>The data include sequences of the pol collected using samples of the patients and
available from HIV Stanford drug resistance database. A detailed description of the
data processing is given in the study by [2]. In our earlier study we tried to simulate
a prospective validation using the set of the sequences collected from the patients,
examined no later than in January, 2006; sequences, collected later were used a the test
set. Nucleotide sequences were divided into the short sequences (descriptors), where
each short sequence was represented by a central nucleotide, eleven nucleotides before
the central one and twelve nucleotides after it. For each sequence, we generated the set
of descriptors. Then we collected all of them in a list and sorted by their frequency in
the whole set of nucleotide sequences of protease. We used 500 descriptors and generated
a set of binary descriptors from them, where each value of the descriptor was “1” if the
particular short sequence in the nucleotide sequence from the training or test set or “0”
if it is not in the while nucleotide sequence.</p>
      <p>3.</p>
    </sec>
    <sec id="sec-4">
      <title>The architecture of the neural networks</title>
      <p>
        In the current study we used two types of artificial neural networks: convolutional
neural networks (CNN) and multilayer perceptron (MLP). The neural network were built
using Python as the programming language and its libraries: Keras and TensorFlow.
These tools include several options to optimize the parameters for the neural networks
and to estimate an accuracy of the classification. Multilayer network included an input
layer, five hidden layers and an output layer. Rectifier Linear Unit (ReLU) was used as
a function of activation in hidden layers and a sigmoid function was used in the output
layer. A dropout option was used to prevent an overtraining [
        <xref ref-type="bibr" rid="ref17">14</xref>
        ].
      </p>
      <p>The architecture of the neural network, which is based on the convolution operation,
was first developed in the late 1990s by Lekun et al. [15]. Today convolutional neural
networks are considered to be the best for solving image recognition problems. In this
work we use the sequences of binary descriptors as training data that take the value «0»
or «1». Thus, this data can be assigned in the same way as images in recognition tasks.
This view allows one to apply the apparatus of convolutional neural networks.</p>
      <p>Convolutional neural network includes the input layer, the host matrix of size
22x22, two convolutional layers with feature map size 44x44 and 88x88, pooling layer
(MaxPooling), two hidden fully connected layers with 352 and 151 neurons and an output
layer. The convolution kernel in all layers is 3. ReLU is used as the activation function
in hidden layers, and sigmoid activation function is used on the output layer.</p>
      <p>
        To assess the quality of neural network for each of the 6 drugs were calculated
indicators such as [
        <xref ref-type="bibr" rid="ref19">16</xref>
        ]:
– Sensitivity, also known as recall, reflects the proportion of positive results that are
correctly identified by the classifier.
– Specificity — reflects the proportion of negative results that are correctly identified
by the classifier.
– Balanced accuracy is the proportion of true results (both positive and true negative)
among the total number of considered cases, i.e. the probability that the class will
be predicted correctly.
– The precision of the classification of positive results is the proportion of positive
results that are correctly identified by the classifier among the total number of
considered cases.
      </p>
      <p>To obtain an assessment of the classifier quality, a cross-validation with a 10-fold
division into training and test samples was used. In tables 1 and 2 the results of the
classifier are presented.</p>
      <sec id="sec-4-1">
        <title>Drug</title>
        <p>FPV
ATV
IDV
LPV
NFV
SQV</p>
        <p>Assessment of the neural network quality MLP</p>
      </sec>
      <sec id="sec-4-2">
        <title>Sensitivity</title>
      </sec>
      <sec id="sec-4-3">
        <title>Specificity</title>
      </sec>
      <sec id="sec-4-4">
        <title>Precision</title>
      </sec>
      <sec id="sec-4-5">
        <title>Balanced accuracy</title>
      </sec>
      <sec id="sec-4-6">
        <title>Drug</title>
        <p>FPV
ATV
IDV
LPV
NFV
SQV</p>
        <p>Assessment of the neural network quality CNN</p>
      </sec>
      <sec id="sec-4-7">
        <title>Sensitivity</title>
        <p>
          Low values of accuracy may be associated with a small amount of training sample, as
well as with the peculiarities of biological data on HIV resistance, which are characterized
by a certain incompleteness and heterogeneity [
          <xref ref-type="bibr" rid="ref20">4, 17, 18</xref>
          ].
        </p>
        <p>The analysis of the results showed that the classifier constructed with the help of
convolutional neural network for the majority of metrics and preparations give better
results than multilayer perceptron. In addition, convolutional neural network use almost
two times less parameters – 1 153 126 MLP parameters against 578 444 CNN parameters,
which improvs convergence and reducs computation time.</p>
        <p>4.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Conclusion</title>
      <p>The paper demonstrates the application of neural networks of diferent topologies to
the problem of classification of human immunodeficiency virus resistance to HIV protease
inhibitors. In the future we plan to improve the performance of the classifier through
the use of other topologies of neural networks, the selection of the optimal number of
layers, maps, features, sizes of the convolution kernel and other hyperparameters.</p>
      <p>The publication has been prepared with the support of the “RUDN University
Program 5-100”.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgments References</title>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <given-names>M.</given-names>
            <surname>Riemenschneider</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Senge</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Neumann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Hüllermeier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Heider</surname>
          </string-name>
          ,
          <article-title>Exploiting hiv-1 protease and reverse transcriptase cross-resistance information for improved drug resistance prediction by means of multi-label classification</article-title>
          ,
          <source>BioData Min 9</source>
          (
          <year>2016</year>
          )
          <fpage>10</fpage>
          -
          <lpage>10</lpage>
          , 26933450[pmid].
          <source>doi:10.1186/s13040-016-0089-1.</source>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          URL https://www.ncbi.nlm.nih.gov/pubmed/26933450
          <string-name>
            <given-names>O.</given-names>
            <surname>Tarasova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Biziukova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Filimonov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Poroikov</surname>
          </string-name>
          ,
          <article-title>A computational approach for the prediction of hiv resistance based on amino acid and nucleotide descriptors</article-title>
          ,
          <source>Molecules</source>
          <volume>23</volume>
          (
          <issue>11</issue>
          ) (
          <year>2018</year>
          )
          <volume>2751</volume>
          , 30355996[pmid].
          <source>doi:10</source>
          .3390/molecules23112751.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          URL https://www.ncbi.nlm.nih.gov/pubmed/30355996 M. J.
          <string-name>
            <surname>Dapp</surname>
            ,
            <given-names>R. H.</given-names>
          </string-name>
          <string-name>
            <surname>Heineman</surname>
            ,
            <given-names>L. M.</given-names>
          </string-name>
          <string-name>
            <surname>Mansky</surname>
          </string-name>
          ,
          <article-title>Interrelationship between hiv-1 iftness and mutation rate</article-title>
          ,
          <source>J Mol Biol</source>
          <volume>425</volume>
          (
          <issue>1</issue>
          ) (
          <year>2013</year>
          )
          <fpage>41</fpage>
          -
          <lpage>53</lpage>
          , 23084856[pmid].
          <source>doi: 10</source>
          .1016/j.jmb.
          <year>2012</year>
          .
          <volume>10</volume>
          .009.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          URL https://www.ncbi.nlm.nih.gov/pubmed/23084856
          <string-name>
            <given-names>O.</given-names>
            <surname>Tarasova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Poroikov</surname>
          </string-name>
          ,
          <article-title>Hiv resistance prediction to reverse transcriptase inhibitors: Focus on Open Data</article-title>
          ,
          <source>Molecules</source>
          <volume>23</volume>
          (
          <issue>4</issue>
          ) (
          <year>2018</year>
          )
          <volume>956</volume>
          , 29671808[pmid].
          <source>doi:10</source>
          .3390/ molecules23040956.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          URL https://www.ncbi.nlm.nih.gov/pubmed/29671808 5.
          <string-name>
            <given-names>N.</given-names>
            <surname>Beerenwinkel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Schmidt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Walter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Kaiser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Lengauer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Hofmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Korn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Selbig</surname>
          </string-name>
          ,
          <article-title>Diversity and complexity of hiv-1 drug resistance: A bioinformatics approach to predicting phenotype from genotype</article-title>
          ,
          <source>Proceedings of the National Academy of Sciences</source>
          <volume>99</volume>
          (
          <issue>12</issue>
          ) (
          <year>2002</year>
          )
          <fpage>8271</fpage>
          -
          <lpage>8276</lpage>
          . arXiv:https: //www.pnas.org/content/99/12/8271.full.pdf, doi:10.1073/pnas.112177799.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          URL https://www.pnas.org/content/99/12/8271 6.
          <string-name>
            <given-names>N.</given-names>
            <surname>Beerenwinkel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Däumer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Oette</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Korn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Hofmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Kaiser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Lengauer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Selbig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Walter</surname>
          </string-name>
          ,
          <article-title>Geno2pheno: Estimating phenotypic drug resistance from HIV-1 genotypes</article-title>
          ,
          <source>Nucleic Acids Res</source>
          <volume>31</volume>
          (
          <issue>13</issue>
          ) (
          <year>2003</year>
          )
          <fpage>3850</fpage>
          -
          <lpage>3855</lpage>
          , 12824435[pmid].
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          URL https://www.ncbi.nlm.nih.gov/pubmed/12824435 7. S.-Y. Rhee,
          <string-name>
            <given-names>J.</given-names>
            <surname>Taylor</surname>
          </string-name>
          , G. Wadhera,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ben-Hur</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. L.</given-names>
            <surname>Brutlag</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. W.</given-names>
            <surname>Shafer</surname>
          </string-name>
          ,
          <article-title>Genotypic predictors of human immunodeficiency virus type 1 drug resistance</article-title>
          ,
          <source>Proceedings of the National Academy of Sciences</source>
          <volume>103</volume>
          (
          <issue>46</issue>
          ) (
          <year>2006</year>
          )
          <fpage>17355</fpage>
          -
          <lpage>17360</lpage>
          . arXiv:https:// www.pnas.org/content/103/46/17355.full.pdf, doi:10.1073/pnas.0607274103.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          URL https://www.pnas.org/content/103/46/17355 8.
          <string-name>
            <given-names>D.</given-names>
            <surname>Heider</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Verheyen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Hofmann</surname>
          </string-name>
          ,
          <article-title>Machine learning on normalized protein sequences</article-title>
          ,
          <source>BMC Res Notes</source>
          <volume>4</volume>
          (
          <year>2011</year>
          )
          <fpage>94</fpage>
          -
          <lpage>94</lpage>
          , 21453485[pmid].
          <source>doi:10</source>
          .1186/
          <fpage>1756</fpage>
          -0500-4-94.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          URL https://www.ncbi.nlm.nih.gov/pubmed/21453485 9.
          <string-name>
            <surname>G. J. P. van Westen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Hendriks</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. K.</given-names>
            <surname>Wegner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. P.</given-names>
            <surname>Ijzerman</surname>
          </string-name>
          , H. W. T. van
          <string-name>
            <surname>Vlijmen</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Bender</surname>
          </string-name>
          ,
          <article-title>Significantly improved hiv inhibitor eficacy prediction employing proteochemometric models generated from antivirogram data</article-title>
          ,
          <source>PLoS Comput Biol</source>
          <volume>9</volume>
          (
          <issue>2</issue>
          ) (
          <year>2013</year>
          )
          <fpage>e1002899</fpage>
          -
          <lpage>e1002899</lpage>
          , 23436985[pmid].
          <source>doi:10</source>
          .1371/journal.pcbi.
          <volume>1002899</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          URL https://www.ncbi.nlm.nih.gov/pubmed/23436985 10.
          <string-name>
            <given-names>O.</given-names>
            <surname>Tarasova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Filimonov</surname>
          </string-name>
          ,
          <string-name>
            <surname>P. V.V.</surname>
          </string-name>
          ,
          <article-title>Computational prediction of human immunodeficiency resistance to reverse transcriptase inhibitors</article-title>
          ,
          <source>Biomeditsinskaya khimiya 63 (5)</source>
          (
          <year>2017</year>
          )
          <fpage>457</fpage>
          -
          <lpage>460</lpage>
          . doi:
          <volume>10</volume>
          .18097/PBMC20176305457.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>O.</given-names>
            <surname>Tarasova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Filimonov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Poroikov</surname>
          </string-name>
          ,
          <article-title>Pass-based approach to predict hiv-1 reverse transcriptase resistance</article-title>
          ,
          <source>Journal of Bioinformatics and Computational Biology</source>
          <volume>15</volume>
          (
          <issue>02</issue>
          ) (
          <year>2017</year>
          )
          <volume>1650040</volume>
          , pMID:
          <fpage>28033735</fpage>
          . doi:
          <volume>10</volume>
          .1142/S0219720016500402.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <given-names>I. I.</given-names>
            <surname>Baskin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Winkler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I. V.</given-names>
            <surname>Tetko</surname>
          </string-name>
          ,
          <article-title>A renaissance of neural networks in drug discovery</article-title>
          ,
          <source>Expert Opinion on Drug Discovery</source>
          <volume>11</volume>
          (
          <issue>8</issue>
          ) (
          <year>2016</year>
          )
          <fpage>785</fpage>
          -
          <lpage>795</lpage>
          , pMID:
          <fpage>27295548</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <source>doi:10.1080/17460441</source>
          .
          <year>2016</year>
          .
          <volume>1201262</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          URL https://doi.org/10.1080/17460441.
          <year>2016</year>
          .
          <volume>1201262</volume>
          13. D.
          <string-name>
            <surname>Dana</surname>
            ,
            <given-names>S. V.</given-names>
          </string-name>
          <string-name>
            <surname>Gadhiya</surname>
            ,
            <given-names>L. G.</given-names>
          </string-name>
          <string-name>
            <surname>St Surin</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Naaz</surname>
            ,
            <given-names>Q.</given-names>
          </string-name>
          <string-name>
            <surname>Ali</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Paka</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Yamin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Narayan</surname>
            ,
            <given-names>I. D.</given-names>
          </string-name>
          <string-name>
            <surname>Goldberg</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Narayan</surname>
          </string-name>
          ,
          <article-title>Deep learning in drug discovery and medicine; scratching the surface</article-title>
          ,
          <source>Molecules</source>
          <volume>23</volume>
          (
          <issue>9</issue>
          ) (
          <year>2018</year>
          )
          <volume>2384</volume>
          , 30231499[pmid].
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>doi:10</source>
          .3390/molecules23092384.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          14.
          <string-name>
            <given-names>N.</given-names>
            <surname>Srivastava</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Hinton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Krizhevsky</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Sutskever</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Salakhutdinov</surname>
          </string-name>
          ,
          <article-title>Dropout: A Simple Way to Prevent Neural Networks from Overfitting</article-title>
          ,
          <source>Journal of Machine Learning Research</source>
          <volume>15</volume>
          (
          <year>2014</year>
          )
          <fpage>1929</fpage>
          -
          <lpage>1958</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          URL http://jmlr.org/papers/v15/srivastava14a.html 15. Y.
          <string-name>
            <surname>Lecun</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Bottou</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <string-name>
            <surname>Bengio</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Hafner</surname>
          </string-name>
          ,
          <article-title>Gradient-based learning applied to document recognition</article-title>
          ,
          <source>Proceedings of the IEEE</source>
          <volume>86</volume>
          (
          <issue>11</issue>
          ) (
          <year>1998</year>
          )
          <fpage>2278</fpage>
          -
          <lpage>2324</lpage>
          . doi:
          <volume>10</volume>
          .1109/5.726791.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          16.
          <string-name>
            <surname>D. M. W. Powers</surname>
          </string-name>
          ,
          <article-title>Evaluation: From precision, recall and f-measure to roc., informedness, markedness &amp; correlation</article-title>
          ,
          <source>Journal of Machine Learning Technologies</source>
          <volume>2</volume>
          (
          <issue>1</issue>
          ) (
          <year>2011</year>
          )
          <fpage>37</fpage>
          -
          <lpage>63</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          17.
          <string-name>
            <given-names>O. A.</given-names>
            <surname>Tarasova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. F.</given-names>
            <surname>Urusova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. A.</given-names>
            <surname>Filimonov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. C.</given-names>
            <surname>Nicklaus</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. V.</given-names>
            <surname>Zakharov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. V.</given-names>
            <surname>Poroikov</surname>
          </string-name>
          ,
          <article-title>Qsar modeling using large-scale databases: Case Study for HIV-1 Reverse Transcriptase Inhibitors</article-title>
          ,
          <source>Journal of Chemical Information and Modeling</source>
          <volume>55</volume>
          (
          <issue>7</issue>
          ) (
          <year>2015</year>
          )
          <fpage>1388</fpage>
          -
          <lpage>1399</lpage>
          . doi:
          <volume>10</volume>
          .1021/acs.jcim.5b00019.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          URL https://doi.org/10.1021/acs.jcim.
          <year>5b00019</year>
          18.
          <string-name>
            <given-names>D.</given-names>
            <surname>Fourches</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Muratov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Tropsha</surname>
          </string-name>
          ,
          <article-title>Curation of chemogenomics data</article-title>
          ,
          <source>Nature Chemical Biology</source>
          <volume>11</volume>
          (
          <year>2015</year>
          )
          <article-title>535, correspondence</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>URL https://doi.org/10.1038/nchembio.1881</mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>