<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>IWSG</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>A Generic Framework and Methodology for Implementing Science Gateways for Analysing Molecular Docking Results</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Damjan Temelkovski</string-name>
          <email>damjan.temelkovski@my.westminster.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Tamas Kiss</string-name>
          <email>t.kiss@westminster.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Gabor Terstyanszky</string-name>
          <email>g.z.terstyanszky@westminster.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of Westminster</institution>
          ,
          <addr-line>London</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2018</year>
      </pub-date>
      <volume>13</volume>
      <fpage>13</fpage>
      <lpage>15</lpage>
      <abstract>
        <p>-Molecular docking and virtual screening experiments require large computational and data resources and high-level user interfaces in the form of science gateways. While science gateways supporting such experiments are relatively common, there is a clearly identified need to design and implement more complex environments for further analysis of docking results. This paper describes a generic framework and a related methodology that supports the efficient development of such environments. The framework is modular enabling the reuse of already existing components. The methodology is agile and encourages the input and participation of end-users. A prototype implementation, based on the framework and methodology, of a science-gateway-based molecular docking environment for recommending a ligand-protein pair for next docking experiment is also presented and evaluated.</p>
      </abstract>
      <kwd-group>
        <kwd>bioinformatics</kwd>
        <kwd>modelling</kwd>
        <kwd>molecular docking</kwd>
        <kwd>science gateway</kwd>
        <kwd>virtual screening</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>I. INTRODUCTION
Molecular docking is a computational simulation that models
biochemical interactions to predict where and how two
molecules would bind. Large-scale molecular docking
simulations are used in areas such as drug discovery where
they can decrease the amount of wet-lab experiments required.
Since molecular docking uses the structure of the receptor,
large-scale molecular docking of hundreds of thousands of
ligands and one receptor is called structure-based virtual
screening (virtual as opposed to the robotics-based high
throughput screening). Although a single docking simulation
is relatively short, a typical virtual screening experiment, that
may combine thousands of simulations, is computationally
demanding, requiring the use of Distributed Computing
Infrastructures (DCIs). Utilising and accessing such
computational resources adds an extra level of complexity to
the task making it increasingly difficult for biomedical
scientists. Science gateways are widely utilised in this area to
help bridging this gap.</p>
      <p>
        Although this field has seen great advancements recently,
feedback from biomedical scientists shows that there is still a
significant gap to bridge. Examples for science gateways
supporting molecular docking and virtual screening
experiments include several WS-PGRADE/gUSE [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] based
This work was partially supported by the COLA Cloud Orchestration at the
level of Applications project, Project No. 731574.
gateways, such as the MosGrid Portal [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], the AutoDock
Gateway [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], and the AMC Docking Gateway [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]; as well as
non-workflow-based pipelines such as the virtual screening
environment for Windows Azure [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], the supercomputer-based
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] or the Linux cluster-based [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] virtual screening pipelines.
However, there is still a need for more complex environments
that enable scientists to access a wide range of computing,
data and network resources for the further analysis of docking
results. Such environments should support complex scenarios
where intelligent support can be provided for the more
efficient execution of large-scale molecular docking
experiments.
      </p>
      <p>This paper investigates such scenarios and proposes a generic
conceptual framework to support the analysis of molecular
docking results, and a related methodology that uses regular
input from scientists when developing complex
sciencegateway-based environments for the storage, analysis and
reuse of molecular docking results. It has been developed
considering biomedical scientists’ requirements collected from
semi-structured interviews and a literature review of 14 related
projects including those mentioned in the paragraph above.
From this generic framework, specific architectures can be
derived supporting various molecular-docking-related
analytical scenarios as shown in Section II. Additionally, a
software development methodology that supports creating
docking experiments based on this framework is explained in
Section III. Finally, a prototype implementation of such
system is presented in Section IV.</p>
      <p>II. GENERIC FRAMEWORK FOR THE ANALYSIS OF MOLECULAR</p>
      <p>DOCKING RESULTS
The aim of our research was to identify potential similarities in
the work of biomedical scientists working with molecular
docking experiments, and to investigate whether a generic
framework for such application scenarios can be defined. The
assumption was that based on this generic framework more
specific science gateway based environments can be
implemented supporting different application scenarios. As
these scenarios have large similarity, deriving and
implementing such specific environments can be speeded up
significantly. In other words, the aim was to formalise and
speed up the development of specific science gateway
environments supporting various molecular docking scenarios.
In order to identify typical user requirements, several
interviews with five scientists from different backgrounds and
with various degrees of experience with molecular docking
simulations were conducted. Since the number of the
interviewees was small and the population localised in London,
this is a not a representative sample of the world-wide
population of scientists that use molecular docking simulations.
However, considering its diversity, the sample was useful in
producing several conclusions. The interviews aimed at
identifying requirements of the scientists when performing
molecular docking experiments and specifying scenarios that
are not supported by currently available science gateways for
molecular docking. These scenarios typically represent
software systems that make a decision based on the molecular
docking results, mimicking the steps that a scientist needs to
take after obtaining the results. Some representative and
identified scenarios are listed below:
1. Suggest a ligand-protein pair that should be used in the
next molecular docking, based on protein similarity and
previous results
2. Filter docking results which are suitable for wet
laboratory experiments, based on ligand properties
3. Find off-target drugs, based on deducing if the estimated
binding is at an active site
4. Enable verification of the docking methodology and
learning from previous docking for novice users
5. Compare results from different molecular docking tools
Based on the conceptual similarities of these scenarios and an
extended review of literature, a generic framework has been
designed. The design focuses on the similar elements in the
scenarios and includes the following components (see Figure
1):
Molecular Docking Environment (MDE): All scenarios
include an environment where the molecular docking
simulation is executed. It could be as simple as running a
single simulation from the command line on a local computer,
to more complex such as executing a virtual screening
experiment on a DCI. This environment includes the software
tool used for the docking itself, and may also include
additional elements to connect to a DCI or to provide a high
level user interface.</p>
      <p>Molecular Docking Results Repository (MDRR): After the
execution of the molecular docking, the results need to be
stored as previous molecular docking results are needed by
various scenarios. The repository should also store
information about the final decision made by the whole
simulation environment.</p>
      <p>Additional Tool (AT): The results which have been stored in
the MDRR are then processed by an AT. This is a generic
element that describes a tool which takes one or more
molecular docking results as input and conducts a calculation.
ATs can refer back to other molecular docking results stored
in the MDRR, communicate with other ATs, or refer to data
stored in an Additional Data Source.</p>
      <p>Additional Data Source (ADS): It contains data that is
relevant for the final decision and usually is an external
database.</p>
      <p>Decision Maker (DM): All the information processed from
the various ATs is passed to a DM. This element groups and
analyses the calculations performed by the ATs in order to
make a decision.</p>
      <p>The numbers in Figure 1 present the order or flow of events
through the different elements:
1. A scientist uses an MDE to conduct the molecular
docking and the result is uploaded to the MDRR.
2. The MDRR sends the results to one or more ATs.
3. An AT may communicate with one or more other ATs.
4. An AT may look up data stored in the ADS.
5. An AT may require additional previous molecular
docking results as input for its calculation.
6. An AT would provide its calculation results to the DM.
7. The MDRR may use data from the ADS directly.
8. Previous results from the MDRR may be used by the DM
9. The DM may use data from the ADS directly.
10. Once the analysis is complete and the decision is made, it
can be passed back to the MDRR.
11. Finally, the decision is passed to the MDE to visualise it.
From this generic framework each specific scenario
introduced earlier, and also the ones covered in the literature
review can be derived. For illustration, a basic architecture
diagram for the first scenario is shown in Figure 2. Similar
figures for each scenario have been designed and analysed
demonstrating that the framework is generic enough to support
at least the five identified scenarios and the 14 related
solutions covered in the literature. However, these figures are
not presented here due to limitations in length of the paper.
In Scenario 1 (Figure 2) the framework would analyse
previous molecular docking results and look for good docking
results that have used a receptor similar to the currently used
receptor. Based on this, the system suggests a new
proteinligand pair that would be an interesting candidate for docking.
Two key issues here are the definitions of good docking result
or similar receptor.</p>
      <p>
        In Figure 2 the building blocks of the Generic Framework
have been replaced with concrete elements supporting this
particular scenario. One of the advantages of this modular
design is that these building blocks can be easily replaced with
other elements if necessary. This way multiple existing tools
can be integrated into the scenario design and evaluated,
requiring only the implementation of components that are not
currently available. Mapping of the generic framework for
this particular scenario in the presented example is as follows:
The MDE is an extended version of the popular Racoon2 [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]
desktop application, a virtual screening environment. The
WSPGRADE/gUSE science gateway framework was integrated
with Raccoon2 to support large-scale experiments on
heterogeneous cloud computing resources, as it was presented
in [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. The MDRR is a custom-made repository based on a
MongoDB database. Three ATs are utilised in this scenario.
The structural alignment tool DeepAlign [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] is used to
calculate similarities between receptors. A custom-made AT is
used to assess whether the structural alignment result means
that the two receptors are similar, while another custom-made
AT is required to assess a docking result and categorise it as
good. Finally, a custom-made DM is needed to suggest which
protein-ligand pair to dock next.
      </p>
      <p>Figure 2 – Basic diagram of scenario to suggest a
ligandprotein pair for next docking (Scenario 1)
The flow of events is shown in Figure 2. Raccoon2 executes
the molecular docking and the results are uploaded to the
MDRR (1). The MDRR sends the receptor pairs to DeepAlign
(2). The results of DeepAlign are assessed by the
custommade AT (3) that sends the results to the MDRR (4) and the
DM (5). All past docking results of similar receptors are sent
to be assessed (6) and the good results are sent to the DM. The
DM combines the results from the ATs, and suggests which
protein-ligand pair to dock as a next step. This suggestion is
returned to the MDRR and stored as meta-data (8). Finally, it
is presented to the user (9).</p>
      <p>
        Based on the basic generic architecture of Figure 1, a more
detailed framework has been developed that consist of a
diagram, a textual description of elements and interfaces, and
a formal description using Z-notation [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. The aim of this
framework is to describe the generic architecture and the way
how the specific scenarios are derived from this in a
formalised way. Based on this formalism we aim to support
application developers to make specific decisions when
evaluating and implementing these scenarios. The designed
framework is independent from the actual implementation, or
indeed, the programming language of choice.
      </p>
      <p>The diagram representing the framework in Figure 3 is a
generic model, showing all generic elements and all possible
interfaces between them. It is based on the UML Component
Diagram in the sense that the elements are drawn as
components and the interfaces between them are the typical
provided and required interface connections. Additionally, it
features arrows pointing towards the direction of the flow of
data in a particular interface.</p>
      <p>User → MDE, provided by the MDE: allows the user to
upload the correct input for the molecular docking or
additional user input values needed by another element.
MDE → user, provided by the MDE: displays the result
of the molecular docking to the user, along with other
results from the MDRR.</p>
      <p>Following this, each element and each interface have been
described formally using Z-notation. As the set of descriptions
is too extensive for this paper, only a representative example is
presented here, describing the MDE and its interfaces (see
Figures 4 and 5).</p>
      <p>
        The docking process expressed by the MDE needs a ligand,
receptor, and optionally configuration (config) files as input,
and provides a docking result file as output. When there is no
config file then the dockingWithoutConfig() function will
generate the docking result, while when there is a config file
then the dockingWithConfig() function will do it. Furthermore,
the Z-notation for dockingWithoutConfig() describes that for
every ligand × receptor pair, as long as the ligand and receptor
are not empty files, there exists a docking result. Similarly,
dockingWithConfig() defines that for each ligand and for each
receptor there exists a configuration file that can be used to
produce a docking result. The corresponding Z-notation
descriptions can be seen in Figure 4.
Based on the above detailed description of the generic
framework, a detailed architecture diagram of each scenario
can now be derived followed by the textual and formal
descriptions of these scenarios. Figure 6 shows part of the
detailed architecture diagram of Scenario 1, representing the
extended Raccoon2 as an MDE, and corresponding to that part
of the basic diagram of Figure 2. In Figure 7 the formal
description of this module is shown. (Please note that full
diagram and description are not provided due to limitation of
length, but has been produced.)
This section describes the methodology for developing
complex environments that reuse and analyse molecular
docking results. This methodology complements the
framework described in the previous section by explaining
how this framework can be used during development. It
clearly states the roles that are required and the specific
subprojects for which they need to collaborate. The methodology
is based on the seven principles identified by Cockburn [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ].
Based on Cockburn’s general recommendations, a
roledeliverable-milestone diagram has been created to represent
the methodology (Figure 8). This diagram illustrates that the
modeller, biomedical scientist and bioinformatician should
collaborate when creating the diagram and textual description
of the scenario. Furthermore, the modeller should collaborate
with the bioinformatician and the software developer when
creating the formal description. Key components of this
diagram, extensions to Cockburn's original model, are the
dotted lines which show that the process is agile. For instance,
in the top section where the life scientist works on the textual
description and go from milestone M4 to M5, there is a dotted
line showing that (s)he could revisit and alter the diagram if
necessary. The same logic is used for the agile development of
the final system code. Figure 8 presents a high level
roledeliverable-milestone diagram where the coding section has an
asterisk (*) indicating that a similar but more detailed
description of this section (not presented in this paper) has
also been developed in the form of a lower-level diagram.
zip files as POST parameters. It parses them and inserts
information into MongoDB, which includes the collections
receptors, ligands, results, and analysis. Another request is
sent to continue with Scenario 1 where the MDRR selects all
receptors from the database, parses and compresses them.
Next, these are sent to Server 2 along with the target receptor
(the receptor used in the original simulation), and a threshold
value (input by the user in Raccoon2). The first AT on Server
2 executes DeepAlign to find similarities between the target
receptor, and each different receptor it received. It then calls
the AT: AssessDeepAlign, located on the same server, in order
to select the similar receptors. In the simplest form of this AT,
it assesses the DeepAlign results by comparing the value of
DeepScore to a user input threshold. A list of these similar
receptors is returned to Server 1 where the analysis collection
is updated to keep track of the events so far. Then, the MDRR
selects past docking results which have used one of the similar
receptors, and compresses them. It sends a request to Server 3,
including a threshold value of the AutoDock Vina affinity,
entered by the user within Raccoon2.
In order to demonstrate how the developed framework and
methodology support implementing molecular docking
science gateways, an implementation of Scenario 1
(https://github.com/damjanmk/mdrr-scenarios) is presented
here. All components in the implementation are accessible via
a basic RESTful API. We used Bottle [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], a minimalist
webframework which enables easy server setup. The MDRR and
the DM have been deployed on Server 1, the DeepAlign AT
and the AT to assess the DeepAlign results on Server 2, while
the docking assessment AT on Server 3 (Figure 9). In order to
insert results from Raccoon2, the MDRR on Server 1 expects
The AT on Server 3 searches through the Vina results for a
result that has at least one model where the Vina affinity is
less than the threshold, and calls this a good docking result (a
Vina docking result can contain for example 10 models). It
returns a list of good docking results to Server 1.
      </p>
      <p>Upon receiving this, Server 1 inserts a document in the
analysis collection before initialising the DM and sending it
the similar receptors and the good results. The DM combines
these two lists into one and sorts it based firstly on the
DeepScore value, then on the affinity. This enables users to
view an ordered list of results that contain ligands which are
suggested for a subsequent docking.</p>
    </sec>
    <sec id="sec-2">
      <title>A. Designing the MongoDB database</title>
      <p>
        At the core of this custom-made MDRR is a MongoDB
database. There were several reasons why we chose this type
of non-relational database:
1. MongoDB’s schеma-less design is ideal because a single
collection can be used for: input files in different formats,
output files of any of the over 50 docking tools [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], or
meta-data about different ATs.
      </p>
      <p>MongoDB scales very well for large amounts of data,
provided it is well designed and features such as sharding
and indexing are utilised.</p>
      <p>
        MongoDB is well-suited for prototyping because it is
easier to change what is stored during development.
In this prototype implementation we have considered .pdbqt
molecules and AutoDock Vina results (as used by Raccoon2).
The ligands collection contains molecular properties
calculated using the OpenBabel and PyBel [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] Python
modules such as canonical_SMILES, logP, mol_weight, etc.
Biomedical scientists at the University of Westminster were
consulted when deciding which properties to store. Both the
ligands and receptors collections include the full parsed 3D
structure from the .pdbqt files. Each line of the .pdbqt file is
stored as an element of an array. The structure of each
molecule should be unique. However, the structure itself
cannot be uniquely indexed due to size limitations, so we have
introduced structure_id - an MD5 hash of the structure. This
uniquely describes the structure and allows for a MongoDB
index to be created.
      </p>
      <p>The results collection contains references to the ligand and
receptor used, specific properties extracted from the result
files (e.g. CPUs, random_seed), a list of the result models,
each model containing affinity, rmsd_from_best, and the
parsed model segment of the Vina result. The parsing process
is simple – it stores all lines between MODEL and ENDMDL
as elements of an array.</p>
    </sec>
    <sec id="sec-3">
      <title>B. Use of the framework and methodology</title>
      <p>The framework was followed as described in Section II. A list
of documented meetings and events is not presented with this
paper, but serves as supporting evidence of following the
methodology. The required roles were taken up by different
researchers at the University of Westminster (with some
doubling as multiple roles). The presented implementation
proves that following the methodology such molecular
docking framework can be implemented. Work is currently
ongoing to quantify advantages when compared to more
adhoc implementation.</p>
    </sec>
    <sec id="sec-4">
      <title>C. Limitations of the prototype implementation</title>
      <p>Due to the Global Interpreter Lock (GIL), Python is not the
optimal language for multi-threading without additional
optimisations. Furthermore, Bottle uses a non-threading type
of servers by default, so using a different specialised server
would improve performance for simultaneous users. The
number of items in the collections may become too big to be
included in one zip file which is used to transfer data from
servers and sending large files through the network could be a
bottleneck. Finally, the current DM joins and sorts two lists
without specific performance optimisations.</p>
      <p>V. CONCLUSION AND FUTURE WORK
This paper presented a generic framework and a corresponding
methodology to implement complex science-gateway-based
environments for the execution of molecular docking
experiments extended with the intelligent analysis and
utilisation of docking results. The framework incorporates a
diagram, and textual and formal description enabling a modular
design and the replacement and reuse of components. The
methodology involves multiple stakeholders and requires their
collaboration in an agile manner. In order to demonstrate the
usability of the above, a scenario for suggesting a
ligandprotein pair for next docking was also presented.</p>
      <p>Future work includes the implementation and detailed
evaluation of multiple scenarios to identify, and where possible
quantify, the advantages provided by the framework and
methodology. In order to achieve this, the implemented
solutions are compared to state-of-the-art methods and
environments to demonstrate the added value of our research.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>P.</given-names>
            <surname>Kacsuk</surname>
          </string-name>
          et al.,
          <article-title>“WS-PGRADE/gUSE generic DCI gateway framework for a large variety of user communities</article-title>
          ”
          <source>J. Grid Comput.</source>
          , vol.
          <volume>10</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>601</fpage>
          -
          <lpage>630</lpage>
          , Dec,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>J.</given-names>
            <surname>Krüger</surname>
          </string-name>
          et al., “
          <article-title>The MoSGrid Science Gateway - A Complete Solution for Molecular Simulations”</article-title>
          ,
          <source>J. Chem. Theory Comput.</source>
          , vol.
          <volume>10</volume>
          , no 6, pp.
          <fpage>2232</fpage>
          -
          <lpage>2245</lpage>
          , May,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Farkas</surname>
          </string-name>
          et al.,
          <article-title>“AutoDock gateway for user friendly execution of molecular docking simulations in cloud systems”, in Cloud Computing with E-science</article-title>
          <string-name>
            <surname>Applications</surname>
          </string-name>
          , Olivier Terzo, Lorenzo Mossucca, Eds. Boca Raton, FL: CRC Press/Taylor &amp; Francis,
          <year>2015</year>
          , pp
          <fpage>217</fpage>
          -
          <lpage>236</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>M.</given-names>
            <surname>Jaghoori</surname>
          </string-name>
          et al.,
          <article-title>“A multi-infrastructure gateway for virtual drug screening”, Concurr</article-title>
          . Comp. Pract. E., vol.
          <volume>27</volume>
          , no.
          <issue>16</issue>
          , pp.
          <fpage>4478</fpage>
          -
          <lpage>4490</lpage>
          , Nov,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>T.</given-names>
            <surname>Kiss</surname>
          </string-name>
          et al., “
          <article-title>Large-scale virtual screening experiments on Windows Azure-based cloud resources”, Concurr</article-title>
          . Comp. Pract. E., vol.
          <volume>26</volume>
          , no 10, pp.
          <fpage>1760</fpage>
          -
          <lpage>1770</lpage>
          , Jul,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>X.</given-names>
            <surname>Zhang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. E.</given-names>
            <surname>Wong</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F. C.</given-names>
            <surname>Lightstone</surname>
          </string-name>
          , “
          <article-title>Toward fully automated high performance computing drug discovery: a massively parallel virtual screening pipeline for docking and molecular mechanics/generalized born surface area rescoring to improve enrichment”</article-title>
          ,
          <string-name>
            <surname>J. Chem. Inf. Model.</surname>
          </string-name>
          , vol.
          <volume>54</volume>
          , no 1, pp.
          <fpage>324</fpage>
          -
          <lpage>337</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>P. D'Ursi</surname>
          </string-name>
          et al., “
          <article-title>Virtual screening pipeline and ligand modelling for H5N1 neuraminidase”</article-title>
          ,
          <source>Biochem. Biophys. Res. Commun.</source>
          , vol.
          <volume>383</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>445</fpage>
          -
          <lpage>449</lpage>
          , Jun,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>S.</given-names>
            <surname>Forli</surname>
          </string-name>
          et al., “
          <article-title>Computational protein-ligand docking and virtual drug screening with the AutoDock suite”</article-title>
          .
          <source>Nat. Protoc.</source>
          , vol.
          <volume>11</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>905</fpage>
          -
          <lpage>919</lpage>
          , Apr.
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>D.</given-names>
            <surname>Temelkovski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Kiss</surname>
          </string-name>
          , and G. Terstyanszky, “
          <article-title>Molecular docking with Raccoon2 on clouds: extending desktop applications with cloud computing”</article-title>
          ,
          <source>in 9th International Workshop on Science Gateways</source>
          , Poznań, Poland,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>S.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Ma</surname>
          </string-name>
          , J. Peng, and
          <string-name>
            <given-names>J.</given-names>
            <surname>Xu</surname>
          </string-name>
          , “
          <article-title>Protein structure alignment beyond spatial proximity”</article-title>
          ,
          <source>Sci Rep</source>
          vol.
          <volume>3</volume>
          , no.
          <issue>1448</issue>
          ,
          <string-name>
            <surname>Mar</surname>
          </string-name>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>J. M. Spivey</surname>
          </string-name>
          ,
          <string-name>
            <surname>The Z Notation - A Reference Manual</surname>
          </string-name>
          , 2nd ed. Oxford, UK: Oriel College,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>A.</given-names>
            <surname>Cockburn</surname>
          </string-name>
          ,
          <source>Agile Software Development: The Cooperative Game</source>
          , 2nd ed. Boston, MA: Addison Wesley,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>M.</given-names>
            <surname>Hellkamp</surname>
          </string-name>
          . Bottle: Python Web Framework [Online]. Available: https://bottlepy.org/docs/dev/.
          <source>[Accessed: 6 Mar</source>
          <year>2018</year>
          ]
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <article-title>Swiss Institute of Bioinformatics. Directory of in-silico Drug Design tools - Docking [Online]</article-title>
          . Available: https://www.click2drug.org /directory_Docking.html.
          <source>[Accessed: 6 Mar</source>
          <year>2018</year>
          ]
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>N. M. O'Boyle</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Morley</surname>
            , and
            <given-names>G. R.</given-names>
          </string-name>
          <string-name>
            <surname>Hutchison</surname>
          </string-name>
          , “
          <article-title>Pybel: a Python wrapper for the OpenBabel cheminformatics toolkit”, Chem</article-title>
          . Cent. J., vol.
          <volume>2</volume>
          , no.
          <issue>5</issue>
          ,
          <string-name>
            <surname>Mar</surname>
          </string-name>
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>