<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Connecting X! Tandem to a database management system</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>General Terms</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
          <xref ref-type="aff" rid="aff4">4</xref>
          <xref ref-type="aff" rid="aff5">5</xref>
          <xref ref-type="aff" rid="aff6">6</xref>
          <xref ref-type="aff" rid="aff7">7</xref>
          <xref ref-type="aff" rid="aff8">8</xref>
          <xref ref-type="aff" rid="aff9">9</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Atin Janki Working Group Databases and Software Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>David Broneske Working Group Databases and Software Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Design</institution>
          ,
          <addr-line>Performance</addr-line>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Dirk Benndorf Chair of Bioprocess Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff4">
          <label>4</label>
          <institution>Gunter Saake Working Group Databases and Software Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff5">
          <label>5</label>
          <institution>Kay Schallert Chair of Bioprocess Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff6">
          <label>6</label>
          <institution>Robert Heyer Chair of Bioprocess Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff7">
          <label>7</label>
          <institution>Rohith Ravindran Working Group Databases and Software Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff8">
          <label>8</label>
          <institution>Roman Zoun Working Group Databases and Software Engineering University of Magdeburg</institution>
        </aff>
        <aff id="aff9">
          <label>9</label>
          <institution>Wolfram Fenske Working Group Databases and Software Engineering University of Magdeburg</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2018</year>
      </pub-date>
      <abstract>
        <p />
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Protein identi cation by mass spectrometry is a valuable
method in the eld of proteomics and metaproteomics. For
protein identi cation, di erent protein search engines are
used such as X! Tandem, MASCOT, OMSSA, SEQUEST
etc. These search engines receive input data in form of
les. With the rapid rise of proteomics and metaproteomics,
new measurement devices are introduced resulting in
increase of research capabilities, consequently producing enormous
chunks of data regularly. Admittedly, le-based search
engines for protein identi cation are at their limits and IT
methods should be introduced for protein identi cation to
manage huge amount of data e ciently in future. In this paper,
we focus on feasibility of Database Management Systems as
an alternative to conventional le-based approaches. We
implement a connector interface and integrate it into the latest
X! Tandem version (2017.02.01) , in order to couple it with a
DBMS keeping its business logic intact and study its
performance. We compared our work with the core X! Tandem and
MetaProteomeAnalyzer tool (which performs protein search
and uses a relational database for data storage). We
observed there was no information loss in our approach and we
were able to successfully implement the DBMS connector
interface to X! Tandem.</p>
    </sec>
    <sec id="sec-2">
      <title>Categories and Subject Descriptors</title>
    </sec>
    <sec id="sec-3">
      <title>1. INTRODUCTION</title>
      <p>
        Proteomics is the comprehensive study of expressed
proteins from one organism for a certain time point; in contrast
metaproteomics is the investigation of samples containing
proteins from di erent organisms [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1, 2, 3</xref>
        ]. Proteomics and
metaproteomics use mass spectrometry (MS) as an
analytical technique to characterize proteins and detect their
accurate masses, which relies upon a protein identi cation
algorithm for cataloging of proteins present in a sample [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. The
protein identi cation process is based on the study of
peptides generated by proteolytic digestion [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ]. Algorithms
such as X! Tandem [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], MASCOT [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], SEQUEST [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ],
OMSSA [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] identi es peptides from MS spectra by searching
them against a database of known peptides [
        <xref ref-type="bibr" rid="ref11 ref12 ref13">11, 12, 13</xref>
        ].
Balgley et al. [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] found OMSSA and X! Tandem to perform
better than SEQUEST and MASCOT with respect to the
number of peptide identi cations per protein and Quandt
et al. [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] in their analysis declared X! Tandem to be more
robust than OMSSA and MASCOT when there were
changes in the precursor mass error and fragment mass error.
Also being an open source software with periodical updates,
X! Tandem appears to be a popular choice among biologists.
      </p>
      <p>
        X! Tandem reads the input data (MS spectra and a
protein sequence database) as les and writes the output into
a le as well, so any analytical study would require parsing
them. The algorithm deals with huge protein libraries
(containing over million peptide sequences) and spectra data,
which makes it laborious to manipulate and visualize the
data as well as the results [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. Moreover, redundant data
tracking and version control is di cult with les. These
issues have already been resolved by DBMS. Therefore our
project aims to replace the conventional le-based approach
with a DBMS. We have implemented a general adapter
inside X! Tandem, which can be connected to any DBMS, by
keeping its business logic intact and only changing the I/O
logic. In this paper we have realized an RDBMS (MySQL)
adapter. An RDBMS facilitated us to well represent the
underlying relation of input and output data [
        <xref ref-type="bibr" rid="ref17 ref18">17, 18</xref>
        ].
      </p>
      <p>
        This paper compares X! Tandem successfully integrated
with an RDBMS, the core X! Tandem algorithm and
MetaProteomeAnalyzer [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ].
      </p>
      <p>Further we discuss basic concepts in the section
Fundamentals, proposed solution in the section Our Approach,
followed by Implementation, Evaluation, and Conclusion.</p>
    </sec>
    <sec id="sec-4">
      <title>RELATED WORK</title>
      <p>
        Zeeberg et al. in their work on GoMiner [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] and Ahmad
et al. in their work on nucleolar proteome database [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]
have used RDBMS as an e cient storage engine. Yu et al.
have realized an RDBMS as a tool for safe warehousing and
analysis of quantitative proteomic data [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. Bjornson et al.
have worked towards parallelization of X! Tandem [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]
whereas He et al. implemented a parallel X! Tandem with Many
Integrated Core (MIC) [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ]. Field et al. [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ] while working
on proteome mass spectral analysis have used RDBMS for
storing processed data and customized reporting.
MetaProteomeAnalyzer developed by Muth et al. [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], comes closest
to our work as they perform protein search using X! Tandem
and use RDBMS for storing search results.
3.
      </p>
    </sec>
    <sec id="sec-5">
      <title>FUNDAMENTALS</title>
      <p>In this section, we explain the basics of a protein search
engine with the focus on X! Tandem and brie y about the
MPA tool.
3.1</p>
    </sec>
    <sec id="sec-6">
      <title>Protein Identification Algorithm</title>
      <p>A protein identi cation algorithm attempts to assign mass
spectra to proteins/peptides. Inputs to the algorithm are:
Protein sequence database (usually found by genetics)
Experimental spectra (tandem mass spectrometry
data usually in MGF1)</p>
      <p>Con guration parameters</p>
      <p>In Figure 1, we show how the experimental spectra relate
to the protein sequences in the database.
3.2</p>
    </sec>
    <sec id="sec-7">
      <title>Experimental Spectra</title>
      <p>
        Experimental spectra are the result of tandem MS/MS
(multiple steps of mass spectrometry, with some form of
molecular fragmentation occurring between the stages). These
spectra are commonly stored in a MASCOT Generic Format
(MGF) le [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ] that encodes a collection of spectra. X!
Tandem is built to use DTA, PKL or MGF les. We use MGF
for our evaluation.
3.3
      </p>
    </sec>
    <sec id="sec-8">
      <title>Protein Sequence Database</title>
      <p>
        Protein sequence database (stored in a le) is a library of
known protein sequences that are represented in a standard
format [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ]. In our work, we used protein sequences stored
1MASCOT Generic Format
in a FASTA format le. For every protein sequence in the
FASTA le, the rst line is the de nition line containing an
access identi er along with some optional description. The
lines following the de nition line represent sequence data.
The protein search algorithm uses these peptide sequences
to create theoretical spectra and matches them with the
experimental spectra.
3.4
      </p>
    </sec>
    <sec id="sec-9">
      <title>X! Tandem Output</title>
      <p>
        The output le is in the BIOML (Biopolymer Markup
Language) [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ] format, which features complex annotations
of proteins in a hierarchical manner and can be processed
using standard XML parsers.
3.5
      </p>
    </sec>
    <sec id="sec-10">
      <title>MetaProteomeAnalyzer Tool</title>
      <p>
        The MetaProteomeAnalyzer (MPA) tool [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] employs X!
Tandem internally with an advanced user interface view. It
extracts the MGF and FASTA information from a MySQL DB
and converts them into .mgf and .fasta les. Once the
protein search is initiated, using these les X! Tandem identi es
the proteins and generates the output. The MPA tool then
parses the output le and stores it in DB. Hence, it uses
both le and DB information for completing the process.
4.
      </p>
    </sec>
    <sec id="sec-11">
      <title>OUR APPROACH</title>
      <p>
        With growing size of data it is di cult for biologists to
manage hundreds of thousands of les where each le is in
gigabytes. Furthermore DBMS have been considered an
appropriate and bene cial data storage strategy as they form a
classic framework for representing and analyzing huge
metaproteomics data [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. We have seen in subsection 3.5 that the
MPA tool stores data in DB but does not read from it
directly, during protein identi cation. Their process of converting
data between DB and le representation is ine cient as it
introduces an overhead of parsing. Rather than using les
if we manage to directly read input from and write output
to a DB, it would remove the parsing step, thus reducing
load on the entire process of protein identi cation. Our goal
was to design and develop a new architecture for X!
Tandem connecting it to a DBMS without altering the protein
identi cation algorithm inside. To store the MGF, FASTA
and output les we designed a database schema preserving
their hierarchical structure (see Figure 2, 3 and 4). We
developed a special adapter interface which could communicate
with any database without in uencing the functionality of
X! Tandem. We used the con guration le input.xml to
dene the database credentials, MGF and FASTA data source
identi er, and parameters.xml to de ne the calculation
criteria to match the protein sequences. Other con guration
information was kept as a le.
      </p>
    </sec>
    <sec id="sec-12">
      <title>5. IMPLEMENTATION</title>
      <p>Our work is implemented in C++ as we have modi ed
X! Tandem classes to read and write data, from and to,
MySQL instead of les. We have developed a MySQL
adapter interface, which can be modi ed to connect X! Tandem
to any other DB without changing its business logic. Further
we study the database design for MGF, FASTA and output
les.</p>
    </sec>
    <sec id="sec-13">
      <title>DATABASE DESIGN</title>
      <p>In this section, we discuss the structure of tables for
spectra, FASTA and output data in detail.
6.1</p>
      <p>MS spectra information is stored into tables: ms dataset
and fragment ion list. While ms dataset stores peptide mass,
charge, precursor intensity, retention time (RT) and
spectrum title, the peak-list of mass and intensity pairs for each
spectrum is stored in fragment ion list table. Records in
fragment ion list table are mapped to a speci c spectrum
in ms dataset using a foreign key constraint `Map ID' (see
Figure 2). Although a join operation on these two tables for
reading spectra information would introduce a performance
penalty, we do get the exibility of studying selective
spectra as and when required instead of reading the entire le.</p>
      <p>Understanding its structure (see subsection 3.3), we split
each protein sequence into access identi er, description and
sequence data and store them in protein reference data(see
Figure 3). The protein reference data info table stores the
information about the FASTA library loaded into DB.</p>
      <p>The X! Tandem output data objects are stored in the
tables out group (original mass spectrum), out protein
(protein containing matching peptides), out domain (peptide
sequences that match to a spectrum), out gaml trace histograms
(histograms about statistics of an identi cation),
out gaml attributes (histogram attributes), out gaml xy data
(histogram values) and out parameters info (input
parameters and performance statistics). The output tables conform
to the output standards2 of core X! Tandem. The complete
structure of output tables can be observed in Figure 4.</p>
    </sec>
    <sec id="sec-14">
      <title>FACTORY ADAPTER INTERFACE</title>
      <p>Factory adapter interface is developed to establish a
database connection with X! Tandem. Its implementation only
modi es the I/O logic of X! Tandem. The database
entities are not coupled with C++ objects of X! Tandem, which
means X! Tandem functions without any knowledge of the
DB schema. This provides a generic interface where any
database can be connected to X! Tandem with changes in
input and output schema (pertaining to the DB used) without
even worrying about the access and manipulation of data.
In our case, we developed a factory adapter interface for
MySQL.
8.</p>
    </sec>
    <sec id="sec-15">
      <title>EVALUATION</title>
      <p>We evaluated our work to study the feasibility of
integrating X! Tandem with a DBMS with an aim to perform as
good as the core X! Tandem. The evaluation was performed
on the following hardware:</p>
      <p>RAM : 8GB
Processor : i5 6th Generation Intel core 2.3 GHz
Operating System : Windows 10</p>
      <p>We conducted experiments with varying sizes of
spectra and FASTA data. FASTA datasets used for
evaluation100K FASTA.fasta and 552K FASTA.fasta, which
contained 100,000 and 552,884 protein sequences respectively were
2http://www.thegpm.org/docs/X series output form.pdf
taken from `UniProt Knowledgebase'. Spectra datasets used
were 100 le.mgf, 2k le.mgf and 20K le.mgf which were
100, 2000 and 20000 in spectra counts respectively.</p>
      <p>The evaluation was done by assessing the outcomes of all
experiments on three performance measures namely
computation time, CPU usage, and RAM usage for original
lebased X! Tandem, the MPA Tool and our approach- X!
Tandem using DBMS (MySQL).</p>
      <p>For each performance measure, comparing the
aforementioned systems, the results were presented in two graphs,
one for 100K FASTA and another for 552K FASTA against
all the three datasets of spectra. Consequently we veri ed
them and concluded that there was no information loss from
our approach.
8.1</p>
    </sec>
    <sec id="sec-16">
      <title>Computation time</title>
      <p>For small-sized input data (100 spectra with 100K, 552K
FASTA and 2000 spectra with 552K FASTA) our work (8.48,
24.67 and 32.34 seconds) outperforms the core X! Tandem
(9.06, 46.56 and 73.25 seconds). For 2000 spectra with 100K
FASTA our approach (32.34 seconds) was slightly slower
than the core X! Tandem (23.67 seconds). However instead
for input spectra of size 20K with 100K and 552K FASTA,
our approach (606.06 and 1168.33 seconds) was considerably
slower than core X! Tandem (185.34 and 449.94) as it takes
almost 3 times more time to execute. To deal with this issue,
batch processing of data should be included in our approach.
In comparison to the MPA tool, our approach performs
signi cantly better in all cases (see Figure 5 and 6.</p>
    </sec>
    <sec id="sec-17">
      <title>CPU Usage</title>
      <p>We studied CPU usage of the three systems when no other
process was running on the machine. We noticed that CPU
usage is remarkably less for our approach (varying from 8.88
to 17.95%) irrespective of the size of data whereas in case of
core X! Tandem and the MPA tool, CPU usage varies from
71.69% to 100% and 85-100% respectively (see Figure 7 and
8). Higher CPU usages could lead to performance issues in
the system.
8.3</p>
    </sec>
    <sec id="sec-18">
      <title>RAM Usage</title>
      <p>We can observe from Figure 9 and 10 that RAM
usage is comparatively same in all the systems for small-sized
input data (100 spectra with 100K &amp; 552K FASTA) with
core X! Tandem, MPA and our work having 66.69 &amp; 190.61,
56.96 &amp; 177.54, 54.48 &amp; 248.04 bytes consumption
respectively. However, our approach consumes signi cantly more
amount of RAM (2429.94 &amp; 2974.06 bytes) for large input
data (20K spectra with 100K/552K FASTA) against that of
core X! Tandem (237.94 &amp; 392.83 bytes) and the MPA tool
(47.93 &amp; 177.34 bytes). RAM consumption increases linearly
with data size, in our case. The MPA tool recorded lowest
RAM consumption in all the cases.</p>
      <p>The evaluation results show that core X! Tandem is the
fastest as it is highly optimized. Our approach was noted to
be faster than core X! Tandem while dealing with small-sized
data whereas for larger data it was almost 3 times slower,
further drawing our attention to a necessary
implementation of batch processing. Our approach was quicker than the
MPA tool in all the cases. However our approach exhibited
e cient CPU usages across all the experiments, outshining
the other two systems by a wide margin. In terms of RAM
usage, our approach needs improvement as it consumed a
lot more memory than the other two systems when data
size increased.</p>
    </sec>
    <sec id="sec-19">
      <title>CONCLUSION</title>
      <p>We have not only engineered a connector interface
between X! Tandem and a DBMS but also systematically
investigated the feasibility of moving from le-based protein
search algorithm to DBMS based algorithm without any
information loss. We observed that DBMS o ers accessibility
to data in a structured manner that was much needed for
biologists. A biologist may create SQL queries on results to
create customized reports without going through the hassle
of parsing the les. Also in le-based approach, FASTA data
was separated with respect to taxon, in di erent les.
However with a connection to DBMS, all the FASTA data could
be stored in one database and could be selectively used for
experiments.</p>
      <p>During evaluation we observed core X! Tandem to be the
fastest of the three systems as it is highly optimized. Our
work was faster than core X! Tandem for small datasets but
needed batch processing for handling large datasets e
ciently. We were signi cantly faster than MPA in all the cases.
There was no overhead noticed on database access in our
approach for small-sized input spectra, but a drastic
overhead was noticed for large input spectra. This implies our
approach needs multi-threading for cost-e ective RAM
usage. Our approach exhibited e cient CPU usages across all
the experiments, outshining the other two systems by a wide
margin.</p>
      <p>We have successfully developed an adapter to connect X!
Tandem to any database (Section 7), opening up many
possibilities for future improvements. For instance, an
implementation of NoSQL database using our approach would provide an
easy scale-out architecture with e cient performance
whereas le-based X! Tandem could not scale. Also our work
provides a basis for realizing protein identi cation algorithms
in cloud environments while utilizing features of BigData.
10.</p>
    </sec>
    <sec id="sec-20">
      <title>FUTURE WORK</title>
      <p>Our connector interface for MySQL could be exchanged
(Section 7) for cloud-based endpoints such as Cassandra.
Such cloud-based endpoints provide elastic scalability, high
availability and fault tolerance with high performance. That
way protein identi cation could be developed as a service,
which would bring an e ective way of collaboration amongst
biologists because of its central storage. Multi-threading
approach should be adopted to tackle high RAM usage in our
work.
11.</p>
    </sec>
    <sec id="sec-21">
      <title>ACKNOWLEDGEMENT</title>
      <p>The authors sincerely thank Xiao Chen, Sebastian Krieter,
Andreas Meister and Marcus Pinnecke for their support and
advice. This work is partly funded by the de.NBI Network
(031L0103), the DFG (grant no.: SA 465/50-1), the
European Regional Development Fund (grant no. 11.000sz00.00.0
17 114347 0, the German Federal Ministry of Food and
Agriculture (grants nos. 22404015) and dedicated to the memory
of Mikhail Zoun.
12.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>R.</given-names>
            <surname>Pieper</surname>
          </string-name>
          , S.-T. Huang, and
          <string-name>
            <surname>M.-J. Suh</surname>
          </string-name>
          , \
          <article-title>Proteomics and metaproteomics,"</article-title>
          <source>in Encyclopedia of Metagenomics</source>
          . Springer New York,
          <year>2013</year>
          , pp.
          <volume>1</volume>
          {
          <fpage>11</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>R.</given-names>
            <surname>Heyer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Kohrs</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Reichl</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Benndorf</surname>
          </string-name>
          , \
          <article-title>Metaproteomics of complex microbial communities in biogas plants,"</article-title>
          <source>Microbial Technology</source>
          , vol.
          <volume>8</volume>
          ,
          <issue>04</issue>
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>R.</given-names>
            <surname>Heyer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Schallert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Zoun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Becher</surname>
          </string-name>
          , G. Saake, and
          <string-name>
            <given-names>D.</given-names>
            <surname>Benndorf</surname>
          </string-name>
          , \
          <article-title>Challenges and perspectives of metaproteomic data analysis,"</article-title>
          <source>Journal of Biotechnology</source>
          , vol.
          <volume>261</volume>
          , no. Supplement C, pp.
          <volume>24</volume>
          {
          <issue>36</issue>
          ,
          <year>2017</year>
          , bioinformatics
          <article-title>Solutions for Big Data Analysis in Life Sciences presented by the German Network for Bioinformatics Infrastructure</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>R.</given-names>
            <surname>Aebersold</surname>
          </string-name>
          and
          <string-name>
            <given-names>M.</given-names>
            <surname>Mann</surname>
          </string-name>
          , \
          <article-title>Mass spectrometry-based proteomics,"</article-title>
          <source>Nature</source>
          , vol.
          <volume>422</volume>
          , no.
          <issue>6928</issue>
          , p.
          <fpage>198</fpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M. W.</given-names>
            <surname>Duncan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Aebersold</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R. M.</given-names>
            <surname>Caprioli</surname>
          </string-name>
          , \
          <article-title>The pros and cons of peptide-centric proteomics,"</article-title>
          <source>Nature Biotechnology</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>J.</given-names>
            <surname>Eriksson</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Fenyo</surname>
          </string-name>
          , \
          <article-title>Modeling mass spectrometry-based protein analysis," Bioinformatics for Comparative Proteomics</article-title>
          , pp.
          <volume>109</volume>
          {
          <issue>117</issue>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>R.</given-names>
            <surname>Craig</surname>
          </string-name>
          and
          <string-name>
            <given-names>R. C.</given-names>
            <surname>Beavis</surname>
          </string-name>
          , \
          <article-title>Tandem: matching proteins with tandem mass spectra,"</article-title>
          <source>Bioinformatics</source>
          , vol.
          <volume>20</volume>
          , no.
          <issue>9</issue>
          , pp.
          <volume>1466</volume>
          {
          <issue>1467</issue>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>J. S.</given-names>
            <surname>Cottrell</surname>
          </string-name>
          and U. London, \
          <article-title>Probability-based protein identi cation by searching sequence databases using mass spectrometry data," electrophoresis</article-title>
          , vol.
          <volume>20</volume>
          , no.
          <issue>18</issue>
          , pp.
          <volume>3551</volume>
          {
          <issue>3567</issue>
          ,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>J. K.</given-names>
            <surname>Eng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. L.</given-names>
            <surname>McCormack</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and J. R.</given-names>
            <surname>Yates</surname>
          </string-name>
          , \
          <article-title>An approach to correlate tandem mass spectral data of peptides with amino acid sequences in a protein database,"</article-title>
          <source>Journal of the American Society for Mass Spectrometry</source>
          , vol.
          <volume>5</volume>
          , no.
          <issue>11</issue>
          , pp.
          <volume>976</volume>
          {
          <issue>989</issue>
          ,
          <year>1994</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>L. Y.</given-names>
            <surname>Geer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. P.</given-names>
            <surname>Markey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Kowalak</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Wagner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Xu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. M.</given-names>
            <surname>Maynard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Shi</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S. H.</given-names>
            <surname>Bryant</surname>
          </string-name>
          , \
          <article-title>Open mass spectrometry search algorithm,"</article-title>
          <source>Journal of proteome research</source>
          , vol.
          <volume>3</volume>
          , no.
          <issue>5</issue>
          , pp.
          <volume>958</volume>
          {
          <issue>964</issue>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>L. J.</given-names>
            <surname>Everett</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bierl</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S. R.</given-names>
            <surname>Master</surname>
          </string-name>
          , \
          <article-title>Unbiased statistical analysis for multi-stage proteomic search strategies,"</article-title>
          <source>Journal of proteome research</source>
          , vol.
          <volume>9</volume>
          , no.
          <issue>2</issue>
          , pp.
          <volume>700</volume>
          {
          <issue>707</issue>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>M. V.</given-names>
            <surname>Ivanov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. I.</given-names>
            <surname>Levitsky</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M. V.</given-names>
            <surname>Gorshkov</surname>
          </string-name>
          , \
          <article-title>Adaptation of decoy fusion strategy for existing multi-stage search work ows,"</article-title>
          <source>Journal of The American Society for Mass Spectrometry</source>
          , vol.
          <volume>27</volume>
          , no.
          <issue>9</issue>
          , pp.
          <volume>1579</volume>
          {
          <issue>1582</issue>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>R. D.</given-names>
            <surname>Bjornson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. J.</given-names>
            <surname>Carriero</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Colangelo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Shifman</surname>
          </string-name>
          ,
          <string-name>
            <surname>K.-H. Cheung</surname>
            ,
            <given-names>P. L.</given-names>
          </string-name>
          <string-name>
            <surname>Miller</surname>
            , and
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Williams</surname>
          </string-name>
          , \X!
          <article-title>! tandem, an improved method for running x! tandem in parallel on collections of commodity computers,"</article-title>
          <source>The Journal of Proteome Research</source>
          , vol.
          <volume>7</volume>
          , no.
          <issue>1</issue>
          , pp.
          <volume>293</volume>
          {
          <issue>299</issue>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>B. M.</given-names>
            <surname>Balgley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Laudeman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Song</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C. S.</given-names>
            <surname>Lee</surname>
          </string-name>
          , \
          <article-title>Comparative evaluation of tandem ms search algorithms using a target-decoy search strategy,"</article-title>
          <source>Molecular &amp; Cellular Proteomics</source>
          , vol.
          <volume>6</volume>
          , no.
          <issue>9</issue>
          , pp.
          <volume>1599</volume>
          {
          <issue>1608</issue>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>A.</given-names>
            <surname>Quandt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Espona</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Balasko</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Weisser</surname>
          </string-name>
          , M.-
          <string-name>
            <given-names>Y.</given-names>
            <surname>Brusniak</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Kunszt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Aebersold</surname>
          </string-name>
          , and L. MalmstrA~ u}m, \
          <article-title>Using synthetic peptides to benchmark peptide identi cation software and search parameters for ms/ms data analysis," EuPA Open Proteomics</article-title>
          , vol.
          <volume>5</volume>
          , pp.
          <volume>21</volume>
          {
          <issue>31</issue>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>R.</given-names>
            <surname>Zoun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Schallert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Broneske</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Heyer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Benndorf</surname>
          </string-name>
          , and G. Saake, \
          <article-title>Interactive chord visualization for metaproteomics,"</article-title>
          <source>in 2017 28th International Workshop on Database and Expert Systems Applications (DEXA)</source>
          ,
          <year>Aug 2017</year>
          , pp.
          <volume>79</volume>
          {
          <fpage>83</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>C.</surname>
          </string-name>
          <article-title>Turker and</article-title>
          G. Saake, \
          <article-title>Objektrelationale datenbanken: Ein lehrbuch. 1," Au age</article-title>
          . Heidelberg: dpunkt.
          <source>verlag GmbH</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>G.</given-names>
            <surname>Saake</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Sattler</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Heuer</surname>
          </string-name>
          , \
          <article-title>Datenbanken-konzepte und sprachen</article-title>
          ,
          <source>mitp professional</source>
          ,
          <year>2013</year>
          ."
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>T.</given-names>
            <surname>Muth</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Behne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Heyer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Kohrs</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Benndorf</surname>
          </string-name>
          , M. Ho mann, M. LehtevA~ d', U. Reichl,
          <string-name>
            <given-names>L.</given-names>
            <surname>Martens</surname>
          </string-name>
          , and E. Rapp, \
          <article-title>The MetaProteomeAnalyzer: A powerful open-source software suite for metaproteomics data analysis and interpretation,"</article-title>
          <source>Journal of Proteome Research</source>
          , vol.
          <volume>14</volume>
          , no.
          <issue>3</issue>
          , pp.
          <volume>1557</volume>
          {
          <issue>1565</issue>
          , feb
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>B. R.</given-names>
            <surname>Zeeberg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Feng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. D.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. T.</given-names>
            <surname>Fojo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Sunshine</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Narasimhan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. W.</given-names>
            <surname>Kane</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. C.</given-names>
            <surname>Reinhold</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lababidi</surname>
          </string-name>
          et al.,
          <article-title>\Gominer: a resource for biological interpretation of genomic and proteomic data," Genome biology</article-title>
          , vol.
          <volume>4</volume>
          , no.
          <issue>4</issue>
          , p.
          <fpage>R28</fpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Ahmad</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.-M.</given-names>
            <surname>Boisvert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Gregor</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Cobley</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <surname>A. I. Lamond</surname>
          </string-name>
          , \Nopdb:
          <article-title>Nucleolar proteome database,"</article-title>
          <source>Nucleic Acids Research</source>
          , vol.
          <volume>37</volume>
          , no.
          <issue>1</issue>
          , pp.
          <source>D181{D184</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>K.</given-names>
            <surname>Yu</surname>
          </string-name>
          and
          <string-name>
            <given-names>A. R.</given-names>
            <surname>Salomon</surname>
          </string-name>
          , \
          <article-title>Peptidedepot: exible relational database for visual analysis of quantitative proteomic data and integration of existing protein information,"</article-title>
          <source>Proteomics</source>
          , vol.
          <volume>9</volume>
          , no.
          <issue>23</issue>
          , pp.
          <volume>5350</volume>
          {
          <issue>5358</issue>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>R. D.</given-names>
            <surname>Bjornson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. J.</given-names>
            <surname>Carriero</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Colangelo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Shifman</surname>
          </string-name>
          ,
          <string-name>
            <surname>K.-H. Cheung</surname>
            ,
            <given-names>P. L.</given-names>
          </string-name>
          <string-name>
            <surname>Miller</surname>
            , and
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Williams</surname>
          </string-name>
          , \X!
          <article-title>! tandem, an improved method for running x! tandem in parallel on collections of commodity computers,"</article-title>
          <source>The Journal of Proteome Research</source>
          , vol.
          <volume>7</volume>
          , no.
          <issue>1</issue>
          , pp.
          <volume>293</volume>
          {
          <issue>299</issue>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>P.</given-names>
            <surname>He</surname>
          </string-name>
          and
          <string-name>
            <given-names>K.</given-names>
            <surname>Li</surname>
          </string-name>
          , \
          <article-title>Mic-tandem: parallel x! tandem using mic on tandem mass spectrometry based proteomics data,"</article-title>
          <source>in 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing</source>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>H. I.</given-names>
            <surname>Field</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Fenyo</surname>
          </string-name>
          , and R. C. Beavis, \
          <article-title>Radars, a bioinformatics solution that automates proteome mass spectral analysis, optimises protein identi cation, and archives data in a relational database,"</article-title>
          <source>Proteomics</source>
          , vol.
          <volume>2</volume>
          , no.
          <issue>1</issue>
          , p.
          <fpage>36</fpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26] \
          <article-title>Mascot generic format documentation." [Online]</article-title>
          . Available: http://www.matrixscience.com/help/data le help.html
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>N. C. for Biotechnology Information.</surname>
          </string-name>
          (
          <year>2002</year>
          , Nov.)
          <article-title>Fasta format</article-title>
          . [Online]. Available: https://blast.ncbi.nlm.nih.gov/Blast.cgi? CMD=
          <article-title>Web&amp;PAGE TYPE=BlastDocs&amp; DOC TYPE=BlastHelp</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>D.</given-names>
            <surname>Fenya</surname>
          </string-name>
          ^LZa^LC, \
          <article-title>The biopolymer markup language."</article-title>
          <source>Bioinformatics</source>
          (Oxford, England), vol.
          <volume>15</volume>
          , no.
          <issue>4</issue>
          , pp.
          <volume>339</volume>
          {
          <issue>340</issue>
          ,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>