<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Estimation of Proportions of SARS-CoV-2 Variants in a Mixed Sequencing Sample</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Askar Gafurov</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Andrej Baláž</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Tomáš Vinarˇ</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Bronˇa Brejová</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Applied Informatics, Faculty of Mathematics</institution>
          ,
          <addr-line>Physics and Informatics</addr-line>
          ,
          <institution>Comenius University in Bratislava</institution>
          ,
          <country country="SK">Slovakia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Computer Science, Faculty of Mathematics</institution>
          ,
          <addr-line>Physics and Informatics</addr-line>
          ,
          <institution>Comenius University in Bratislava</institution>
          ,
          <country country="SK">Slovakia</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Estimation of proportions of SARS-CoV-2 genome variants (e.g. variant B.1.1.7 originating from Britain, variant B.1.351 originating from South-Africa) in a population is currently done by sequencing individual samples, which demands individual laboratory processing of each sample. This labor can be significantly reduced by mixing several samples together and processing them in one batch. Our project aims to estimate the proportion of samples with given variants from such mixtures using probabilistic modeling.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1 Introduction</title>
      <p>
        Since December 2019, the severe acute respiratory
syndrome coronavirus 2 (SARS-CoV-2) is rapidly spreading
across the world, causing the coronavirus infectious
disease (COVID-19) pandemic. Up to April 5, 2021, there
were 132 mil. cases of COVID-19 infection. Among these,
109 mil. cases have already had an outcome from which
there was 2:8 mil. deaths, resulting in an estimate of the
mortality rate around 2:5% [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ].
      </p>
      <p>
        Due to its worldwide spread and a mutation rate of about
2 mutations per month [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], the SARS-CoV-2 developed
multiple variants. Some emerging variants have
accumulated significantly more mutations and proved to be more
dangerous, causing concerns around the globe [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
Scientists estimate that continued transmission of SARS-CoV-2
and selective pressures, such as vaccines, are creating ideal
conditions for additional significant virus evolution [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
Therefore, it is critical to characterize the virus strains
further and monitor the spread of the variants in the
population in order to inform public policies and assess the
effectiveness of containment strategies.
      </p>
      <p>Monitoring of the SARS-CoV-2 variants requires
conducting many sequencing experiments to determine the
genome of the virus and to identify its mutations compared
to a reference genome. Based on this, it is then
possible to decide which variant was sequenced. Some part of
the laboratory work is done individually for each sample,
which is time-consuming and resource-intensive. In this
work, we propose a model capable of estimating the
proportions of different variants in a pooled sample, which
allows to combine multiple samples into one sequencing
experiment.
2
2.1</p>
    </sec>
    <sec id="sec-2">
      <title>Problem description</title>
      <sec id="sec-2-1">
        <title>Variants</title>
        <p>
          In our experiments, we consider several variants of the
SARS-CoV-2 virus, which are characterized by specific
mutations described in the Table 1. For instance, if a
particular SARS-CoV-2 genome has the nucleotide at
position 3267 mutated from cytosine to thymine, it has one
of the mutations characteristic for the variant which
originated in the United Kingdom [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] and is known as a
variant of concern alpha [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. In the Pangolin SARS-CoV-2
lineage classification [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ], it is denoted B.1.1.7. In this
paper, we will denote this variant by acronym UK listed
in the table. In our list of variants, we have included two
additional variant of concern (beta and gamma) as well as
several variants that had high prevalence in Slovakia in the
fall of 2020 and early 2021, which is the time from which
our data originate. For each variant, the table also
contains the minimum number of these characteristic
mutations which need to be observed in the genome in order to
be considered as belonging to a given variant in our study.
If a genome did not reach the number of characteristic
mutations for any considered variant, it was assigned the label
“other”.
        </p>
        <p>
          This characterization was used on all samples from the
GISAID database (downloaded on Mar 4, 2021) [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ], which
provides a comprehensive resource of more than 650
thousands fully assembled SARS-CoV-2 genomes.
Individual genomes were aligned to the SARS-CoV-2 reference
Wuhan/Hu-1/2019 by minimap2 [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. Then, each
sequence was assigned a label as described earlier.
Finally, a matrix P was constructed, which consisted of
entries pk;i;a representing the relative frequency of
nucleotide a at genomic position i of variant k among aligned
genomes.
        </p>
        <p>This matrix thus characterizes each variant not only
by the selected mutations listed in Table 1, but also
captures any additional significant mutations present in the
genomic sequences classified to the variant. This approach
can be easily applied to a different set of variants by simply
modifying the input table of variants and their
characteristic mutations.
2.2</p>
      </sec>
      <sec id="sec-2-2">
        <title>Sequencing data</title>
        <p>
          In order to identify the SARS-CoV-2 variant of a new
patient, it is necessary to sequence an obtained
sample. In this work, the used samples were sequenced
using the ARTIC amplicon protocol [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ] and the Oxford
Nanopore MinION sequencer.
        </p>
        <p>
          In the ARTIC protocol, the RNA of the virus is first
reverse transcribed into DNA and specific regions of
the DNA are amplified. These amplicons cover almost
the entire length of the genome with slight overlaps
between adjacent regions. In our data, the amplicons had
length about 1700 bp. Amplified DNA was sequenced
by the MinION sequencer, and the resulting sequences are
called reads. The reads are then mapped to the reference
genome by tool minimap2 [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ] and stored in the BAM
format.
        </p>
        <p>Because each read comes from a genome of a specific
variant, it is expected that it contains mutations that are
characteristic for this variant. When analyzing a mixture
of reads from different patient samples, the task at hand is
to estimate the proportions of reads coming from
individual variants, based on the presence of mutations
characteristic for these variants.
3</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Related work</title>
      <p>There had been several attempts to identify SARS-CoV-2
variants from mixed samples. Most commonly, these were
wastewater-based epidemiological studies.</p>
      <p>
        Crits-Christoph et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] showed that single nucleotide
variants (SNVs)1 detected from the sewage water from
San Francisco Bay Area were significantly similar to local
California-based patient-derived genotypes, thus
demonstrating the possibility of identifying local SARS-CoV-2
variants in the wastewater samples. The SNVs were
obtained via SNV caller inStrain v1.3.2 [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] and the
similarity of genotypes was established via Fisher’s exact test.
      </p>
      <p>
        Another study [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] analyzed 91 wastewater samples from
11 states in the USA and identified 7973 SNVs, of which
5680 were “novel” at the time of the analysis with respect
to the global clinically derived data. Interestingly, almost
half of the “novel” variants were confirmed within the next
1Single nucleotide variants are simple mutations substituting one
DNA base for another, not to be confused with virus variants, which are
groups of evolutionarily related viruses.
5 months after the analysis. This suggests that sewage
samples may provide a more comprehensive snapshot of
currently circulating SARS-CoV-2 variants in comparison
with solely clinical cases.
      </p>
      <p>
        A wastewater study conducted in Spain [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] identified
238 SNVs and 6 deletions in comparison with the
reference genome of SARS-CoV-2 isolate Wuhan-Hu-1. The
study used 40 samples, which were sequenced with
ARTIC protocol v.3, analysed using the iVAR software.
      </p>
      <p>
        In the wastewater study from Switzerland [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], the
authors showed that it is possible to use Illumina reads to
detect the variants before they appear in clinical cases. The
48 samples were collected as 24-hours composite or grab
samples. Amplicons were created using the ARTIC v.3
protocol and sequenced using Illumina NovaSeq 6000,
resulting in paired reads of length 250bp. The mutations
were identified from these reads using the V-pipe
bioinformatics pipeline. These mutations were then compared
with the clinical cases from Switzerland in the GISAID
database for the presence of characteristic mutations of
the B.1.1.7 and B.1.351 variants using Fisher’s exact test.
The authors also looked at mutations co-occurring in the
same read and found further evidence of the presence of
the B.1.1.7 variant in Switzerland already in early
December.
      </p>
      <p>
        Mixed samples can also originate from environmental
sources other than sewage water. The first report on
recovering near-complete SARS-CoV-2 genome sequences
from environmental surface swabs assessed the
contamination with the virus in a hospital [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. The authors of this
study confirmed the low likelihood that SARS-CoV-2
contamination on hospital surfaces contains infectious virus.
      </p>
      <p>Although many studies have identified SARS-CoV-2
variants from mixed environmental samples, all have
qualitative results. In this paper, we estimate the proportions
of the variants in the sample, which we believe is
correlated with the proportions of the variants circulating in the
community.</p>
    </sec>
    <sec id="sec-4">
      <title>Model description</title>
      <p>In our model, we assume that in a set of aligned reads
coming from one variant, the expected observed symbol
counts at any given position in genome are proportional
to the symbol frequencies among all sequences in a given
variant, which are captured in matrix P described in
subsection 2.1. The observed symbol counts at a particular
position can be thus described by a multinomial
distribution. To simplify the model, we further assume that
individual positions in the genome are independent. When
sequencing a mixed sample, we assume that individual
variants are present at some unknown proportions which are
represented as weights in a mixture model. We assume
that these proportions stay the same at all positions in the
genome. A more detailed description of the model is given
in the next subsection.
4.1</p>
      <sec id="sec-4-1">
        <title>Basic model</title>
        <p>Let S = fA;C; G; T g be the symbol alphabet. Let K
denote the number of virus variants and L denote the length
of the reference genome. Let W = (w1; : : : ; wK ) denote
the (unknown) weights of individual variants in a mixture.
Let pk;i;a denote the probability of observing symbol a at
position i in the k-th variant and Oi;a denote the count of
symbol a at position i in the observed reads.</p>
        <p>The probability of observing symbol a at position i
in a mixture with weights W is then equal to mi;a(W ) :=</p>
        <p>K
åk=1 wk pk;i;a.</p>
        <p>The total probability of observations O given mixture
weights W is then proportional to:</p>
        <p>Pr[OjW ]</p>
        <p>L
Õ Õ mi;a(W )Oi;a :
i=1 a2S</p>
        <p>The inference of the mixture weights from observations
can be then solved via maximisation of the log-likelihood
function:</p>
        <p>L
W := arg max å å Oi;a log mi;a(W ):</p>
        <p>W i=1 a2S
This task is equivalent to minimisation of cross-entropy
between m and O:</p>
        <p>W := arg min</p>
        <p>W</p>
        <p>L
å å Oi;a log mi;a(W )
i=1 a2S
4.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>Minimisation and efficiency</title>
        <p>
          The minimisation is done using the L-BFGS-B algorithm
[
          <xref ref-type="bibr" rid="ref19">19</xref>
          ] implemented in Python library scipy [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. Since the
input data for the minimisation process are essentially two
tables of fixed sizes L jSj and K L jSj, the time and
memory requirements of the optimisation process itself do
not depend on the amount of sequencing reads and neither
do the memory requirements of data preprocessing,
allowing for the efficient processing of gigabytes of sequencing
data even on common computers. All processes are easily
parallelizable and could be potentially accelerated using
a GPU.
4.3
        </p>
      </sec>
      <sec id="sec-4-3">
        <title>Adding error model</title>
        <p>Sequencing reads contain errors, where the symbol in the
read differs from the actual genome being sequenced. We
assume that these errors occur uniformly at random and
add them to the model as follows. Let e 2 (0; 1) be the
substitution error rate. Let m¯ i;a(W ) := 1 mi;a(W ) denote
the probability of observing a base different from base a
at position i in a mixture sample without sequencing
errors. The probability of observing symbol a at position i
in a mixture sample sequenced with errors is then equal to:
qi;a(W; e) = (1
e) mi;a(W ) +
m¯ i;a(W ):</p>
        <p>The likelihood of observing O given mixture weights W
and substitution rate e is then proportional to:
e
3
Pr[OjW; e]</p>
        <p>L
Õ Õ qi;a(W; e)Oi;a :
i=1 a2S</p>
        <p>The error rate e can be set to a particular value, or be
inferred simultaneously with the mixture weights W :
(W ; e ) := arg min</p>
        <p>W;e</p>
        <p>L
å å Oi;a log qi;a(W; e)
i=1 a2S</p>
        <p>In our experiments, we used the latter option.
4.4</p>
      </sec>
      <sec id="sec-4-4">
        <title>Evaluation of a posteriori probability of a given read belonging to a particular variant</title>
        <p>We can represent a read as a data set consisting of only one
read, i.e. Oi;a is equal to 1 if read has symbol a aligned to
reference’s position i, and 0 otherwise. Let V 2 f1; : : : ; Kg
be a random variable representing the variant that the read
belongs to. The likelihood of a read belonging to variant
k is equal to Pr[OjV = k] = Pr[OjW = ek], where ek is the
vector of length K with value 1 at position k and value 0
elsewhere. The probability of the read belonging to variant
k given its sequence is then, by the Bayes theorem, equal
to:</p>
        <p>Pr[V = kjO] =</p>
        <p>Pr[OjV = k] Pr[V = k]</p>
        <p>K
å j=1 Pr[OjV = j] Pr[V = j]
:
Assuming a uniform prior Pr[V ] =
to:</p>
        <p>Pr[V = kjO] =
1</p>
        <p>K
Pr[OjV = k]</p>
        <p>K
å j=1 Pr[OjV = j]
:</p>
        <p>The notion of posterior probability enables us to
classify individual reads into variants by choosing the variant
, the formula reduces
with the maximum a posteriori probability (MAP). We use
this read classification for visualization and filtering as
discussed in the next section.</p>
        <p>Unfortunately, this approach requires to set the
substitution rate e in advance. In our experiments, we estimated
the substitution rate by first running the minimisation
algorithm on the available data.
5
5.1</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Experiments</title>
      <sec id="sec-5-1">
        <title>Simulated data</title>
        <p>
          To evaluate the model, we created simulated data sets,
where the correct proportions of the variants were known.
We have used 37 FASTQ files from the ARTIC
sequencing experiments, conducted at Biomedical Research
Center of the Slovak Academy of Sciences, using protocols
described in [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ], where each SARS-CoV-2 sample was
sequenced separately (using a different barcode).
        </p>
        <p>The list of samples and their classification to variants is
shown in Table 2. Each file was mapped to the reference
and the resulting BAM files were merged to create four
mixed samples as follows:
1. 1x CZ, 1x EU1, 1x EU2, 1x UK (92 840 reads,
average base coverage 5 858)
2. 4x CZ, 2x EU1, 1x EU2, 3x UK, 1x UKBA318
(264 113 reads, average base coverage 16 562)
3. 2x EU1, 1x EU2, 1x other (99 711 reads, average base
coverage 6 273)
4. 3x CZ, 2x EU1, 9x UK, 1x other (335 601 reads,
average base coverage 21 044)</p>
        <p>The true proportions of variants in the samples is
calculated as the proportion of the sums of read lengths for each
variant.
5.2</p>
      </sec>
      <sec id="sec-5-2">
        <title>Results on all reads</title>
        <p>The results on the simulated mixes are shown in Table 3.
We can see that the model is able to distinguish between
present and absent variants, while significantly
overestimating the “other” variant in all samples.</p>
        <p>In order to find an explanation for this discrepancy, we
analysed the posterior probabilities (see subsection 4.4) of
individual reads (see Figure 1). The analysis shows that
there are many “uncertain” reads (i.e. reads with no strong
affinity towards any variant), particularly so among reads,
classified into “other” variant.</p>
        <p>We decided to formalize the notion of “uncertainty” and
filter out such reads from the data set before running the
model.
5.3</p>
      </sec>
      <sec id="sec-5-3">
        <title>Filtering out uncertain reads</title>
        <p>Let k-certainty of a read be defined as a sum of its k highest
posterior probabilities of belonging to a particular variant.</p>
        <p>We decided to use 1-certainty criterion in our
experiments.</p>
        <p>The pipeline works as follows: first we estimate the
substitution error rate by running the model on the full data
set. Then, using the estimated error rate, we evaluate the
1-certainty for each read. Reads with 1-certainty below
a given threshold are then removed from the data set, and
the observed counts Oi;a are calculated. Finally, we run
the model on these counts, reporting the mixture weights
W as the final answer.</p>
        <p>We ran our algorithm on the simulated mixes with
different filtering thresholds. The estimated weights at
different filtration thresholds are shown in Figure 2. The
KL-divergence between the true and estimated weights are
shown in Figure 3.</p>
        <p>This heuristic works reasonably well on data sets
without reads from “other” variant (Figures 2a and 2b).
However, it fails on data sets with “other” variant present
(Figures 2c and 2d) because reads from “other” variant
have generally lower 1-certainty (Figure 1), and are
consequently filtered out.
6</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Discussion</title>
      <p>We have demonstrated that our method can predict the
proportions of individual SARS-CoV-2 variants in mixed
samples when all variants in a sample are known to the
model (i.e. there are no reads from “other” variant).</p>
      <p>In our future work on this topic, we would like to
address the following issues:
variant
Brazil</p>
      <p>CZ
EU1</p>
      <p>EU2
South.Afr</p>
      <p>UK
UKBA318
other</p>
      <p>As we discussed in the previous section, our model
tends to overestimate the “other” variant, and our
current filtering heuristics is not a robust solution. The
filtering may introduce bias into the estimation. This
issue may be addressed by a more systematic
approach to defining the “other” variant.</p>
      <p>We would like to be able to evaluate confidence
intervals on our estimations. That would give us, for
example, the ability to rule out the presence of a
variant with a low estimated weight.</p>
      <p>There are many non-informative reads coming from
the areas of the genome with no significant
alterations between individual variants. The proportion of
such reads is even higher when the reads are shorter
(e.g. obtained by Illumina sequencing). The goal is
to extend the model so that it could handle such data.
The sequencing data shows severe non-uniformity of
the coverage. This issue may be addressed by a
selective weighting of individual reads based on the
coverage at their position.</p>
      <p>The current model does not take into account
insertions and deletions, both in the individual variants and
during the sequencing process.</p>
      <p>The way we characterize the individual variants using
the averaging of all available data from the GISAID
database may incur a bias toward specific subvariants
(e.g. some countries, such as the UK, submit a much
higher number of sequence samples).</p>
      <p>Our approach splits all reads into individual bases,
thus potentially losing all long-range information.
In the future, we would like to adjust the model, so
that it would keep the reads intact.</p>
      <p>In our model we assume that the individual positions
are independent. But, the way the variant profiles
are created (by demanding at least n mutations to be
present in a sequence) introduces some dependency
between them. In the future, we would like to resolve
that inconsistency either by changing of the way the
variant profiles are estimated or by relaxing the
assumption in the model.</p>
      <p>Code availability. The code is available
https://github.com/fmfi-compbio/covid-pooling.
Acknowledgements. We would like to thank the reviewers
for their thorough reviews and helpful suggestions.
Funding. This research was supported by the Operational
Program Integrated Infrastructure within project:
Pangenomics for personalized clinical management of infected
persons based on identified viral genome and human
exome (Code ITMS:313011ATL7, 80%) co-financed by the
European Regional Development Fund. It was also
supported by VEGA 1/0458/18 (10%) and Comenius
University Grant UK/336/2021 (10%).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>B.</given-names>
            <surname>Brejova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Borsova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Hodorova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Cabanova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gafurov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Fricova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Nebohacova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Vinar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Klempa</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Nosek</surname>
          </string-name>
          .
          <article-title>Nanopore sequencing of SARS-CoV-2: Comparison of short and long PCR-tiling amplicon protocols</article-title>
          . medRxiv,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>D. A.</given-names>
            <surname>Coil</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Albertson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Banerjee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Brennan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. J.</given-names>
            <surname>Campbell</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. H.</given-names>
            <surname>Cohen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Dandekar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Díaz-Muñoz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Eisen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Goldstein</surname>
          </string-name>
          , et al.
          <article-title>SARS-CoV-2 detection and genomic sequencing from hospital surface samples collected</article-title>
          at UC Davis. medRxiv,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A.</given-names>
            <surname>Crits-Christoph</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Kantor</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. R.</given-names>
            <surname>Olm</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O. N.</given-names>
            <surname>Whitney</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Al-Shayeb</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y. C.</given-names>
            <surname>Lou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Flamholz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. C.</given-names>
            <surname>Kennedy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Greenwald</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Hinkle</surname>
          </string-name>
          , et al.
          <article-title>Genome sequencing of sewage detects regionally prevalent SARS-CoV-2 variants</article-title>
          . medRxiv,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>ECDC.</surname>
          </string-name>
          <article-title>Rapid increase of a SARS-CoV-2 variant with multiple spike protein mutations observed in the United Kingdom-20 december 2020</article-title>
          . ECDC: Stockholm,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S.</given-names>
            <surname>Elbe</surname>
          </string-name>
          and
          <string-name>
            <given-names>G.</given-names>
            <surname>Buckland-Merrett</surname>
          </string-name>
          .
          <article-title>Data, disease and diplomacy: GISAID's innovative contribution to global health</article-title>
          .
          <source>Global Challenges</source>
          ,
          <volume>1</volume>
          (
          <issue>1</issue>
          ):
          <fpage>33</fpage>
          -
          <lpage>46</lpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Fontenele</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Kraberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hadfield</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. M.</given-names>
            <surname>Driver</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Bowes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. A.</given-names>
            <surname>Holland</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. O.</given-names>
            <surname>Faleye</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Adhikari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Kumar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Inchausti</surname>
          </string-name>
          , et al.
          <article-title>High-throughput sequencing of SARS-CoV-2 in wastewater provides insights into circulating variants</article-title>
          . medRxiv,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>K.</given-names>
            <surname>Jahn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Dreifuss</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Topolsky</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Kull</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Ganesanandamoorthy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Fernandez-Cassi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bänziger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Stachler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Fuhrmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. P.</given-names>
            <surname>Jablonski</surname>
          </string-name>
          , et al.
          <article-title>Detection of SARS-CoV-2 variants in Switzerland by genomic analysis of wastewater samples</article-title>
          . medRxiv,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>E.</given-names>
            <surname>Jones</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Oliphant</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Peterson</surname>
          </string-name>
          , et al.
          <source>SciPy: Open source scientific tools for Python</source>
          ,
          <fpage>2001</fpage>
          -.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>K.</given-names>
            <surname>Kupferschmidt</surname>
          </string-name>
          .
          <article-title>The pandemic virus is slowly mutating</article-title>
          .
          <source>but does it matter?</source>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>H.</given-names>
            <surname>Li</surname>
          </string-name>
          .
          <article-title>Minimap2: pairwise alignment for nucleotide sequences</article-title>
          .
          <source>Bioinformatics</source>
          ,
          <volume>34</volume>
          (
          <issue>18</issue>
          ):
          <fpage>3094</fpage>
          -
          <lpage>3100</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <source>[11] NCIRD. Science brief: Emerging SARS-CoV-2 variants</source>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>M. R.</given-names>
            <surname>Olm</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Crits-Christoph</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Bouma-Gregson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. A.</given-names>
            <surname>Firek</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. J.</given-names>
            <surname>Morowitz</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J. F.</given-names>
            <surname>Banfield</surname>
          </string-name>
          .
          <article-title>inStrain profiles population microdiversity from metagenomic data and sensitively detects shared microbial strains</article-title>
          .
          <source>Nature Biotechnology</source>
          , pages
          <fpage>1</fpage>
          -
          <lpage>10</lpage>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>A.</given-names>
            <surname>Pérez</surname>
          </string-name>
          <string-name>
            <given-names>Cataluña</given-names>
            , Á.
            <surname>Chiner-Oms</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. Cuevas</given-names>
            <surname>Ferrando</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Díaz-Reolid</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Falcó</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Randazzo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Girón-Guzmán</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Allende</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Bracho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Comas</surname>
          </string-name>
          , et al.
          <article-title>Detection of genomic variants of SARS-CoV-2 circulating in wastewater by high-throughput sequencing</article-title>
          .
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>A.</given-names>
            <surname>Rambaut</surname>
          </string-name>
          et al.
          <article-title>Preliminary genomic characterisation of an emergent SARS-CoV-2 lineage in the UK defined by a novel set of spike mutations</article-title>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>A.</given-names>
            <surname>Rambaut</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. C.</given-names>
            <surname>Holmes</surname>
          </string-name>
          ,
          <string-name>
            <surname>Á. O'Toole</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Hill</surname>
            ,
            <given-names>J. T.</given-names>
          </string-name>
          <string-name>
            <surname>McCrone</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Ruis</surname>
          </string-name>
          , L. du
          <string-name>
            <surname>Plessis</surname>
            , and
            <given-names>O. G.</given-names>
          </string-name>
          <string-name>
            <surname>Pybus</surname>
          </string-name>
          .
          <article-title>A dynamic nomenclature proposal for SARS-CoV-2 lineages to assist genomic epidemiology</article-title>
          .
          <source>Nature microbiology</source>
          ,
          <volume>5</volume>
          (
          <issue>11</issue>
          ):
          <fpage>1403</fpage>
          -
          <lpage>1407</lpage>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>[16] The World Health Organization. Tracking SARSCoV-2 variants</source>
          ,
          <year>2021</year>
          . https://www.who.int/en/ activities/tracking-SARS-CoV-2-variants/.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>J. R.</given-names>
            <surname>Tyson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>James</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Stoddart</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Sparks</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Wickenhagen</surname>
          </string-name>
          , G. Hall,
          <string-name>
            <given-names>J. H.</given-names>
            <surname>Choi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Lapointe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Kamelian</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. D.</given-names>
            <surname>Smith</surname>
          </string-name>
          , et al.
          <article-title>Improvements to the ARTIC multiplex PCR method for SARS-CoV-2 genome sequencing using nanopore</article-title>
          . bioRxiv,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>M.</given-names>
            <surname>Wasiuta</surname>
          </string-name>
          . Worldometers,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>C.</given-names>
            <surname>Zhu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. H.</given-names>
            <surname>Byrd</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Lu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Nocedal</surname>
          </string-name>
          . Algorithm 778:
          <string-name>
            <surname>L-BFGS-B</surname>
          </string-name>
          :
          <article-title>Fortran Subroutines for Large-Scale BoundConstrained Optimization</article-title>
          .
          <source>ACM Transactions on Mathematical Software</source>
          ,
          <volume>23</volume>
          (
          <issue>4</issue>
          ):
          <fpage>550</fpage>
          -
          <lpage>560</lpage>
          ,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>