<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Conference and Labs of the Evaluation Forum, September</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Overview of SnakeCLEF 2024: Revisiting Snake Species Identification in Medically Important Scenarios</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Lukas Picek</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marek Hrúz</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Andrew M. Durso</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Biological Sciences, Florida Gulf Coast University</institution>
          ,
          <addr-line>Florida</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Cybernetics, Faculty of Applied Sciences, University of West Bohemia</institution>
          ,
          <country country="CZ">Czech Republic</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Inria, LIRMM, University of Montpellier</institution>
          ,
          <addr-line>CNRS, Montpellier</addr-line>
          ,
          <country country="FR">France</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <volume>0</volume>
      <fpage>9</fpage>
      <lpage>12</lpage>
      <abstract>
        <p>The SnakeCLEF challenge serves as a major benchmark for evaluating the performance of AI-driven methods in snake species recognition on a global scale. The 5th edition of the SnakeCLEF challenge builds on last year's training data and extends the test set with new data from private collections originating from southern Africa. Similar to last year, SnakeCLEF 2024 focuses on (i) evaluating incremental improvements in automatic snake species identification, (ii) testing global generalization in three specific scenarios: India, Central America, and southern Africa, and (iii) assessing the impact of uneven error costs, such as mistaking a venomous snake for a harmless one. In this paper, we highlight the crucial importance of a robust automatic snake identification system, especially in resource-limited environments and in neglected regions, and its potential benefits for biodiversity conservation and global health. We present (i) a detailed description of the provided data, (ii) the evaluation methodology, (iii) an overview of the submitted methods, and (iv) insights gained from the results.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;LifeCLEF</kwd>
        <kwd>SnakeCLEF</kwd>
        <kwd>fine grained visual categorization</kwd>
        <kwd>global health</kwd>
        <kwd>epidemiology</kwd>
        <kwd>snake bite</kwd>
        <kwd>snake</kwd>
        <kwd>reptile</kwd>
        <kwd>benchmark</kwd>
        <kwd>biodiversity</kwd>
        <kwd>species identification</kwd>
        <kwd>machine learning</kwd>
        <kwd>computer vision</kwd>
        <kwd>classification</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        With annual deaths and disabilities exceeding half a million due to venomous snakebites, a robust
image-based system for snake species identification could significantly enhance eco-epidemiological
data and treatment outcomes [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ]. Most herpetological expertise is concentrated in developed countries
like those in North America and the EU, where snake diversity is relatively low and snakebites are not
a major public health concern. In contrast, remote regions of developing countries with high snake
diversity and many snakebites often need more expertise and data. Therefore, providing assistance with
snake species identification has excellent potential for saving lives, especially in these high-need regions
[
        <xref ref-type="bibr" rid="ref3 ref4">3, 4</xref>
        ]. Enhancing the ability to correctly identify snake species can directly impact the efectiveness of
medical responses to snakebites, ensuring the correct antivenom is provided. Additionally, AI-powered
tools can enhance education, helping populations in remote and neglected regions recognize dangerous
species and take preventative measures.
      </p>
      <p>
        The main challenge in identifying snake species arises from high intra-class variation and low
inter-class similarities in appearance, influenced by factors such as geographic location, color morph,
sex, or age. Besides, many species have evolved to have visual similarities to other species as a
defense mechanism (e.g., mimicry). Interestingly, some snake species resemble species found on
diferent continents without overlapping ranges. Therefore, leveraging information about an unknown
snake’s geographic origin could help with the identification process, given that only about 125 out
of approximately 4,000 snake species co-occur in any specific location [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. Species with broader
distributions and more images tend to be more accurately predicted than rare species with limited
photographic records [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. This poses significant challenges when identifying species that are widespread
across regions with sparse image data.
      </p>
      <p>
        In order to assess the performance of AI-driven methods for snake species recognition on a global
scale, we are organizing an annual competition – SnakeCLEF. The SnakeCLEF 2024 took part in the
LifeCLEF [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], and FGVC-CVPR workshops and was hosted on the HuggingFace platform to attract
participation from machine learning experts and boost idea exchange. This edition of SnakeCLEF
focuses on (i) evaluating incremental improvements in automatic snake species identification, (ii) testing
generalization across three specific scenarios (e.g., India, Central America, and southern Africa), and (iii)
assessing error costs, particularly mistaking venomous snakes for harmless ones. In order to underscore
the importance of a robust automatic snake identification system, especially in resource-limited regions,
and highlight its potential impact on biodiversity conservation and global health outcomes, we provide
(i) a comprehensive description of the dataset, (ii) details of the evaluation methodology, (iii) an overview
of submitted methods, and (iv) insights derived from the results.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Objective</title>
      <p>In previous SnakeCLEF editions [8, 9, 10, 11], we have learned that AI-based methods achieve high
performance (F1 and Accuracy &gt; 90%) even in scenarios with long-tailed distributions and thousands
of species but tend to underperform in medically important neglected regions. In light of that, we kept
the training data and objective the same as last year but extended the test set. This allows us to easily
compare this year’s performance with last year’s and allows for a more comprehensive generalization
evaluation in neglected regions. The main objective was to create a classification model that returns
a ranked list of predicted species for each real snake observation and minimize the danger to human
life and the waste of antivenom if a bite from those observations were treated as coming from the
top-ranked prediction. The classification model had to fit memory footprint limits and a prediction time
limit (60 minutes) within a given server instance (Nvidia T4 small 4vCPU, 15GB RAM, 16GB vRAM).</p>
      <sec id="sec-2-1">
        <title>2.1. Test Scenarios</title>
        <p>This year, we focused on three geographic scenarios significant for global snakebites: (i) a biodiverse
country historically focused on a few species of medically important venomous snakes (MIVS), (ii) a
small tropical region comprising four adjacent countries with varying socio-economic statuses but
shared snake fauna, and (iii) a large temperate region socio-economically similar to (ii) with more
realistic test data.</p>
        <p>Central America: In this region, snakebite is a significant public health issue, particularly afecting
rural populations [12]. Panama reports the highest incidence of snakebite cases in the Americas [13].
Costa Rica is a global leader in antivenom research with robust epidemiological data and good healthcare
quality [14, 15]. Together, these four countries host 43 MIVS, of which we cover 38, as well as 68% of
non-venomous species. Central America has many species of venomous coralsnakes (genus Micrurus)
and their harmless mimics [16], which are frequently misclassified by both humans and AI systems
[17, 18]. However, most bites are caused by Bothrops asper (Terciopelo) [19], a pit viper with several
harmless lookalikes (see Figure 1).
Southern Africa: Across South Africa, Namibia, Botswana and Zimbabwe, there are a total of 160
snake species (32 MIVS and 128 non-venomous species) [20, 21], of which we cover 121 [75.6%] (28
MIVS [87.5%] and 93 non-venomous [72.7%]). About 43% of species are endemic to one of the countries,
and 31% of species are found in all the countries, with the remaining 26% being found in either two or
three. This region is over 8× larger than Central America and is temperate and arid rather than tropical.
Snakebite is a public health issue in rural areas, but snakebite epidemiology in Africa generally lags
behind other regions [22, 23]. South Africa, the most developed country, has lower per-capita incidence
and mortality. There is a high potential for generalization to adjacent countries, as political boundaries
do not align with ecoregions.</p>
        <p>India: India is the global leader in snakebites [24] with 40,000–60,000 annual deaths [25]. Although &gt;75
MIVS are found in India [26], the "Big Four"—spectacled cobra (Naja naja), common krait (Bungarus
caeruleus), Russell’s viper (Daboia russelii), and saw-scaled viper (Echis carinatus) — are the most
wellknown and widespread and receive the most research attention and focus [27]. In this challenge, we
include 44 MIVS (Big Four + 40 others) and 51% (128/253) of non-venomous snakes found in India,
including many visually similar species (see Figure 3).</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Dataset</title>
      <p>Same as in the previous edition [11], the dataset was assembled from three sources. The training, and
validation subsets were derived from observations submitted to the citizen science platforms iNaturalist
and HerpMapper. The competition test set (Private Test) consists of private images from individuals and
natural history museums, which have not been published online, and thus, participants do not have
access to this data.</p>
      <p>In total, the collected data include approximately 110,000 real snake observations of 1,784 species
with community-verified species labels. The number of selected species covers roughly half of all
existing snake species, which makes it the dataset with the biggest species and geographical coverage.
Compared to previous editions (i.e., 2020–2022), there has been a significant increase in observations
from remote geographic areas that previously had few or no samples, improving the distribution of
species data across various countries. The dataset comparison with previous SnakeCLEF editions is
available in Table 2. Despite coverage improvements, there are still fewer observations and poorer
species coverage in some developing countries.</p>
      <p>Apart from the image data, the dataset includes information about the medical importance of each
species, indicating whether a species is venomous or not. A country-species relevance matrix is also
provided. Detailed statistics of the dataset are presented in Table 1, with a direct comparison to previous
editions of the SnakeCLEF datasets shown in Table 2.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Evaluation Protocol</title>
      <p>Similarly, as in the previous edition [11], we ask participants to provide a code that predicts snake
species on unseen snake observations. Participants had to submit their solutions as HuggingFace models
and submit them through the competition space. Participants could run any model or architecture but
must fit limits for memory footprint and a prediction time limit (60 minutes) within a given HuggingFace
server instance (Nvidia T4 small 4vCPU, 15GB RAM, 16GB VRAM). For an easier start, we provided a
sample submission based on ONNX and PyTorch.</p>
      <sec id="sec-4-1">
        <title>4.1. Evaluation Metrics</title>
        <p>To allow a direct comparison to last year’s performance, we used the same metric, which motivates
research in recognition scenarios with uneven costs for diferent errors, such as mistaking a venomous
snake for a harmless one. Last year, we introduced two metrics that consider venomous →← harmless
confusion with diferent error costs. Those are defined as follows:
Track1 metric: The first track metric is the sum of () over all test observations:
(1)
(2)
L = ∑︁ (, ˆ).</p>
        <p>Let us consider a function p such that p(s) = 1 if species s is venomous; otherwise, p(s) = 0. For a correct
species y and predictor species ˆ, the loss (, ˆ) is given as follows:
⎧0 if  = ˆ
⎪⎪⎪⎪1 if  ̸= ˆ and () = 0 and (ˆ) = 0
(, ˆ) = ⎨2 if  ̸= ˆ and () = 0 and (ˆ) = 1
⎪⎪⎪2 if  ̸= ˆ and () = 1 and (ˆ) = 1
⎪⎩5 if  ̸= ˆ and () = 1 and (ˆ) = 0
Track2 metric: The second track metric includes the overall classification rate (macro averaged F 1) and
the venomous species confusion error. The metric is a weighted average between the macro F1-score
and the weighted accuracy of diferent types of confusion.</p>
        <p>= 1F1 + 2Cℎ ˃ℎ + 3Cℎ ˃ + 4C ˃ + 5C ˃ℎ , (3)</p>
        <p>1 + 2 + 3 + 4 + 5
where 1 = 1, 2 = 1, 3 = 2, 4 = 2, and 5 = 5 are the weights of individual confusions.
C ˃ℎ is the percentage of wrongly classified venomous species as a harmless species.
Cℎ ˃ is the percentage of wrongly classified harmless species as a venomous species,
C ˃ is the percentage of wrongly classified venomous species as another venomous species,
Cℎ ˃ℎ is the percentage of wrongly classified harmless species as another harmless species, and the F 1 is
the macro averaged F1-score.</p>
        <p>This metric ranges from 0% to 100%. A score of 0% is achieved if every species is misclassified,
especially if all harmless species are mistaken for venomous ones and vice versa. Conversely, a score of
100% (where every species is correctly classified) means every  must be zero by definition.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Timeline</title>
        <p>The SnakeCLEF 2024 competition was announced together with the dataset release on February 29,
2023, through the LifeCLEF, HuggingFace, and FGVC challenge pages. Anyone was allowed to register
and participate in the competition. The competition deadline was May 24, making the competition
duration approximately 3 months. The test data was kept secret, as was the case last year. Participants
were allowed five submissions per day. Two weeks before the deadline, this limit was increased to 10
submissions per day. After the submission phase closed, all participants could submit post-competition
submissions to evaluate their ablation studies.</p>
      </sec>
      <sec id="sec-4-3">
        <title>4.3. Working Notes</title>
        <p>All participants were encouraged to submit both their code and a technical report (Working Note)
detailing the information required to reproduce their results. Each Working Note was reviewed by 2–3
reviewers with strong publication records in computer vision and machine learning, ensuring high
standards of reproducibility and quality. The review process was single-blind and allowed for up to two
rounds of rebuttals.</p>
      </sec>
      <sec id="sec-4-4">
        <title>4.4. Baseline</title>
        <p>
          To allow an easier start for all participants and straightforward model evaluation, we provide a relatively
strong baseline based on the ONNX Swin-v2-tiny model [28] wrapped into a HuggingFace repository that
can be directly submitted for evaluation on the private test set. Besides the PyTorch-based submission,
we also provide an example for submitting the PyTorch model that was primarily provided for another
LifeCLEF [
          <xref ref-type="bibr" rid="ref7">7, 29</xref>
          ] competition – FungiCLEF 2024 [30]. The Swin-v2-tiny model was trained for 50
epochs with SeeSaw loss [31] using the FGVC framework. Other training hyperparameters are as
follows: batch size of 128, initial learning rate of 0.01, SGD optimizer, plateau learning rate schedule,
and Randaugment [32].
        </p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Challenge Results</title>
      <p>In Figure 4, we report private leaderboard performance achieved by individual teams in terms of Track 1
(1) and Track 2 (2) metrics. The best-performing team in both tracks upupup [33], achieved 83.6% in
Metric 1 and 1, 840 in Metric 2 on the private part of the test set. Interestingly, only 5 teams outperformed
the provided baseline. Unlike last year, when the top two participants achieved a competitive and
comparable performance, this year’s diference between the participants was relatively large, 3.6% and
640 points in terms of Track1 and Track2 metrics between the top1 and top2 teams. More precisely, in
the case of the Track2 metric, the best-performing team reduced the relative error of the runner-up by
more than 25%.</p>
      <p>100</p>
      <p>80
c
i
tr 60
e
M
1
k
c 40
a
r
T
20
0
jacukp-eutphuepredgeTZTEMKithZilCsSsUw-iKnTK-evYn2s-otirnSyDnSalo@koeGpsTc-iLdwieferuCmLbEoFlogist
jacukp-eutphuepredgTeZTMEKithZiSlCswUsi-nTKe-KvnY2s-otrinSDynSal@okoeGpsTc-iLdwifeuermCLbEoFlogist</p>
      <p>Comparing the performance of SnakeCLEF 2023 and 2024 participants (see Table 3) revealed a
relatively large drop in performance from last year. This noticeable decline in the participants’ performance
is most likely related to the more stringent hardware limitations imposed in SnakeCLEF 2024. The
hardware constraints introduced this year have posed significant challenges for the teams, as the
computational resources allowed are considerably less than those available in the previous year’s competition.
None of the teams from SnakeCLEF 2023 would have been able to meet these new, stricter hardware
requirements, highlighting the substantial impact these limitations have had on the ability to develop
and deploy high-performance solutions. Consequently, the stricter hardware constraints have likely
necessitated adjustments in strategy and implementation, resulting in a performance drop as teams
adapt to the new conditions. This change underscores the importance of hardware resources in ML
competitions and the direct correlation between computational power and algorithmic eficiency and
efectiveness.
Central America: In the Central America scenario, with snake species observations from Costa
Rica, Nicaragua, Honduras, and Panama, the competition was "intense". The top four teams achieved
comparable scores across both tracks’ metrics, demonstrating a high level of performance and
consistency. Specifically, in Track 1, these teams attained impressive accuracy rates ranging from 85%
to 90%. Similarly, in Track 2, which measures performance based on a diferent set of criteria, the
scores ranged from 358 to 504. These results reflect the teams’ adeptness in handling the varied and
complex data from the Central American region. The high Track 1 and low Track 2 scores highlight the
robustness of the provided approaches and the efectiveness of their methods in adapting to the specific
environment, such as Central America.</p>
      <p>India: As with last year, the Indian scenario proved to be a significant challenge and was a deal
breaker for many participants. Despite the first-runner-up team’s strong performance in the Central
America and Africa scenarios, where they matched the winning team in Track1 and Track2 metrics,
their error in Metric 2 for the India scenario increased by around 50%. This highlights the unique and
complex dificulties of the Indian scenario, possibly due to a diverse range of snake species, varied
environmental conditions, and more complex data sets, although none of these seem fundamentally
diferent from Central America (in fact, mimicry of Indian snakes is less exact than in the Neotropics).
The consistent struggle with this scenario emphasizes the need for robust models that can generalize
well across diferent geographic contexts and suggests that further research and targeted improvements
are necessary to address these challenges efectively.</p>
      <p>Southern Africa: The African scenario stands out due to its notably low Track 1 scores across the
board, with a baseline of 56% and a top score of 75%, but relatively low Track 2 scores (i.e., 688 for
baseline and 358 for the best team). These scores are particularly significant because the test data are
the most realistic, having been drawn from completely independent sources and collected by hand from
relevant scenarios, especially individuals in Southern Africa seeking snake identifications. This method
of data collection ensures a high level of authenticity and relevance, making the results particularly
indicative of real-world performance. Interestingly, the relatively low number of mimic species in
Southern Africa might explain why the Track 2 metric wasn’t as high as in India, where mimicry is more
common. This scenario’s unique challenges highlight the need for models that can accurately interpret
and analyze data from diverse and independently sourced datasets, emphasizing the importance of
adaptability and precision in developing efective snake identification tools.
1
0
.
1
9 6
1
.81 .82
5
7
upupup
jack-etheredge</p>
      <sec id="sec-5-1">
        <title>TZTEK</title>
        <p>jack-etheredge</p>
      </sec>
      <sec id="sec-5-2">
        <title>TZTEK</title>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Participants and Methods</title>
      <p>This year, a total of 14 teams participated in the SnakeCLEF. However, just nine teams submitted
solutions diferent from the baseline, and four submitted working notes. Details of the best methods
and systems used are synthesized below, with further elaboration available in the individual working
notes submitted by the participants [33, 37, 38, 39].
upupup (Top1) [33]: The main idea used by the team is that the venomous and the harmless snakes
should be recognized by diferent models. This is in the spirit of the competition when diferent
misclassifications are penalized. The rationale behind this approach is that common visual features
are detected for both types of snakes, and the later stages of the network focus on fine-grained visual
features of the venomous and harmless species, but then each branch uses diferent weights for the
fourth stage of the ConvNeXt model. The first branch is used for the classification of all the species and
is also responsible for the computation of the gating parameter. This parameter is estimated from the
globally averaged features produced by the first three stages of the network, which are combined with
the features produced by the first branch of the model. The authors use global maximum pooling for
both feature maps individually and combine them afterward. Then, the sigmoid activation function is
used to compute the scalar gating parameter. The second branch focuses on venomous snakes, while
the third one focuses on harmless species. The gating parameter is used to decide which of these
branches will be used for prediction and gradient-based weight optimization. This approach is used
only in the training phase and is omitted for the inference. However, the authors show that this training
setup helps the model to perform well overall. A combination of Seesaw loss [31] and CE loss is used
for optimization. The team reached the best overall results in the competition with 81.16% and 91.01%
in Metric 1 in India and Central America, respectively, and penalties of 1,844 pts and 325 pts in Metric 2.
jack-etheredge (Top2) [37]: The team uses a CAFormer [40] model in the final solution. They
introduce a new venom loss, which considers the diferent penalties for misclassification. A cost matrix
between the predicted class and the misclassification penalty is used to re-weight the softmax values of
the prediction. This loss is similar to the weighted cross entropy loss but uses the costs directly instead
of utilizing a weighted log loss. The addition of the venom loss significantly improves the performance
of the tested models across all metrics, even the F1 score. The team uses an ensemble of models
trained on diferent data splits. Contrary to open set problems, the LogitNorm [ 41] did not improve the
recognition rate. The team reached the second overall result in the competition with 76.7% and 89.11%
in Metric 1 in India and Central America, respectively, and penalty of 2,657 pts and 401 pts in Metric 2.
ZCU-KKY (Top5) [39]: The team uses a Swin-v2 Tiny [28] model for the recognition. This is so that the
model can be used on mobile devices for fast and practical inference. This is aligned with the mission of
this competition when a fast response of the system is essential. The team combines two heads - one is
for the species classification, and the other one is for venomous/harmless classification. This approach
is similar to the "upupup" team and shows that handling the venomous and harmless species as two
distinct groups produces better results. A combination of the Seesaw loss for species recognition and
the binary cross entropy loss for venomousness detection is used for optimization. The one-hot target
for the species identification CE loss is transformed by using a temperature parameter that is based on
the diferent penalties set for the misclassification of venomous and harmless species. The authors
show a relative improvement of around 9% when compared to the CE loss with one-hot targets. Even
though the results are not as good as the results of other teams, they show an improvement over the
baseline model by introducing the head responsible for venomousness recognition. The team reached
the result of 64.84% and 78.51% in Metric 1 in India and Central America, respectively, and penalty of
4,591 pts and 933 pts in Metric 2.</p>
      <p>DS@GT-LifeCLEF (Top14) [38]: Although the results of this team are not as good as the baseline
approach, the presented methods show the potential of self-supervised models. The authors use the
Dinov2 model (ViT-B/14 distilled) pre-trained on the LVD-142M dataset to extract the features from
the image. Both the CLS token is used and a DCT of the patch features. The approach to represent
the patches is such that the 16 × 16 patch representations with 768 dimensions are reshaped into a
matrix of size 256 × 768, and a 2D DCT is computed on this signal. The coeficients of the top left
8 × 8 cosine representation (i.e., the smallest frequencies) are flattened into a 64-dimensional final
representation. The authors use this representation together with the CLS token representation to
visualize their discriminative capabilities. The DCT representation does not seem to form any useful
clusters, unlike the CLS representation. However, the pipeline used to compute the DCT coeficients
might not take full advantage of the patch representation. The reshaping of the patch tokens might
be the culprit and, if handled in a more efective way, might provide some additional insides into the
process of self-supervised learning. The authors train a linear layer on top of the representations to
classify them into predefined classes. The results are sub-par, far behind the baseline experiment. The
idea itself is interesting. However, the results do not show the full potential of the solution.</p>
    </sec>
    <sec id="sec-7">
      <title>7. Conclusions and Perspectives</title>
      <p>This paper presents an overview and results evaluation of the 5th edition of the SnakeCLEF challenge,
organized in conjunction with the CLEF, LifeCLEF lab and the CVPR-FGVC11 – The 11th Workshop
on Fine-Grained Visual Categorization, organized in conjunction with the CVPR conference. The
SnakeCLEF challenge focuses on the challenging task of snake species identification from images,
promoting advancements in fine-grained visual categorization and the development of robust machine
learning models not just for global health. In summary, the fifth edition of the SnakeCLEF challenge
has not only advanced state-of-the-art snake species recognition but also promoted a collaborative
environment for researchers to exchange ideas and methodologies. The main outcomes from this year’s
evaluation are as follows:
Respect the penalties: Every team showed that customizing the loss according to given penalties leads
to better results. The systems are more careful when dealing with possibly venomous snakes. Even
though this is an expected behavior given the evaluation metric, it is proof that carefully designed
penalties lead to the desired properties of the systems. Other desired properties may be reached by carefully
designing the evaluation metric. For example, it may incorporate the economic, social, or medical needs
of diferent geographical areas. To conclude, the introduction of a custom loss that accounts for the
diferent penalties for misclassification always helps. It seems to be the leading factor in improving the results.
Handle venomous and harmless species diferently : A branching or multi-head approach to
the classification of venomous vs. harmless species is another important factor in achieving better
results. Although these mechanisms predominantly aim at optimizing the competition metric, they also
improve the F1 scores. This is interesting because it shows that there are recognizable visual queues
for venomousness, and it is best to model them explicitly. The winning team uses the detection of
venomousness as a proxy task, which leads the model to specialize on diferent species in its branches.
However, the main branch that provides the decision for the other branches then becomes a better
species recognizer.</p>
      <p>CNNs are back in the game? Maybe with no metadata.: The architecture of the model (CNN vs.
Transformer) is not a major cause of the success. The winning team used a CNN, while the second team
used a Transformer. Choosing the architecture according to other factors, such as run time or memory
limitations, might be possible. Also, in the last year’s competition, it has been shown that metadata
helps a lot. In such cases, a transformer-based architecture is able to utilize them more eficiently. One
may estimate that the future belongs to Transformers, however, for purely vision-based tasks one may
not rule out CNNs.</p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgments</title>
      <p>LP and MH were supported by the Technology Agency of the Czech Republic, project No. SS05010008.
AMD was supported by NSF BRC-BIO award no. 2313356.
on species distribution prediction and identification, in: International Conference of the
CrossLanguage Evaluation Forum for European Languages, Springer, 2024.
[8] L. Picek, R. Ruiz De Castañeda, A. M. Durso, P. M. Sharada, Overview of the snakeclef 2020:
Automatic snake species identification challenge, in: Working Notes of CLEF 2020 - Conference
and Labs of the Evaluation Forum, 2020.
[9] L. Picek, A. M. Durso, I. Bolon, R. Ruiz de Castañeda, Overview of snakeclef 2021: Automatic
snakespecies identification with country-level focus, in: Working Notes of CLEF 2021 - Conference
and Labs of the Evaluation Forum, 2021, pp. 1463–1476.
[10] L. Picek, A. M. Durso, M. Hrúz, I. Bolon, Overview of SnakeCLEF 2022: Automated snake species
identification on a global scale, in: Working Notes of CLEF 2022 - Conference and Labs of the
Evaluation Forum, 2022, pp. 1957–1969.
[11] L. Picek, M. Šulc, R. Chamidullin, A. M. Durso, Overview of snakeclef 2023: Snake identification
in medically important scenarios, in: Working Notes of CLEF 2023 - Conference and Labs of the
Evaluation Forum, 2023.
[12] J. M. Gutiérrez, Clinical toxicology of snakebite in central america, in: Handbook of: Clinical</p>
      <p>Toxicology of Animal Venoms and Poisons, CRC Press, 2017, pp. 645–665.
[13] J. M. Gutiérrez, H. W. Fan, C. L. Silvera, Y. Angulo, Stability, distribution and use of antivenoms
for snakebite envenomation in latin america: report of a workshop, Toxicon 53 (2009) 625–630.
[14] J. M. Gutiérrez, K. Maduwage, G. Iliyasu, A. Habib, Snakebite envenoming in diferent national
contexts: Costa rica, sri lanka, and nigeria, Toxicon: X 9 (2021) 100066.
[15] J. M. Gutiérrez, C. F. Teixeira, H. W. Fan, Instituto butantan and instituto clodomiro picado: A
long-standing partnership in science, technology, and public health, Toxicon 202 (2021) 75–81.
[16] A. R. Davis Rabosky, C. L. Cox, D. L. Rabosky, P. O. Title, I. A. Holmes, A. Feldman, J. A. McGuire,
Coral snakes predict the evolution of mimicry across new world snakes, Nature communications
7 (2016) 11484.
[17] A. M. Durso, I. Bolon, A. R. Kleinhesselink, M. R. Mondardini, J. L. Fernandez-Marquez, F.
GutscheJones, C. Gwilliams, M. Tanner, C. E. Smith, W. Wüster, et al., Crowdsourcing snake identification
with online communities of professional herpetologists and avocational snake enthusiasts, Royal
Society open science 8 (2021) 201273.
[18] I. Bolon, L. Picek, A. M. Durso, G. Alcoba, F. Chappuis, R. Ruiz de Castañeda, An artificial
intelligence model to identify snakes from across the world: Opportunities and challenges for
global health and herpetology, PLoS neglected tropical diseases 16 (2022) e0010647.
[19] R. Otero-Patino, Epidemiological, clinical and therapeutic aspects of bothrops asper bites, Toxicon
54 (2009) 998–1011.
[20] D. G. Broadley, F. V. F. M., Fitzsimons’ Snakes of Southern Africa, Delta Books, Johannesburg, 1983.
[21] P. Uetz, P. Freed, J. Hošek, et al., The reptile database, 2024. URL: https://reptile-database.reptarium.</p>
      <p>cz/advanced_search.
[22] J.-P. Chippaux, Estimate of the burden of snakebites in sub-saharan africa: a meta-analytic
approach, Toxicon 57 (2011) 586–599.
[23] S. Halilu, G. Iliyasu, M. Hamza, J.-P. Chippaux, A. Kuznik, A. G. Habib, Snakebite burden in
sub-saharan africa: estimates from 41 countries, Toxicon 159 (2019) 1–4.
[24] W. Suraweera, D. Warrell, R. Whitaker, G. Menon, R. Rodrigues, S. H. Fu, R. Begum, P. Sati,
K. Piyasena, M. Bhatia, et al., Trends in snakebite deaths in india from 2000 to 2019 in a nationally
representative mortality study, Elife 9 (2020) e54076.
[25] B. Mohapatra, D. A. Warrell, W. Suraweera, P. Bhatia, N. Dhingra, R. M. Jotkar, P. S. Rodriguez,
K. Mishra, R. Whitaker, P. Jha, et al., Snakebite mortality in india: a nationally representative
mortality survey, PLoS Negl Trop Dis 5 (2011) e1018.
[26] R. Whitaker, A. Captain, Snakes of india. the field guide. 2nd edi, 2008.
[27] R. S. Laxme, S. Khochare, H. F. de Souza, B. Ahuja, V. Suranse, G. Martin, R. Whitaker, K. Sunagar,
Beyond the ‘big four’: Venom profiling of the medically important yet neglected indian snakes
reveals disturbing antivenom deficiencies, PLoS neglected tropical diseases 13 (2019) e0007899.
[28] Z. Liu, H. Hu, Y. Lin, Z. Yao, Z. Xie, Y. Wei, J. Ning, Y. Cao, Z. Zhang, L. Dong, et al., Swin
transformer v2: Scaling up capacity and resolution, in: Proceedings of the IEEE/CVF conference
on computer vision and pattern recognition, 2022, pp. 12009–12019.
[29] A. Joly, L. Picek, S. Kahl, H. Goëau, V. Espitalier, C. Botella, B. Deneu, D. Marcos, J. Estopinan,
C. Leblanc, et al., Lifeclef 2024 teaser: Challenges on species distribution prediction and
identification, in: European Conference on Information Retrieval, Springer, 2024, pp. 19–27.
[30] L. Picek, M. Sulc, J. Matas, Overview of FungiCLEF 2024: Revisiting fungi species recognition
beyond 0-1 cost, in: Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum,
2024.
[31] J. Wang, W. Zhang, Y. Zang, Y. Cao, J. Pang, T. Gong, K. Chen, Z. Liu, C. C. Loy, D. Lin, Seesaw loss
for long-tailed instance segmentation, in: Proceedings of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition, 2021, pp. 9695–9704.
[32] E. D. Cubuk, B. Zoph, J. Shlens, Q. V. Le, Randaugment: Practical automated data augmentation
with a reduced search space, in: Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition workshops, 2020, pp. 702–703.
[33] P. Wang, Y. Li, B.-F. Tan, Y.-C. Zhou, Y. Li, X.-S. Wei, Multibranch co-training to mine venomous
feature representation: A solution to snakeclef2024, in: Working Notes of CLEF 2024 - Conference
and Labs of the Evaluation Forum, 2024.
[34] F. Hu, P. Wang, Y. Li, C. Duan, Z. Zhu, F. Wang, F. Zhang, Y. Li, X.-S. a. Wei, Metaformer model
with arcfaceloss and contrastive learning for snakeclef2023 fine-gained classification, in: Working
Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum, 2023.
[35] B. Bracke, L. Bloch, M. Bagherifar, C. M. Friedrich, Combination of object detection, geospatial
data, and feature concatenation for snake species identification, in: Working Notes of CLEF 2023
Conference and Labs of the Evaluation Forum, 2023.
[36] Z. Shi, H. Chen, C. Liu, J. Qiu, Metaformer model with arcfaceloss and contrastive learning for
snakeclef2023 fine-gained classification, in: Working Notes of CLEF 2023 - Conference and Labs
of the Evaluation Forum, 2023.
[37] J. Etheredge, Generalizable training techniques for fine-grained long-tailed image recognition:
Transferring methods optimized for fungiclef 2024 to snakeclef 2024, in: Working Notes of CLEF
2024 - Conference and Labs of the Evaluation Forum, 2024.
[38] A. Miyaguchi, M. Gustineli, A. Fischer, R. Lundqvist, Transfer learning with self-supervised vision
transformers for snake identification, in: Working Notes of CLEF 2024 - Conference and Labs of
the Evaluation Forum, 2024.
[39] M. Sieber, T. Železný, Do not lose to losses for snakeclef2024, in: Working Notes of CLEF 2024</p>
      <p>Conference and Labs of the Evaluation Forum, 2024.
[40] W. Yu, C. Si, P. Zhou, M. Luo, Y. Zhou, J. Feng, S. Yan, X. Wang, Metaformer baselines for vision,</p>
      <p>IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).
[41] H. Wei, R. Xie, H. Cheng, L. Feng, B. An, Y. Li, Mitigating neural network overconfidence with logit
normalization, in: International conference on machine learning, PMLR, 2022, pp. 23631–23644.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>I.</given-names>
            <surname>Bolon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Durso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. Botero</given-names>
            <surname>Mesa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Ray</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Alcoba</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Chappuis</surname>
          </string-name>
          , R. Ruiz de Castañeda,
          <article-title>Identifying the snake: First scoping review on practices of communities and healthcare providers confronted with snakebite across the world</article-title>
          ,
          <source>PLoS ONE 15</source>
          (
          <year>2020</year>
          )
          <article-title>e0229989</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>R.</given-names>
            <surname>Ruiz de Castañeda</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Durso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Ray</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. L.</given-names>
            <surname>Fernández</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. J.</given-names>
            <surname>Williams</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Alcoba</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Chappuis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Salathé</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Bolon</surname>
          </string-name>
          ,
          <article-title>Snakebite and snake identification: empowering neglected communities and health-care providers with ai</article-title>
          ,
          <source>The Lancet Digital Health</source>
          <volume>1</volume>
          (
          <year>2019</year>
          )
          <fpage>e202</fpage>
          -
          <lpage>e203</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Durso</surname>
          </string-name>
          , R. Ruiz de Castañeda,
          <string-name>
            <given-names>C.</given-names>
            <surname>Montalcini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. R.</given-names>
            <surname>Mondardini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. L.</given-names>
            <surname>Fernandez-Marques</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Grey</surname>
          </string-name>
          ,
          <string-name>
            <surname>M. M. Müller</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Uetz</surname>
            ,
            <given-names>B. M.</given-names>
          </string-name>
          <string-name>
            <surname>Marshall</surname>
            ,
            <given-names>R. J.</given-names>
          </string-name>
          <string-name>
            <surname>Gray</surname>
          </string-name>
          , et al.,
          <article-title>Citizen science and online data: Opportunities and challenges for snake ecology and action against snakebite</article-title>
          ,
          <source>Toxicon: X</source>
          <volume>9</volume>
          (
          <year>2021</year>
          )
          <fpage>100071</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>J.</given-names>
            <surname>Longbottom</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. M.</given-names>
            <surname>Shearer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Devine</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Alcoba</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Chappuis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. J.</given-names>
            <surname>Weiss</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. E.</given-names>
            <surname>Ray</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Ray</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. A.</given-names>
            <surname>Warrell</surname>
          </string-name>
          ,
          <string-name>
            <surname>R. R. de Castañeda</surname>
          </string-name>
          , et al.,
          <article-title>Vulnerability to snakebite envenoming: a global mapping of hotspots</article-title>
          ,
          <source>The Lancet</source>
          <volume>392</volume>
          (
          <year>2018</year>
          )
          <fpage>673</fpage>
          -
          <lpage>684</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>U.</given-names>
            <surname>Roll</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Feldman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Novosolov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Allison</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Bauer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Bernard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Böhm</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Castro-Herrera</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Chirio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Collen</surname>
          </string-name>
          , et al.,
          <article-title>The global distribution of tetrapods reveals a need for targeted reptile conservation</article-title>
          ,
          <source>Nature Ecology &amp; Evolution</source>
          <volume>1</volume>
          (
          <year>2017</year>
          )
          <fpage>1677</fpage>
          -
          <lpage>1682</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Durso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. K.</given-names>
            <surname>Moorthy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. P.</given-names>
            <surname>Mohanty</surname>
          </string-name>
          , I. Bolon,
          <string-name>
            <given-names>M.</given-names>
            <surname>Salathé</surname>
          </string-name>
          , R. Ruiz De Castañeda,
          <article-title>Supervised learning computer vision benchmark for snake species identification from photographs: Implications for herpetology and global health</article-title>
          ,
          <source>Frontiers in Artificial Intelligence</source>
          <volume>4</volume>
          (
          <year>2021</year>
          )
          <fpage>17</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.</given-names>
            <surname>Joly</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Picek</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Kahl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Goëau</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Espitalier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Botella</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Deneu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Marcos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Estopinan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Leblanc</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Larcher</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Šulc</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hrúz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Servajean</surname>
          </string-name>
          , et al.,
          <source>Overview of lifeclef 2024: Challenges</source>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>