The BirdCLEF+ 2025 challenge requires classifying 206 species, including birds, mammals, insects, and amphibians, from soundscape recordings under a strict 90-minute CPU-only inference deadline, making many state-of-the-art deep learning approaches impractical. To address this constraint, the DS@GT BirdCLEF team explored two strategies. First, we establish competitive baselines by optimizing pre-trained models from the Bioacoustics Model Zoo for CPU inference. Using TFLite, we achieved a nearly 10x inference speedup for the Perch model, enabling it to run in approximately 16 minutes and achieve a final ROC-AUC score of 0.729 on the public leaderboard post-competition and 0.711 on the private leaderboard. The best model from the zoo was BirdSetEficientNetB1, with a public score of 0.810 and a private score of 0.778. Second, we introduce a novel, lightweight pipeline named Spectrogram Token Skip-Gram (STSG) that treats bioacoustics as a sequence modeling task. This method converts audio into discrete "spectrogram tokens" by clustering Mel-spectrograms using Faiss K-means and then learns high-quality contextual embeddings for these tokens in an unsupervised manner with a Word2Vec skip-gram model. For classification, embeddings within a 5-second window are averaged and passed to a linear model. With a projected inference time of 6 minutes for a 700-minute test set, the STSG approach achieved a ifnal ROC-AUC public score of 0.559 and a private score of 0.520, demonstrating the viability of fast tokenization approaches with static embeddings for bioacoustic classification. Supporting code for this paper can be found at https://github.com/dsgt-arc/birdclef-2025.
The BirdCLEF+ 2025 challenge in the LifeCLEF Lab involves classifying fauna in the Middle Magdalena
Valley of Colombia [
One important limitation of solutions is that they must fit within a 90-minute inference deadline on a CPU-only instance provided on Kaggle. The computational constraint discourages solutions that rely on gross computation to reach the top of the leaderboard through mechanisms like model ensembling.
We address the BirdCLEF+ challenge in two parts. First, we provide baseline transfer-learning
solutions by training a classification head on pre-trained birdcall classification models. We hypothesize
that the representation space of existing bioacoustic models will transfer well to the domain-shifted
dataset. Existing research has shown that birdcall classifiers, such as BirdNET and Perch, handle domain
shifts well [
We then experiment with a technique that utilizes discrete audio tokens for soundscape classification, which we refer to as Spectrogram Token Skip-Gram (STSG). First, we convert audio into discrete tokens derived from Mel-spectrograms. We then contextualize tokens into continuous space via skip-gram embeddings and average tokens within prediction intervals to build a classification head, ofering some reasonable defaults for the hyperparameters involved. Finally, we experiment with pretraining a classification model on a surrogate task using a student-teacher model on unlabeled training soundscapes, leveraging predictions from a strong bioacoustical model.
The dominant strategy for bioacoustic classification, particularly in BirdCLEF, involves treating audio as
a computer vision problem and applying CNN ensembles to Mel-spectrograms, 2D time-frequency
representations of audio signals [
In the broader field of audio processing, many newer models represent audio as compact sequences of
discrete tokens. This allows the use of powerful sequence modeling primarily used in the text domain,
including LLMs. These tokens can be divided into two types: acoustic tokens and semantic tokens.
Neural audio codecs, such as SoundStream [
The word2vec skip-gram model is a simple method for learning dense vector embeddings from
large text corpora [
We split our methods into two main parts. We explore transfer learning in depth for the BirdCLEF 2025+ dataset using a variety of pre-trained soundscape backbones. Here, we establish baseline classification and computational performance characteristics on state-of-the-art models and determine the engineering work required to deploy them to resource-constrained systems. We then explore our central hypothesis that a tokenized representation of spectrograms can be run with marginal classification degradation while reducing inference computation by an order of magnitude.
We build a baseline for the competition by utilizing pre-trained backbones for bioacoustic fauna
classification. The process of building a new classification head transfers the knowledge of the backbone
to a new task, efectively reutilizing the representation space learned from the original task. A
domainspecific classifier for birds like Perch will be able to tightly cluster sounds that belong to specific classes
like those in Figure 2. These properties make pre-trained backbones desirable for similar domains.
We utilize the Bioacoustic Model Zoo to source our backbones, which serve as a companion to the
OpenSoundscape project [
A large variety of models are available from the Bioacoustics Model Zoo, but they cannot be run directly in the competition for various reasons. First, additional parameters and nodes in the computation graph needed for training can be pruned, fused, and optimized for inference. We achieve an order of magnitude speedup for Perch when using TFLite over TensorFlow, enabling the model to run within competition time limits.
The other source of contention is the impedance mismatch between the clip length of models and the clip length expected by the competition. The BirdNET model exemplifies this issue since it is lightweight enough to deploy as-is but requires adaptation for the competition. In cases like this, we apply a sliding window over the 5-second target frame and extract embeddings from each window. These embeddings can be aggregated by directly averaging them and applying a classification head or by running the classifier on each window individually and averaging the resulting probabilities.
To simplify model training, we first train the classifier on the original window embeddings. During inference, we average the window embeddings over the target frame and run the classifier on the average embedding. A window is within the target frame if it overlaps with it by at least 50% of the window size, ensuring that each window contributes to only one target frame. We plan to explore other aggregation methods in the future, such as max pooling; however, averaging works suficiently well as a baseline.
The requirements for the Spectrogram Token Skip-Gram (STSG) embeddings are computationally eficient transformations that map raw waveforms into discrete token space and tokens into continuous embedding space. One advantage of this representation is that we retain the sequential nature of audio, allowing us to contextualize a discrete token of audio with the surrounding tokens in time. We approach this problem in three parts, as shown in Figure 3, specifically through tokenization, embedding, and model pre-training.
We choose a simple tokenizing scheme and sequence model to maximize inference throughput. Our
tokenizer clusters Mel-spectrograms across the training soundscape using K-means to generate a
codebook for discrete representation [
We set the spectrogram parameters in Table 2a in such a way that we obtain eight spectrogram frames per second; in other words, each resulting token will represent 0.125 seconds of context from the original spectrogram. This value is chosen as a power of two for eficiency and is selected to yield a total sequence length of 40, corresponding to a 5-second prediction frame. We can increase the tokens per second to increase our target sequence length but at the expense of losing temporal relevance at the spectrogram level. We use a total of 768 Mel-frequency bands for testing. This number is less than 10% of the window size, which helps reduce the number of zero-valued frequency bands.
Once we choose parameters for the Mel-spectrograms, we compute them over the entire
training soundscape dataset. We perform principal component analysis (PCA) on the normalized
Melspectrogram vectors to reduce their dimensionality for K-means and retrieval. Normalizing the
spectrogram vector helps reduce the efects of decibel intensities and instead focus on the distribution of the
audio spectra. PCA will project the spectrogram frames into a low-rank space, which helps clustering
by reducing high-frequency noise and the overall computational and memory requirements. We run
K-means via Faiss over the frames to quantize them into integer tokens [
To obtain a new token, we generate the spectrogram frame for a given time interval, normalize it, and project it into a low-rank space. We then find the closest K-means vector that corresponds to it using the L2 distance. The distance can be eficiently estimated through approximate nearest neighbor algorithms, such as those implemented in Faiss, including hierarchical navigable small worlds (HNSW).
Once we have discrete tokens from the audio, we learn a model that embeds the token into a continuous space, which we can use for downstream classification tasks. Our soundscape data lacks proper labels that we can utilize, but we can still learn about the relationship between tokens because audio is naturally sequential. We achieve this in an unsupervised manner using a skip-gram negative-sampling embedding model, where we learn the location of a token in a high-dimensional space that captures temporal and semantic relevance within a sequential window.
We use the Word2Vec implementation in gensim to obtain a static lookup table for a learned skip-gram
embedding model [
∑︁ log( (− ⃗ · ⃗,)) =1
The negative samples are drawn from a unigram distribution based on token frequency () smoothed by hyperparameter (the ns_exponent) normalized over the corpus vocabulary: () () = ∑︀′ (′)
The value of is a critical, task-dependent hyperparameter. = 0.75 is standard for NLP tasks. A
higher exponent exaggerates the sampling of frequent noise tokens, forcing the model to become highly
discriminative between rare signal tokens and common background sounds. In contrast, a negative
would make the model focus on distinguishing rare signals from each other [
We train a classification head on the resulting embeddings. An embedding-centric workflow ensures that discrete token-based methods are comparable against vision methods over spectrograms.
In addition to learning a continuous representation of the spectrogram tokens, we experimented with student-teacher modeling to see if we could impart the bioacoustical domain knowledge from Perch into a smaller student model based on the STSG embedding space. We configure the student with a 1D-CNN to aggregate STSG embeddings, project these into a latent space, and then attempt to match the probability distribution of Perch using KL-divergence as the loss. We apply temperature scaling to the outputs such that teacher probabilities are softened by a squared factor of temperature = 3.
We use the student model from Table 3 and train it against 80% of the training soundscape, using the remaining 20% as validation. Then, we can pre-compute embeddings for the training species dataset, just as we do for other models in the bioacoustics model zoo. In our modeling, we rely on complete sequences, i.e., masking is not implemented.
To validate our transfer learning and token embedding models, we require a task that reflects the characteristics of the competition task. We rely on the train species dataset in order to validate our models. The surrogate task serves as a proxy for multi-label inference on soundscapes, for which we lack labeled data to work. The training data skews in favor of commonly appearing species that are easier for individual people to crowd-source recordings. We pre-compute vector representations of the audio aligned to the step size of the original classifier. We drop species that have fewer than two samples, then stratify the dataset into test and validation in an 80/20 split. We train a classification head on the embeddings with a hidden layer, non-linearity, and then cross-entropy on a linear projection to the output space. We document the architecture of the classification head in Table 4.
We select a smaller semi-representative set of species to facilitate hyperparameter tuning sweeps over the STSG embeddings. In Table 5, we use Gemini 2.5 Pro (gemini-2.5-pro-preview-05-06 with Gemini App system prompting) to select species using provided taxonomy information, with explicit prompting about the region and competition to ensure mammals, insects, and amphibians have representation. We validate the number of intervals processed by Perch and note the distribution across our small validation set.
The competition metric is a macro-averaged ROC-AUC that skips classes with no true positive labels. We use the macro-averaged ROC-AUC as a rough proxy for our experiments. This measure often fails to provide precise measurements, as it frequently remains near 1.0 in transfer learning experiments. We will also report the F1-macro score, which enables more precise observations between models with diferent hyperparameters.
The STSG training process has a large number of parameters that can afect the output results. For the sake of time, we select a set of hyperparameters that sufice for validating that the pipeline works in both training and inference and that it yields improvements over baseline methods. Refer to Table 2a for the set of default parameters.
Our first attempt at modeling used Mel-Frequency Cepstral Coeficients (MFCCs), which are efectively the discrete cosine transform (DCT) of the Mel-spectrogram. We denote this STSG v1, with hyperparameter plots in Appendix A. However, we artificially limit the representational power of the tokens in v1 by being overly conservative with the parameters of 128 Mel bands and 20 MFCCs. In Table 6, we describe the parameters for the v2 line of models that use the Mel-spectrogram representation of 768 Mel-bands. The remaining hyperparameter results that follow utilize the Mel-spectrogram representation.
Given a spectrogram representation, we must determine how many centroids we compute over the entire soundscape dataset. The centroids act as a data-driven vocabulary for the audio corpus. In Figure 4, we sweep over the parameters of the vocabulary. We settle on a token set size of 16k, which both fits inside a 2-byte integer and provides a reasonable amount of support, given that there are 3.73 million tokens in the 80% training split of the soundscape. While increasing the number of tokens to 32k improves the validation F1-macro scores, it also increases the overall required training time for the embedding proportionally.
Applying PCA to the tokenizer is also a net positive to the model. We choose 128 dimensions, as shown in Table 7, which retains 87% of the original variance of the 768 Mel-spectrogram bands. Reducing the number of dimensions increases modeling performance, likely due to the omission of high-frequency noise and latent modeling of sparse substructures in the original data. Additionally, many of the algorithms that we use (K-means and approximate K-NN) have algorithmic complexity that is linearly proportional to the number of dimensions. Reducing the number of dimensions allows us to fit more rows into memory and execute fewer instructions overall when tokenizing the signal.
Once we have determined our token size, we conduct hyperparameter sweeps over the Gensim Word2Vec model, as shown in Table 8. We vary over vector size, window size, negative sampling exponent, and sample rate. The vector size determines the adequate capacity of the model and sweeps over 128, 256, 384, 512, and 1028 dimensions. The dimension size is the most important parameter, as it directly afects the model’s ability to learn complex relationships between tokens. Increasing the vector size by a factor of three increases the F1-macro score by 0.018 but also increases the training time by a factor of 1.7. We find that lowering the window size from the default of 80 decreases the score by 0.013, whereas increasing the window size does not significantly improve the score. However, the training time is significantly afected by the window size, with a smaller window size reducing the training time by 112 seconds.
We find that sweeping over the negative sampling exponent does not yield a clear pattern; however, in this particular sweep, setting to 0.0 yields the best performance. This parameter controls the distribution of negative samples, and a setting of 0.0 indicates a uniform distribution over the vocabulary. A positive exponent means negative samples are more likely to be drawn from the more frequent tokens, while a negative exponent means negative samples are more likely to be drawn from the less frequent tokens. Finally, we sweep over the subsampling rate, which controls the frequency of token sampling during training. We find that decreasing the sample rate to 1e-4 does not change the score but increases the training time by a factor of two. Increasing the sample rate to 1e-6 results in a significant drop in performance, as only the most frequent tokens are sampled, and the model is unable to learn from the less frequent ones. These results help motivate the final choice of hyperparameters, which are balanced to provide a good trade-of between performance and training time.
(a) F1 score vs. ns_exponent. (b) Word2Vec validation curves
Given the hyperparameter sweeps, we then fix over the parameters and run over the negative sampling exponent . In our parameter scans, it was unclear how the negative sampling exponent and sample sizes interact with each other. However, we find that when we set the sample to 1e-5, then, the best is 0.0. When we increase the number of samples by setting the sampling threshold to 1e-4, the best decreases to -0.75. When we aggressively subsample tokens in the former setting, we remove many of the most frequent tokens, which likely correspond to silence or background noise, making the resulting token distribution flatter. The uniform strategy is efective since the noise is already downsampled. When we are less aggressive with the samples, the distribution more closely resembles the original distribution. Setting this to a negative value then forces the model to learn how to distinguish between tokens corresponding to diferent species rather than focusing on the frequent tokens.
We also look at the performance over time in Figure 5b and find that 100 epochs are a reasonable length of time to run the Word2Vec algorithm. In the first two iterations (v2.0 and v2.1), where we set subsampling to 1e-5, we observe either overfitting or significant stochasticity in the validation curve over time. We increase the number of tokens available for training in v2.2, and this stabilizes training to a large degree.
We report the results of the STSG student model in Table 9 for the validation task, comparing it to the Perch teacher model and our best token-based model. These initial results show that the teacher model is competent, achieving a macro F1 score of 0.80 and a micro F1 score of 0.91. The Mel-spectrogram, used directly by averaging the features over the time step, achieves a score of 0.12. The STSG model performs better than the spectrogram baseline but worse than the teacher, with a score of 0.56. The token model can learn a representation space of the soundscape that can be transferred to the training dataset to a lower degree than the Perch model but is significantly better than choosing at random.
Finally, we observe that the student model, trained on the STSG representation space and then ifne-tuned on the surrogate task, achieves a macro F1 score of 0.47. The student model’s performance represents a significant regression compared to using the STSG embeddings directly, suggesting that the STSG embeddings are unable to capture the complex relationships needed to represent the Perch logits efectively and that the geometry of the learned embedding space does not transfer well to classifying the training species dataset.
We report the transfer learning surrogate task in Table 10. Here, we note that the ROC-AUC score does not help us disambiguate between the relative performance of models. We note that the F1-macro score correlates well with the ranked performance of the models while providing larger diferences in scores for individual observations. The metric is also built into the classification report in the scikit-learn library. We find that our best STSG models yield results around 0.56, whereas the best computer vision backbone models achieve results around 0.87.
We report the final model performance and timing on the leaderboard in Table 11. We note that the primary metric that we are looking at here is the modified ROC-AUC scores, which are much lower than the ROC-AUC metrics in our surrogate task. We note that the bioacoustic model zoo performs well for this simple transfer learning task without requiring complex modeling to address the domain shift in the datasets. In comparison, the token embedding models achieve a modest score of around 0.56.
However, the STSG model is lightweight and fast. It is three times as fast (0.5 seconds per file vs. 1.4 seconds per file) when compared to Perch on TFLite. The Word2Vec modeling is weak but can be replaced by more robust and complex token embeddings without altering the runtime characteristics as long as the token embeddings are static. Additionally, we found that many of the Torch-based bioacoustical models did not require any compilation or optimization to run within the competition bounds. A significant amount of tuning can be done despite the computational constraints, as evident from the final leaderboard for the 2025 competition.
While the transfer learning experiments we have run are methodologically simple, we were able to test a much wider variety of models than in previous years. The bioacoustic model zoo is an excellent library for development, as it provides a reasonable API into many popular bioacoustic backbones. We visualize the embedding space extracted from the library in Figure 6, which provides a qualitative view of the geometric structure of the backbones. The tight clustering behavior of BirdNET, Perch, and BirdSet models is desirable, as it facilitates more straightforward mapping to our final objective space. The color of the data corresponds to the one-dimensional projection of the embedding in BirdNET, which efectively ranks data points on a number line. We expect to see a smooth transition in color based on distance in two dimensions. The discoloration in RanaSierraCNN reflects the leaderboard performance and the decrease in performance due to ambiguity between similar points. HawkEars is a model that we were unable to test on the leaderboard due to its ensembling structure; however, the domain-specific nature of Hawks leads to a more substantial domain shift, also observed in the Rana Sierra Frog classifier. We also note that we did not test YAMNet due to the unusual clip length, although it may be helpful as a model for filtering.
BirdNET and Perch continue to be staple models that provide state-of-the-art performance with
broad reach in real deployments. The BirdSet dataset has made it much easier to develop new backbone
models that rival the performance of BirdNET and Perch while staying in the Torch ecosystem and
with prediction windows corresponding to the BirdCLEF challenge [
(b) BirdSetConvNeXT.
(c) BirdSetEficientNetB1. (d) HawkEars.
(e) Perch.
(f) RanaSierraeCNN.
In our experiment, we tested the ability to model bioacoustic data as discrete sequences and to determine if it could serve as a basis for student-teacher modeling. We discovered a fast and lightweight token representation for our data that enables unsupervised modeling despite its limited discriminative power. Spectrogram tokens are an interesting result because they rely on algorithmic building blocks of clustering and nearest-neighbor search to represent the sequence sparsely. The information bottleneck in our process lies in the signal representation given by the spectrogram. When we increased the total number of Mel-bands and avoided dimensionality reduction early in the pipeline, our validation scores increased by approximately 0.1 in F1-macro. A better representation leads to improved tokenization, enabling the sequence model to better represent the audio for classification purposes.
The idea of quantizing audio for downstream applications is not unique to this work. There are
data-driven neural codecs for compressing and decompressing audio, such as EnCodec [
Future work will place greater emphasis on modeling complex sequences. In particular, we would like to see better token representations and the use of Transformers to contextualize sequences of tokens. It would be interesting to see how much of the wav2vec framework applies to BirdCLEF and to see whether contextualized tokens via Transformers can run on constrained systems.
While the STSG representation is fast, it leaves significant room for improvement. We can use up most of the inference budget to calculate high-quality tokens, assuming we apply a static lookup table with a lightweight classification head. The first direction is to replace the spectrogram token with a YAMNet-based token, which, although resulting in a shorter 2-token-per-second sequence, provides a more powerful representation that will form more meaningful clusters under K-means. Another approach might be to use a clustering algorithm, such as HDBSCAN, to identify centroids that are better suited to the topology. However, we lose the performance characteristics from having a highly optimized ANN algorithm in -space. The final direction is to learn tokens using wav2vec and to optimize the framework for CPU inference heavily. Wav2vec is promising, although it requires a significant amount of engineering work to execute.
One of the factors that make transformer-based models challenging to use in the BirdCLEF+
competition is that inference is expensive with naive implementations due to the sheer number of parameters
involved. However, transformers are a natural way to represent sequential data because attention
eficiently contextualizes tokens within sequences [
Through this work, we address computational limitations in bioacoustical classification by leveraging transfer learning of domain-specific classifiers and a lightweight token-based approach. We demonstrate state-of-the-art models and their performance on the domain-shifted data through models like BirdNET and Perch in the bioacoustic model zoo. Our best model was BirdSetEficientNetB1, with a private ROC-AUC score of 0.778 and a public score of 0.810. We also investigate the foundations of distilling complex sequence models into a static lookup of contextualized tokens. Our Spectrogram Token SkipGram (STSG) pipeline achieves a runtime of 6 minutes on the test dataset, yielding a final ROC-AUC private score of 0.520 and a public score of 0.559. The STSG pipeline provides the basis for more complex sequence models, as it enables the discovery of a representational space that can be parsed and transformed eficiently on a CPU. Although the overall performance is low, there are interesting directions that this research can take, given that spectrogram tokens possess representational power.
Supporting code for this paper is located at https://github.com/dsgt-arc/birdclef-2025.
We thank the Data Science at Georgia Tech (DS@GT) CLEF competition group for their support. This
research was supported in part through research cyberinfrastructure resources and services provided by
the Partnership for an Advanced Computing Environment (PACE) at the Georgia Institute of Technology,
Atlanta, Georgia, USA [
During the preparation of this work, the author(s) used Gemini Pro in order to: Abstract drafting, formatting assistance, grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.
The first version of STSG uses a spectrogram with 128 Mel bands and 20 MFCCs. +0 Baseline Varying vector_size Varying window Varying ns_exponent Varying sample Combination window window sample sample window ns_exponent -0.043 +0.025 +0.021 +0.071 +0.002 +0.003 +0.016 -0.023 -0.001 -0.045 -0.019 +363