This working note describes methods to automatically identify a large number of different bird species by their songs and calls. It focuses primarily on new techniques introduced for this year's task like advanced spectrogram segmentation and decision tree based feature selection during training. Considering the identification of dominant species, previous results of the LifeCLEF Bird Identification Task could be further improved by 29%, achieving a mean Average Precision of 59% (mAP). The proposed approach ranked second place among all participating teams and provided the best system to identify birds in soundscape recordings.
Automated acoustic methods of species identification can serve as a useful tool for
biodiversity assessments. Within the scope of the LifeCLEF 2016 Bird Identification
Task researchers are challenged to identify 999 different species in a large and highly
diverse set of audio files. The audio recordings forming the training and test data set
are built from the Xeno-canto collaborative database (www.xeno-canto.org). A
novelty in this year’s challenge is the enrichment of the test data set by including a new set
of soundscape recordings. These soundscapes are not targeting any specific species
during recording and can contain an arbitrary number of singing birds. To establish
reliable acoustic methods for assessing biodiversity it is essential to improve the
automated identification of birds in general but especially within these soundscape
recordings. An overview and further details about the Bird Identification Task are given
in [
Two main categories of features (the same as last year) were used for training and prediction: matching probabilities of species-specific 2D spectrogram segments (see 2.1 Segment-Probabilities) and acoustic features extracted with openSMILE (see 2.2 Parametric Acoustic Features). For this year’s task a large number of new SegmentProbability features were added for training by extracting new sound segments from audio files using the following two methods.
Re-segmentation of large segments. Using the automated segmentation method of
spectrograms described in [
Fig. 1. Spectrogram re-segmentation example (MediaID: 8). top: initial segmentation (large segments marked in yellow), bottom: via re-segmentation of large segments extracted additional segments in red With re-segmentation of previously segmented files 1,671,600 new segments were extracted and subsequently used for template matching to generate additional Segment-Probability features.
Extracting more features by segmenting files with low average precision. Besides re-segmentation of segmented files, additional files from the training set were chosen for segment extraction. However, instead of a random selection, a small number of files were chosen for each species (approx. 2 to 4) by selecting the ones having the lowest average precision score calculated during cross validation in previous training steps. This was done in two iterations (with new training and feature selection steps in between) increasing the number of features by additional 1,375,928 SegmentProbabilities. 2.1
For each species an individual feature set was formed by sweeping all segments
related to that particular species over the spectrogram representations of all training and
test recordings. The features were extracted via multiresolution template matching
followed by selecting the maxima of the normalized cross-correlation [
Besides Segment-Probabilities, for some models also parametric acoustic features
were used for prediction. To extract these features the openSMILE Feature Extractor
Tool [
The classification task was transformed to 999 one-vs-rest multi-label regression
tasks. This way the number of selected features could be optimized separately and
independently for each species during training. For each audio file in the training set
the target function was set to 1.0 for the dominant species and 0.5 for all background
species. Ensembles of randomized decision trees (ExtraTreesRegressor [
Fig. 2. Progress of AUC, mAP and R2 scores during feature selection for left: Scytalopus latrans (SpeciesID: lzezgo) and right: Psarocolius decumanus (SpeciesID: cxyhrl)
In Table 1 results of the submitted runs are summarized using two evaluation statistics: mean of the Area Under the Curve calculated per species and mean Average Precision on the public training and the private test sets. For all runs no external resources and only audio features (features extracted from audio files) were used for training and prediction. Run 1. For the best performing first run just a single model was used. This model was trained using only a small but highly optimized selection of Segment-Probabilities (as described in the previous section). For this run, features were selected per species by optimizing the mAP score on the training set. A total of 125,402 features (with a minimum of 20, a maximum of 1833 and an average of 126 features per species) were used to predict all species in the test files.
Run 2. The second run was submitted quite early as an interim result and is therefore
not worthy of being discussed here. It was actually supposed to be replaced by the
submission of another run averaging the predictions of several different models.
Unfortunately uploading could not be completed before the submission deadline.
Run 3. For the third submitted run blending of different models followed by
postprocessing was used as described in [
Figure 3 visualizes the official scores of the LifeCLEF 2016 Bird Identification Task.
The here proposed approach ranked second place among all teams (MarioTsaBerlin
Run 1 & 4) and provided the best system to identify birds in soundscape recordings.
Interestingly, for the best performing submission (Run 1) just a single model was used
with only one category of features. Although using a good selection of features for
this model one would expect that blending several models with different feature sets
would perform better than just a single one. One possible explanation for the
comparatively weak results achieved with blending could be the inclusion of the best
combined models from 2015. Those combined and post-processed predictions already
showed a fairly high overfitting on the training set and blending was perhaps done in
favor of these predictions at the cost of the maybe better generalizing new models.
On the other hand achieving an improvement by almost 30% on the mAP score with a
single model (25% if taking background species into account) clearly shows that the
techniques introduced this year could be applied very successfully. They also seem to
complement each other quite well. Extracting additional, fine-grained spectrogram
segments for template matching by re-segmenting larger segments captures typical
sub-elements of songs or call sequences. Matching these sub-elements can give better
identification performance than matching larger song structures, especially if those
show a high variability between different individuals of the same species. The
downside of the new segmentation method is a collection of many redundant or even
useless segments e.g. when dealing with noisy recordings or overlapping sounds from
other species or sources. However, the proposed feature selection method can
compensate for that by successively discarding irrelevant features during training.
This year also deep learning techniques were successfully applied to the BirdCLEF
dataset [
Fig. 33. Rufous-collared Sparrow / Zonotrichia capensis, (ID: yyjrms) Acknowledgments. I would like to thank Hervé Glotin, Hervé Goëau, Willem-Pier Vellinga, Alexis Joly and Henning Müller for organizing this task and the XenoCanto foundation for nature sounds as well as the French projects Floris'Tic (INRIA, CIRAD, Tela Botanica) and SABIOD Mastodons for their support. I also want to thank the BMUB (Bundesministerium für Umwelt, Naturschutz, Bau und Reaktorsicherheit), the Museum für Naturkunde and especially Dr. Karl-Heinz Frommolt for supporting my research and Wolfram Fritzsch for providing me with additional hardware resources.