Detection of bird species with bird songs is a challenging and meaningful task. Two scenarios are presented in BirdCLEF challenge this year, which are monophone and soundscape. We trained convolutional neural network with both spectrograms extracted from recordings and additionally provided metadata. Focusing on the soundscape situation, we applied bird event detection to reduce false alarm. Besides, we rescored the retrievals using masks which are designed for all species being identi ed. In addition, context information was also taken into consideration in our system. Our system was evaluated in BirdCLEF 2018 and we achieved an o cial mean average precision (MAP) score of 0.6548 for monophone classi cation without background bird songs and 0.5882 for identi cation with background bird songs. For soundscape, we achieved 0.1196 in classi cation mean average precision (C-MAP).
BirdCLEF challenge is hosted by the LifeCLEF lab [
The training features of our model mainly consist of two parts. The original
part is the frequency information of each recording and the additional part is
the metadata [
Above is the method of our model training. Based on our model, we made
some improvements focusing on the problem of soundscape in the test period.
Firstly, a simple bird event detection [
Pytorch was used for our model training and evaluation. 2
We transform the problem of bird songs' classi cation to image classi cation.
Each 5 seconds' segment of given audios is turned into a spectrogram with the
resolution of 512 256 pixels [
Data augmentation is necessary for building robust models and handling data imbalance. Adding noises is a commonly used data augmentation method. We try to add two kinds of noises to spectrograms. For each epoch of training, 10 percent of data are added Gaussian noises and 10 percent are added noise samples.
Gaussian noises: Gaussian noises [
Noise samples: Besides Gaussian noises, noise samples are also considered
and added to our spectrograms. Noises of audios recorded by similar equipment
under similar environments often share some common patterns. Adding similar
noises will help improve the performance. During data processing, we have
obtained many spectrograms which are thought to be noise samples [
Researchers [
We use a relatively shallow architecture of convolutional neural network as
our basic model [
As the problem can be viewed as a multi-class identi cation problem, cross entropy loss is used here to be minimized. We use Adam [15] as our optimizer. Adam optimizer can be regarded as RMSprops [16] with momentum, which makes the best use of the rst moment and the second moment of the gradient. Parameters can be updated more stably using it.
Learning rate decay technique [17] is used in our training process. At the very beginning, learning rate is set to 0.0001. After nearly 15 epochs of training, it is lowered to 0.00001 in order to optimize the updating. We stop the process when the accuracy converges.
Above we mention that metadata is also used for training in our system.
Spectrograms are attened to a vector of 512 elements by our convolutional
neural network. We construct an additional fully connected layer for metadata
[
In the competition of last year showed that the performances on soundscape still had a large room for improvement. The performance of model has a great impact on the nal result. While for the limitation of hardware resources and time, we did not lay stress on the model training. Instead, we tried to nd out methods that make the best use of our current models. Several methods we applied will be introduced below.
Bird event detection: False alarm of target species will in uence the metric
of C-MAP. Introduction of bird event detection [19] is able to reduce false alarm
and improve the nal performance. At the very beginning, we planned to use
the soundscape evaluation set to train a neural network. While for the limitation
of labeled data, performance was not good enough for use. At last, we directly
used the method mentioned above [
Masking and rescoring: For birds belonging to a speci c species, the
frequency of their songs always falls in a certain range. Outside this range of
frequency, any other information including environmental sound or songs of other
kinds of birds can be considered as noises. Inspired by this idea, we designed
a mask for each kind of birds. We accumulated spectrograms of a species on
the frequency axis and normalized it. The range of values under 0.6 would be
masked. Here, we consider 0.6 a relatively proper threshold. The masks for all
birds being classi ed can be viewed as band-pass lters [
Considering context: We found that, at most of the time, a bird song often lasts for a period of time more than 5 seconds. For a 5 seconds' segment in soundscape, the nal result is strongly relevant to the result of previous and next 5 seconds' segments. This context information is considered in monophone scenario by overlapping while seldom considered in soundscape. Here, we simply added the outputs of the previous and next 5 seconds' segments to current output with a given weight which can be 0.2 or 0.3 and so on. Here, we set this value to 0.3 which we found that it resulted in a relatively better result in validation set. By this method, we took the context into consideration of classi cation. We totally trained 4 models for our classi cation task. Methods of data augmentation and addition of metadata are introduced above. Besides the basic convolutional neural network, we also trained a Resnet for further improving the nal fused results.
1. ConvNet with Data augmentation without metadata addition; 2. ConvNet with Metadata addition without data augmentation; 3. ConvNet with Data augmentation and metadata addition; 4. Resnet with Data augmentation without metadata addition.
This year, a labeled soundscape's evaluation set is given. We are able to test our improvemnts with it. Model 3 is used to test the e ect of our methods. From table 1, we can see that masking and rescoring method as well as context considering can improve C-MAP.
From table 2, we can nd that with increasing of the fused systems, the performance is getting better. As expected, system of run5 has the highest scores on MAP without background species among our submissions.
Soundscape scenario:
DKU SMIIP run1: The output of model 3; DKU SMIIP run2: Fusion of model 2 and 3; DKU SMIIP run3: Fusion of model 1, 2 and 3; DKU SMIIP run4: Fusion of model 1, 2, 3, 4.
In this competition, there are two scenarios, monophone and soundscape. We trained models using the convolutional neural network with bird songs' spectrograms. Besides the regular model training, we made data augmentation to improve the robustness. We also added metadata to further improve the performance.
Focusing on soundscape scenario, we made some improvements based on our current models in the test period. Firstly, bird event detection was introduced to reduce false alarm. Secondly, masks were designed for each kind of birds. Rescoring is done on the top 3 or 5 of sorted bird species list after being masked. Thirdly, context is considered by adding outputs of previous and next 5 seconds' segments to current output.
Above methods still have many spaces for improvement. Bird event detection [19] can be done using neural network models if enough labeled data provided. Bandpass lters of birds can be more delicate. In our work, context information is considered using a relatively simple method. During the evaluation, we found that this kind of information can obviously improve the performance. Further investigations need to be done in this direction.
In addition, due to the lack of hardware resources and time, performances of our basic models still have room for improvement. Further, more model structures and fusion methods will be explored. 12. Io e, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167. 13. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from over tting. The Journal of Machine Learning Research, 15(1), 1929-1958. 14. Clevert, D. A., Unterthiner, T., & Hochreiter, S. (2015). Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289. 15. Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization.
arXiv preprint arXiv:1412.6980. 16. Tieleman, T., & Hinton, G. (2012). Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2), 26-31. 17. Zeiler, M. D. (2012). ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. 18. Hinton, G. E., & Salakhutdinov, R. R. (2009). Replicated softmax: an undirected topic model. In: Advances in neural information processing systems (pp. 16071614). 19. Stowell, D., Wood, M., Stylianou, Y., & Glotin, H. (2016). Bird detection in audio: a survey and a challenge. In: Machine Learning for Signal Processing (MLSP), 2016 IEEE 26th International Workshop on (pp. 1-6). IEEE.