In this paper, we present an Xception based method for bird sound recognition of BirdCLEF2020. The goal of BirdCLEF2020 is to detect and classify 960 bird species within the provided soundscape recordings, it is more complex to di erentiate such a large number of birds than BirdCLEF2019. In our approach, logmel or loglinear spectrograms are extracted as features, and some data augmentation techniques are utilized to improve the performance of detecting the bird sounds. Finally, we evaluate our system on BirdCLEF2020 test dataset and achieve a classi cation mean average precision (c-mAP) score of 0.0421.
It is di cult to take clear photos of birds, which requires an open environment, professional equipment, high level of photography, and it takes a lot of time to actively look for the objects to be photographed. Instead, we can identify a bird's specie through a segment of its voice. Hearing the sound and distinguishing birds is important for many environmental and scienti c purposes. Some successful techniques will be used in monitoring of ecological environment in the future. Birds are highly sensitive to the environment. If some areas are polluted, some birds will gradually y away. Therefore, the changes of bird habits and population can re ect the changes of the environment. If some microphones are installed in the forests, the information of bird population and activities can be collected, which can signi cantly improve the e ciency and accuracy compared with manual observation. The sounds of di erent birds are indeed speci c. A large number of sound data can be collected by installing speci c recording equipment. This can be done automatically in an unattended environment, without the need for people to invest extra time and money.
With the development of deep learning, a large body of research in sound
classi cation is proven to outperform traditional methods in bird sound classi
cation [
The training data consists of audio recordings for bird species from South and North America and Europe. The Xeno-canto community contributes this data and provides more than 70,000 high-quality recordings across 960 species to this year's challenge. Each recording is accompanied by metadata containing information on recording location, date and other high-level descriptions provided by the recordists.
The test data consists of 153 soundscapes recorded in Peru, the USA, and
Germany. Each soundscape is of ten-minute duration and contains high
quantities of (overlapping) bird vocalizations [
The number of recordings for each specie are calculated, although the training dataset contains over 70,000 recordings, there are less than 90 recordings in some species, the rest species are between 90 and 100 recordings. For example, the bird specie with a ebird name of "banfru1" only contains 1 recording, but there are 100 recordings in another specie "whcspa". The number distribution of the dataset is shown in Fig.1. The imbalance of the dataset can have e ect on recognizing bird species.
To separate bird sound and background noise from a original recording, we apply
dilation, erosion and smooth masking. Similar techniques are presented in [11]
and used in [
In recent years, some data augmentation techniques are successfully applied in sound recognition tasks. Two data augmentation metheds are utilized in our approach.
Some time and frequency augmentation methods have been used in [
To handle the imbalanced dataset and prevent over tting, we apply mix-up during training stage [13]. It can be expressed as: x~ = y~ = xi + (1 yi + (1 )xj )yj where xi; xj are input features, yi; yj are target labels, 2 [0; 1] is a random number drawn from the Beta (c; a) distribution. We can get more training samples without extra computing resource, new samples are linear interpolated of real samples. 3.3
All the bird sounds come from diverse area around the world with special environmental background noise. In bird sound recordings, loud raining sound, (1) (2) quiet background, sound of wind and sound of other animals may exist and even cover some bird sounds. To eliminate this, we propose a denoising method to do spectral subtraction on a spectrogram: { Load a 5-second sound le and generate an amplitude spectrogram. { Calculate mean amplitude values over frames, mark the minimum 20 frames and calculate the mean amplitude values of these frames, we can get a primary subtraction vector. Then subtract the primary subtraction vector over frequencies for each frame, we can get the nal denoised spectrogram. 3.4
During the training, separated bird sing, augmented bird sound and spectral subtraction sound with logmel and loglinear spectrograms of 128 bands are generated as input. All the spectrograms are calculated by logarithm. 4 4.1
Inception-v3 is one of the state of art architectures in image classi cation
challenge [12]. And it is con rmed that Inception-based CNNs on Mel spectrograms
provide the best performance [
Xception is a improvement of Inception-v3 proposed by Google. In [
To recognize 960 species and handle such a large amount of recordings, we used Xception architecture instead of other CNN architectures. As for features, we selected logmel and loglinear spectrogram as input. We applied a denoising method and a data augmentation method during the data preprocessing. Pytorch was implemented to train model, and python librosa library was applied to process recordings and generate features.
During the training, categorical cross entropy was used as loss function and stochastic gradient descent was used as optimizer with weight decay of 1e-4 and a constant learning rate of 0.001.
The evaluation metric is the classi cation mean Average Precision (c-mAP), considering each class c of the ground truth as a query. This means that for each class c, all predictions are extracted from the run le with ClassId(c), rank them by decreasing probability and compute the average precision for that class, which can be expressed as c mAP =
PC c 1 AveP (c)
C AveP (c) =
Pn k=1 P (k) nrel(c) rel(k) where C is the number of species in the ground truth and AveP (c) is the average precision for a given species c computed as: where k is the rank of an item in the list of the predicted segments containing c, n is the total number of predicted segments containing c, P (k) is the precision at cut-o k in the list, rel(k) is an indicator function equaling 1 if the segment at rank k is a relevant one (i.e. is labeled as containing c in the ground truth) and nrel is the total number of relevant segments for c. 5.2
For each validation or test le, we smoothed the prediction vector by applying a moving window size of 5 seconds. Then we only selected rst 3 maximum probabilities as the result of a 5-second test fragment.
To achieve the performance of our methods, we only trained 76 bird species included in the validation dataset, the best c-mAP scores of the experiments are shown in Table 1, data augmentation with adding sounds randomly is denoted as AR, spectral subtraction is denoted as Spec sub. Experiment 1 and 2 only used the extracted bird sing to generate features, the rest of experiments extracted features Continuously from original recordings.
From Table 1 we can see, logmel with mix-up method achieves the best c-mAP score, loglinear performs well with separated bird sings and data augmentaion with randomly adding. 5.3
Four teams submitted 29 submissions in LifeCLEF 2020 Bird Monophone.
Finally we got the 2th rank among the teams and achieved a best c-mAP score of
0.0421 and r-mAP of 0.0671. Details are shown in Table 2 [
In this paper, we proposed a system for bird recognition based on Xception with some data augmentation and denoising techniques. Finally, we got a c-mAP score of 0.0421 on o cial test dataset. To handle more than 70,000 recordings, Xception was chosen because of its great feature extraction ability. During training, mix-up and randomly adding data augmentation methods were applied to prevent over tting and improve generalization performance.
We nally submitted 7 submissions of three main methods. Ensemble of networks is banned this year. We will focus on the performance of convolutional recurrent neural networks and other data augmentation methods without more computing resources for bird recognition. Features can also have great impact on performance sometimes and they would be studied as well. There is still a lot of work to improve in bird sound recognition in the future. Conference and Labs of the Evaluation Forum, Dublin, Ireland, September 11-14, 2017. (2017), http://ceur-ws.org/Vol-1866/paper 177.pdf 11. Sprengel, E., Jaggi, M., Kilcher, Y., Hofmann, T.: Audio based bird species identi cation using deep learning techniques. Tech. rep. (2016) 12. Szegedy, C., Vanhoucke, V., Io e, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2818{2826 (2016) 13. Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)