The GeoLifeCLEF 2024 competition [1] is held jointly as part of the LifeCLEF 2024 lab [2] and the FGVC11 workshop. Just like in the GeoLifeCLEF 2023 competition [3] the goal is to predict a list of species most likely to be observed at a given location using various geographical and environmental data such as satellite images and time series, climatic time series, and other rasterized data: land cover, human footprint, bioclimatic, and soil variables. Typically, the task of species distribution modelling [4] has challenges associated with imbalances in species presence and absence in the data, large-scale multimodal learning, and plant species diversity. Its results could be useful for predicting biodiversity change and mitigating environmental pressures from human activities.
The GeoLifeCLEF 2024 training data includes a collection of observations of plants in Europe. Each survey consists of a list of plant species with the GPS coordinates and a set of variables characterizing the landscape and environment around them. There are around 90K surveys with around 5K unique plant species in the dataset. This technical report presents selected approach to the competition, which is a multimodal network based on bioclimatic cubes, sentinel image patches (RGB-patch and NIR-patch) and vector of climate, elevation, human footprint, land cover, soilgrid and landsat time series data. Traing code can be found here1 .
Data plays an important role in prediction plant species distribution in a given location and time. In this section, we briefly present the data and the evaluation metric used for the competition.
This paragraph is simply a description of the standard GeoLifeCLEF 2024 dataset. The training dataset contains presence-absence (PA) surveys and presence-only (PO) surveys. PO data includes about 5 million observations and reports only presence and not absence of certain plant species in specific areas. On the other hand, PA data combines around 90K surveys with about 5K unique species of the European flora and reports presence and absence of plant species. In solution only presence-absence surveys were used and everywhere below the report will only be about this type of data. The total number of surveys in the test set was 5K.
Training dataset distribution of the number of observations of each plant species is shown in Figure 1. Almost 50% of plant species in training data have a number of occurrences less than 16 and only 20% have more than 110 occurrences. Almost all observations were made in Western Europe, a map of locations can be seen in Figure 2. More detailed descriptions can be found at competitions's homepage 2 .
Each survey is paired with the following covariates:
β’ Satellite image patches: 128mΓ128m RGB-NIR patches centered at each observation, at a resolution of 1 meter per pixel; β’ Satellite time series: Up to 20 years of values for six satellite bands (R, G, B, NIR, SWIR1, and SWIR2); β’ Environmental rasters Various climatic, pedologic, land use, and human footprint variables at the European scale. It was provided as scalar values, time-series, and original rasters;
The evaluation metric for the GeoLifeCLEF 2024 competition is the samples-averaged πΉ 1 -score computed on a set made of species presence-absence samples. The πΉ 1 -score is an average measure of overlap between the predicted and actual set of species present at a given location and time. Each observation π is associated with a list of ground-truth labels π π corresponding to the observed plant species. For each observation, the submissions provide a set of species predicted presence π π,1 , π π,2 , ..., π π,π π . The micro πΉ 1 -score is then computed using:
where π π π , πΉ π π and πΉ π π are the true positive, the false positive and the false negative of the j-th input sample, respectively. π is the number of samples for evaluation.
This section describes the methods that were tried during the competition. Strategy was centered around the baseline model 3 provided by the competition organizers. The baseline πΉ 1 -score is 0.31 on the public set. This model leveraged all environmental data and utilized a multimodal neural network with separated features extractors to return a single prediction set in order to take advantage of every modality (satellite images, bioclimatic cubes, landsat cubes). The main change was to replace landsat cubes with a vector of satellite time series and environmental scalar values, everywhere below it is called feature vector. In addition, plant species with an occurrence number greater than 10 was used to train the model.
Feature vector consists of climate, elevation, human footprint, land cover, soilgrid and landsat time series data. Methods for compiling this data are taken from the public notebook 4 . Climatic time series data was merged within a 10-year time window. Some positions had missing values, which were filled with spatial interpolation. It appeared that there were densely populated measurements near the missing regions, so missing values were filled with values from the nearest neighbors. Finally, each survey had 1198 values of feature vector. The train and test versions can be found here. Before going to model feature vectors are normalized with standard scaler.
The architecture closely follows the baseline model, incorporating a multimodal neural network that utilizes three distinct feature extractors for bioclimatic rasters (19 channels), satellite images (4-channel RGB with NIR), and feature vectors (1198 channels). These outputs are combined and processed through fully connected layers to generate predictions. The first bioclimatic head involves layer normalization, ResNet18 [5] without pretrained weights, and a dropout [6] with a 0.1 probability. The second image head employs a , swin transformer [7] model with ImageNet [8] weights and a dropout layer with a 0.1 probability. Prior to this stage, image data undergo augmentation techniques like random rotation, random brightness contrast, and normalization. The third head comprises a sequence of layer normalization and three linear layers with GELU [9] activation function, along with dropout set at a 0.1 probability (the first layer mapping from 1198 to 1198, the second and the third layers map to 1000 outputs). Subsequently, the bioclimatic and feature outputs are normalized and combined with the image output. The final classifier is constructed with three linear layers utilizing GELU activation function and dropout at a 0.1 probability.
The model was trained on PA data for 12 epochs using the Adam optimizer with a learning rate of 8e-5 and binary cross entropy (BCE) loss and batch size equal to 128. During training, we focused on plant species with an occurrence number greater than 10, resulting in 2857 unique species out of a total of 5015. It's important to highlight that the occurrence threshold value was determined through experimentation.
In final approach to inference, the strategy used in the baseline notebook was changed. Rather than forecasting the 25 most probable species for every observation in the test dataset, selected threshold of 0.18 was used. This threshold determined that species with probabilities surpassing this value were classified as present. Additionally, test observations featuring fewer than 4 represented species was assigned with the 4 most likely species.
Experiments were conducted with the multimodal network described in Section 3.2. The detailed settings of training are shown in Table 1. For comparing different versions of models we used 25 most probable species to remove bias with probability threshold described in Section 3.3.
In order to investigate the impact of using the feature vector head we conducted ablation study. Table 2 represents the detailed results. It seems that with selected hyperparameters combination
As was mentioned before, the dataset is strongly unbalanced, which means that for almost all species the number of observations detecting their presence is much less than the number of observations detecting their absence. we tried to solve this problem in different ways, for example, adding pos_weight to bce loss, adding different data augmentation. The final option was to limit the number of species on which the model is trained, taking only those with occurrence number greater than 10. Table 2 shows how the score depends on the threshold for the occurrence number. Another thing was lowering the threshold for a species having a probability higher than which it was considered present. For those observations that had fewer than 4 species present we assigned the 4 most likely plant species. Results of different probability thresholds are presented in Table 3.
We presented the working principles of submission to the GeoLifeCLEF 2024 challenge and discussed some of the key findings of the results. We have not conducted an expansive, let alone exhaustive hyperparameter search and believe that doing so could raise performance a bit. The main achievement was to use proper model architecture, choosing training data and changing the inference strategy. In final solution, we did not use PO data and training strategies used in previous years [10,11]. Obviously, using more data would help for better generalization and it is certainly high on the list of improvements that need to be made. Also, possible improvements can be achieved by additionally searching for better backbone models, like Inception-v4 [12] or Vision Transformer, ViT B / 16 [13] for different modalities and using an ensemble of various models.