This paper presents several approaches for plant predictions given their location in the context of the GeoLifeCLEF 2018 challenge. We have developed three kinds of prediction models, one convolutional neural network on environmental data (CNN), one neural network on co-occurrences data and two other models only based on the spatial occurrences of species (a closest-location classi er and a random forest tted on the spatial coordinates). We also evaluated the combination of these models through two di erent late fusion methods (one based on predictive probabilities and the other one based on predictive ranks). Results show the e ectiveness of the CNN which obtained the best prediction score of the whole GeoLifeCLEF challenge. The fusion of this model with the spatial ones only provides slight improvements suggesting that the CNN already captured most of the spatial information in addition to the environmental preferences of the plants.
Automatically predicting the list of species that are the most likely to be observed at a given location is useful for many scenarios in biodiversity informatics. First of all, it could improve species identi cation processes and tools by reducing the list of candidate species that are observable at a given location (be they automated, semi-automated or based on classical eld guides or ora). More generally, it could facilitate biodiversity inventories through the development of location-based recommendation services (typically on mobile phones) as well as the involvement of non-expert nature observers. Last but not least, it might serve educational purposes thanks to biodiversity discovery applications providing innovative features such as contextualized educational pathways. This challenge is highly related to the problem known as Species Distribution Modeling (SDM) in ecology. SDM have become increasingly important in the last few decades for the study of biodiversity, macro ecology, community ecology and the ecology of conservation. An accurate knowledge of the spatial distribution of species is actually of crucial importance for many concrete scenarios including landscape management, preservation of rare and/or endangered species, surveillance of alien invasive species, measurement of human impact or climate change on species, etc. Concretely, the goal of SDM is to infer the spatial distribution of a given species, and they are often based on a set of geo-localized occurrences of that species (collected by naturalists, eld ecologists, nature observers, citizen sciences project, etc.). However, it is usually not reliable to learn that distribution directly from the spatial positions of the input occurrences. The two major problems are the limited number of occurrences and the bias of the sampling e ort compared to the real underlying distribution. In a real-world dataset, the raw spatial distribution of the occurrences is actually highly in uenced by the accessibility of the sites, the preferences and habits of the observers. Another di culty is that an occurrence means a punctual presence of the species, while no occurrences doesn't mean the species is absent, which makes us very uncertain about regions without observed specimens.
For all these reasons, SDM is usually achieved through environmental niche
modeling approaches, i.e. by predicting the distribution in the geographic space
on the basis of a representation in the environmental space. This environmental
space is in most cases represented by climate data (such as temperature, and
precipitation), but also by other variables such as soil type, land cover, distance
to water, etc. Then, the objective is to learn a function that takes the
environmental feature vector of a given location as input and outputs an estimate
of the abundance of the species. The main underlying hypothesis is that the
abundance function is related to the fundamental ecological niche of the species.
That means that in theory, a given species is likely to live in a single privileged
ecological niche, characterized by an unimodal distribution in the environmental
space. However, in reality, the abundance function is expected to be more
complex. Many phenomena can actually a ect the distribution of the species relative
to its so called abiotic preferences. For instance, environment perturbations, or
geographical constraints, or interactions with other living organisms (including
humans) might have encourage specimens of that species to live in a di erent
environment. As a consequence, the realized ecological niche of a species can be
much more diverse and complex than its hypothetical fundamental niche.
Very recently, SDM based on deep neural networks have started to appear [
In this paper, we report an evaluation study of three main kinds of SDM in the
context of the GeoLifeCLEF challenge [
Section 2 gives an overview of the data and evaluation methodology of the GeoLifeCLEF challenge. Section 3 provides the detailed description of the evaluated models. Section 4 presents the results of the experiments and their analysis. 2
A detailed description of the protocol used to build the GeoLifeCLEF 2018
dataset is provided in [
This dataset was split in 3/4 for training and 1/4 for testing with the constraints that: (i) for each species in the test set, there is at least one observation of it in the train set. and (ii), an observation of a species in the test set is distant of more than 100 meters from all observations of this species in the train set. In the following, we usually denote as x 2 X a particular occurrence, each x being associated to a spatial position p(x) in the spatial domain D, a species label y(x) and an environmental tensor g(x) of size 64x64x33. We denote as P the set of all spatial positions p covered by X. It is important to note that a given spatial position p0 2 P usually corresponds to several occurrences xj 2 X; p(xj ) = p0 observed at that location (18 000 spatial locations over a total of 60 000, because of quantized GPS coordinates or Names-to-GPS transforms). In the training set, up to several hundreds of occurrences can be located at the same place (be they of the same species or not). The occurrences in the test set might also occur at identical locations but, by construction, the occurrence of a given species does never occur at a location closer than 100 meters from the occurrences of the same species in the training set.
The used evaluation metric is the Mean Reciprocal Rank (MRR). The MRR is a statistic measure for evaluating any process that produces a list of possible responses to a sample of queries ordered by probability of correctness. The reciprocal rank of a query response is the multiplicative inverse of the rank of the rst correct answer. The MRR is the average of the reciprocal ranks for the whole test set:
Q M RR = 1 X
Q
1 q=1 rankq where Q is the total number of query occurrences xq in the test set and rankq is the rank of the correct species y(xq) in the ranked list of species predicted by the evaluated method for xq. 3 3.1
It has been previously shown in [
Learning set up and parameters: All our experiments were conducted using PyTorh deep learning framework4 and were run on a single computing node equipped with 4 Nvidia GTX 1080 ti GPU. We used the Stochastic Gradient Descent optimization algorithm with a learning rate of 0.001 (decreased every 10 epoch by 10), a momentum of 0.9 and a mini-batch size of 16. 3.2
For this category of models, we rely solely on the spatial positions p(x) to model the species distribution (i.e. we do not use the environmental information at all). We did evaluate two di erent classi ers based on such spatial data: 1. Closest-location classi er: For any occurrence xq in the test set and its associated spatial position p(xq), we return the labels of the species observed at the closest location pNN in Ptrain (except p(xq) itself if p(xq) 2 Ptrain). The species are then ranked by their frequency of appearance at location pNN . Note that p(xq) is excluded from the set of potential closest locations because of the construction protocol of the test. Indeed, as mentioned earlier, it was enforced that the occurrence of a given species in the test set does never occur at a location closer than 100 meters from the occurrences of the same species in the training set. As a consequence, if we took pNN = p(xq), the right species would never belong to the predicted set of species. One of the problem of the above method is that it returns only a subset of species for a given query occurrence xq (i.e. the ones located at pNN . Returning a ranked list of all species in the training set would be more pro table with regard to the used evaluation metric (Mean Reciprocal Rank). Thus, to improve the overall performance, we extended the list of the closest species by the list of the most frequent species in the training set (up to reaching the authorized number of 100 predictions for each test item). 2. Random forest classi er: Random forests are known to provide good performance on a large variety of tasks and are likely to outperform the naive closest-location based classi er described above. In particular we used the random forest algorithm implemented within the scikit-learn framework5. 4 https://pytorch.org/ 5 http://scikit-learn.org/stable/ We used only the spatial positions p(x) as input variables and the species labels y(x) as targets. For any occurrence xq in the test set, the random forest classi er predicts a ranked list of the most likely species according to p(xq). Concerning the hyper-parametrization of the method, we conducted a few validation tests on the training data and nally used 50 trees of depth 8 for the nal runs submitted to the GeoLifeCLEF challenge. Species co-occurrence is an important information in that it may capture interdependencies between species that are not explained by the observed environment. For instance, some species live in community because they share preferences for a kind of environment that we don't observe (communities of weeds are often specialized to ne scale agronomic practices that are not reported in our environmental data), they use the available resources in a complementary way, or they favor one another by a ecting the local environment (leguminous and graminaceous plants in permanent grasslands). On the opposite, some species are not likely to be observed jointly because they live in di erent environments, they compete for ressources or negatively a ect the environment for others (allelopathy, etc.). To capture this co-occurrence information, it is required to train a model aimed at predicting the likelihood of presence of a given species thanks to the knowledge of the presence of other species (without using the environmental information or the explicit spatial positions). Therefore, we did train a feed-forward neural network taking species abundance vectors as input data and species labels as targets. The abundance vectors were built in a similar way than the closest-location classi er described in section 3.2. For any spatial position q 2 D, we rst aggregate all the occurrences located at the closest location pNN in Ptrain (except q itself). Then, we count the number of occurrences of each species in the aggregated set. More formally, we de ne the abundance vector z(q) 2 RN of any spatial position q 2 D as: 8i; 8x; zi(q) =
X p(x)=pNN 1(y(x) = i) (1) where 1() is an indicator function equals to one if the condition in parenthesis is true and zi(q) is the component of z(q) corresponding to the abundance of the i-th species.
The neural network we used to predict the most likely species based on a given
abundance vector is a simple Multi-Layered Perceptron (MLP) with one hidden
layer of 256 fully connected neurons. We used ReLU activation functions [
We conducted a set of preliminary experiments on the training set before training the nal models to be evaluated within the GeoLifeCLEF challenge. Therefore, we extracted a part of the training set (10% occurrences selected at random) and used it as a validation set. We choose two cross-validation protocol : { For the two models based on neural network we choose to x the split between training set and test set (Holdout cross-validation). As, the neural networks took around a day to be learned completely, it was not workable to repeat split and learning many times. Thus, we worked with a single validation set to calibrate all our neural networks models. If we don't x the train-test split we can't compare the two networks learn once, because the di erence of performance can be due to this split. By xing the train-test split we assume to introduce a bias, but this bias is then constant between the experiments which allows us to compare the performance obtained on a single learning. { For the two spatial models, that require a lower computation time, we choose to not x the train-test split but to learn the model on twenty random train-test split (Monte Carlo cross-validation). The performance of a model is de ned by the average performance of the model on the twenty di erent train-test split. Like this we don't introduce a bias as for the neural networks but we keep the possibility to compare two models. Note that for the random forest classi er of scikit-learn we need to have at least one occurrence of each species in the training set and one occurrence of each species in the test set. However, some species are present only once in the data, so we had to remove them for validation experiments of this model.
model mean valisdtaatnidoanrdMdReRviation mean valsidtaantidoanrdTodpe1viation closest-location 0.0640 0.0011 0.0314 0.0010 random forest 0.0781 0.0008 0.0304 0.0008 Table 1. Performance of the two spatial models in validation experiments (Monte Carlo cross-validation on twenty random train-test splits).
For the validation experiments, in addition to the MRR (see section 2), we also measured the top1 accuracy, i.e. the percentage of well predicted occurrence species by the model at the rst prediction rank. The validation performance of each model is given in tables 1 and 2. The best model is the CNN. It achieves a pretty good MRR of 0:10 knowing that the ideal MRR cannot exceed 0:409 (due to the fact that several outputs exist for the same entry). On average, it returns the correct species in the position with a success rate close to 1/20 (knowing that there is 3336 species in the training set). Nevertheless, the other models achieve good results too, all are over 0:06 of MRR and the random forest reaches almost 0:08. They return the good species label between 1 time out of 40 and 1 time out of 30. These results show that some fairly simple models can capture a strong information. It would be interesting to study the complementarity between these methods and the CNN to produce a highly predictive model. 4.2
Submissions We submitted 10 run les to be evaluated within the LifeCLEF 2018 challenge, each run le containing the prediction of a particular method on the whole test set. It is important to note that this evaluation was conducted entirely in blind, i.e. we never had access to the labels of the test set.
The four rst run les we submitted contained the predictions of our four main models, i.e.: FLO 1: The predictions of the closest-location classi er model. FLO 2: The predictions of the co-occurrence model.
FLO 3: The predictions of the environmental CNN.
FLO 4: The predictions of the spatial random forest classi er.
The other six run les we submitted corresponded to di erent fusion schemes of the four base models. Indeed, the base models being trained on di erent kinds of input data, we expect that their fusion may bene t from their complementarity. We used two di erent kinds of fusion methods: Late fusion based on probabilities: For each test item we simply average the prediction probabilities of the fused models and then we re-sort the predictions. Note that we could'nt do this late fusion with the closest-location classi er as it doesn't output probabilities, but only species ranks. For the three other models, we evaluated the fusion of all possible pairs and the fusion of the three models: FLO 5: late fusion of the probabilities given by the CNN and the co-occurrences models.
FLO 6: late fusion of the probabilities given by the CNN and the spatial random forest models.
FLO 7: late fusion of the probabilities given by the co-occurrences and the spatial random forest models.
FLO 8: late fusion of the probabilities given by the CNN, the co-occurrences and the spatial random forest models.
Late fusion based on Borda count: Borda count is a voting system allowing to merge ranked list of candidates. In our case, it simply consists in summing the rank of a test item in the di erent run les to be fused. Two new run les were generated using this method: FLO 9: Borda count of the predictions given by the CNN, the co-occurrences and the spatial random forest models FLO 10: Borda count of the predictions given by the CNN, the spatial random forest and the closest-location classi er Results The results we obtained within the GeoLifeCLEF challenge are presented in table 3 (along with other participant's runs). Figure 2 illustrates the MRR values obtained by our runs solely. The conclusions we can draw from the results are the following: { Supremacy of the CNN model:The results show that our run FLO 6 is the best performing model among all the evaluated systems in the challenge. It corresponds to the fusion of the environmental CNN model and the spatial random forest classi er. Nevertheless, the CNN model alone obtains a very close performance (FLO 3) so that the contribution of the random forest predictions to the fused model seems to be very limited. As another evidence of the supremacy of the CNN model, all our other runs including its predictions (FLO 8, FLO 5,FLO 9,FLO 10) are above all the other runs submitted to the challenge. However, their performance is degraded compared to the CNN model alone.
The second best model within our four base models seems to be the spatial classi er based on random forest (FLO 4). Indeed, it obtains a very fair performance considering that it only uses the spatial positions of the occurrences (which makes it very easy to implement in a real-world system). The co-occurrence model (FLO 2) obtains signi cantly lower performance, while the closest-location classi er, which uses only the nearest point species data, is the worst model (FLO 1). { Late fusion methods comparison: We can notice that the probabilities late fusion (FLO 8) worked better than Borda's (FLO 9). However, our late fusions, that give the same weights to fused models predictions, never signi cantly outperformed the best of the fused models, especially for fusions based on the environmental CNN. Though, we can wonder if learning an unbalanced weighting, or a local weighting would increase the performance. { Final results vs. cross-validation results: Overall, the MRR values achieved by our models on the blind test set of GeoLifeCLEF are much lower than the ones obtained within our cross-validation experiments (see Tables 1 and 2). We believe that this performance loss is mainly due to the construction of the blind test set, i.e. to the fact that the occurrence of a given species in the test set does never occur at a location closer than 100 meters from the occurrences of the same species in the training set. This rule was not taken into account during our cross-validation experiments on the training set. { Species community: The co-occurrence model FLO 2 seems to generalize better than the closest-location classi er (FLO 1), though both methods used almost the same input information which is the species of the neighborhood. It is likely that the neural network detect the signature of a community from its input co-occurrences. For example, the network is able to predict a common mediterranean species when it gets a rare mediterranean species as entry. Indeed, the probability of observing this same rare species near its known observation is very small, but the closest location classi er would do the error.
This paper reported our participation to the GeoLifeCLEF challenge aimed at evaluating location-based species prediction models. We compared three main types of models: (i) a convolutional neural network trained on environmental variables extracted around the location of interest, (ii) a purely spatial model trained with a random forest and (iii), a co-occurrence based model aimed at predicting the likelihood of presence of a given species thanks to the knowledge of the presence of other species. The main conclusion of our study is that the convolutional neural network model is the best performing model. Indeed, it achieved the best performance of the whole GeoLifeCLEF challenge. Interestingly, the combination of the CNN model with the other models did not allow any signi cant improvement of the results. This is surprising in the sense that the CNN model was trained on environmental data solely whereas the other models focused on complementary information, i.e. the spatial location and the species co-occurrences. This suggests that the CNN model already captured all this information, maybe because the environmental tensor associated to each location is su cient to recognize this particular location. In future work, we will attempt to better understand what information the CNN does capture from that environmental tensors and how it could be improved according to this.