=Paper=
{{Paper
|id=Vol-2540/paper25
|storemode=property
|title=None
|pdfUrl=https://ceur-ws.org/Vol-2540/FAIR2019_paper_13.pdf
|volume=Vol-2540
}}
==None==
https://ceur-ws.org/Vol-2540/FAIR2019_paper_13.pdf
Using neural networks to identify individual
animals from photographs ?
Emmanuel Kabuga1,2 , Ian Durbach1,2 , Bubacarr Bah2,3 , and Allan Clark1
1
University of Cape Town (https://www.uct.ac.za/)
2
AIMS South Africa (https://aims.ac.za/)
3
Stellenbosch University (http://www.sun.ac.za/english)
Wildlife conservation relies on sound knowledge of population information.
This information is crucial for addressing questions related to community, ecosys-
tem function, population dynamics, and behavioural ecology. It is obtained via
studies that recognise individuals. A commonly used technique to achieve indi-
vidual species recognition is the use of invasive methods that apply a tag to the
animal’s body. These methods have been applied to both marine and terrestrial
species to assess both theoretical and applied questions [5, 6]. However, inva-
sive procedures are expensive to implement and potentially reduce the animal’s
natural behaviour and performance, disturb its activities and relationship to oth-
ers. In addition to being impracticable with large populations, invasive methods
cause ethical and welfare conflicts due to temporary or permanent application
of tags. Alternatively, many species have body marks such as spots and fins that
are individual-specific and hence can be utilised for individual recognition. These
methods are cost-effective and not harmful to the animal’s life. These approaches
have evolved as a reliable alternative to invasive methods and have been applied
to a range of animals including mammals, amphibians, reptiles and fishes [8,
9]. This paper aims at developing a machine learning algorithm which exploits
individual-specific marks to automate the individual identification task and com-
pares the model results with some of the existing computer-aided software used
by the ecology community.
The developed model is tested on two case studies, a humpback whale (HBW)
dataset and a western leopard toad (WLT) dataset. The HBW dataset consists
of 25 631 images from 14 668 individuals. They are originally collected by vari-
ous institutions across the globe and uploaded to the Happywhale platform [4].
HBWs can be identified by their fins and special marks. The WLT dataset con-
sists of 1 770 images collected by citizen scientists in South Africa. They were
either uploaded to iSpot [7], a citizen science project which collects images or sent
to the WLT project, a conservation project staffed by volunteers. WLTs can be
identified by their unique spots. One part of this dataset consists of 164 labelled
individuals comprising 430 images and an unlabelled proportion comprising 1
340 images.
The model developed in this paper consists of two main components, an
object detection model and a matching classifier model. In some images, the
?
Acknowledgement to AIMS South Africa for the Research Masters scholarship,
CHPC for computational resources, Dr. John Measey and Mr. Alex Rebelo for col-
lecting and providing the toad photographs.
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0)
2 E. Kabuga et al.
animal only occupies a small region of the image. This makes the individual-
specific marks – spots for WLTs, and tail fins for HBWs to not be clearly visible
while they are the key feature of this study. As a result, the goal of the object
detection model is to – detect the region of the image containing the animal,
localise it using a bounding box, and extract the animal, which is then taken
to the next level of photo-matching. The object detection model is a custom
convolutional neural network (CNN) originally inspired from VGG16 [1] which
takes an image as input and outputs the coordinates of the region containing
the animal within the image. The matching classifier model is a special kind of
CNN called a Siamese network. The Siamese network is a custom ResNet [2]
model which uses a pair of CNNs that share weights to summarise the images,
followed by some dense layers which combine the summaries into measures of
similarity which can be used to predict a match. This model takes a pair of
two regions containing animals extracted by the detection model and outputs
their matching probability. A threshold probability set by the user is used to
decide if a pair of two animal images originates from the same individual or
not. The individual IDs are extracted from the obtained matches. One of the
computer-aided photo-matching algorithms used by the ecology community is
WildID [3]. It utilises the scaled invariant feature transform (SIFT) to extract
distinctive features from the images. It compares SIFT features of a new image
with ones of the existing images in the catalogue and ranks the top 20 potential
matches. The true match can appear anywhere between 1 and 20 or not. For
a fair comparison with the developed Siamese network, we checked if WildID
ranked the true match in the first position or not (top-1 accuracy).
The detection model achieved reliable results on both datasets for the task
of localising the region of the image containing the animal on both datasets.
The model achieved the intersection over union (IoU) of 0.90 and the coefficient
of determination R2 of 0.91 for HBWs and the IoU of 0.86 and R2 of 0.85 for
WLTs. The Siamese network model results are good for both HBWs and WLTs.
The model correctly identified if a pair of images is from the same individual
or not respectively in 95% of cases for HBWs and 87% of cases for WLTs. The
main difference in the performance is due to the different amount of data used
to train the model. In this study, the semi-supervised approach on WLT unla-
belled dataset has been partially successful. The model was able to identify 47
new matches from 26 individuals comprising 63 images. These identified matches
seem to be relatively few in numbers. Without an exhaustive check of the data,
it is not clear whether this is due to the failure of the semi-supervised approach,
or because there are not many matches in the data. After adding the newly iden-
tified and labelled individuals to the WLT labelled dataset, the model slightly
improved its performance and correctly identified 89% of WLT pairs. WildID
achieved good results on WLTs compared to HBWs. It ranked the true match
in the first position in 64% of cases for WLTs and 36% of the cases for HBWs.
The Siamese network model achieved good results on the individual identi-
fication task for species dotted with individual-specific marks. Its performance
was very competitive compared with WildID.
Using neural networks to identify individual animals from photographs 3
References
1. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale
image recognition. arXiv preprint arXiv:1409.1556, 2014
2. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition.
In Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 770-778, 2016.
3. D. T. Bolger, T. A. Morrison, B. Vance, D. Lee, and H. Farid. A computer-assisted
system for photo- graphic markrecapture analysis. Methods in Ecology and Evolu-
tion, 3(5):813822, 2012.
4. https://happywhale.com/home
5. J. N. Auckland, D. M. Debinski, and W. R. Clark. Survival, movement, and resource
use of the butterfly parnassius clodius. Ecological Entomology, 29(2):139149, 2004.
6. D. J. Booth. Synergistic effects of conspecifics and food on growth and energy allo-
cation of a damselfish. Ecology, 85(10):28812887, 2004.
7. https://www.ispotnature.org/node/137767
8. L. Gamble, S. Ravela, and K. McGarigal. Multi-scale features for identifying indi-
viduals in large biological databases: an application of pattern recognition technol-
ogy to the marbled salamander ambystoma opacum. Journal of Applied Ecology,
45(1):170180, 2008.
9. C. W. Speed, M. G. Meekan, and C. J. Bradshaw. Spot the matchwildlife photo-
identification using information theory. Frontiers in zoology, 4(1):2, 2007.