In this paper, we present a system for visually retrieving ancient inscriptions, developed in the context of the ongoing Europeana network of Ancient Greek and Latin Epigraphy (EAGLE) EU Project. The system allows the user in front of an inscription (e.g, in a museum, street, archaeological site) or watching a reproduction (e.g., in a book, from a monitor), to automatically recognize the inscription and obtain information about it just using a smart-phone or a tablet. The experimental results show that the Vector of Locally Aggregated Descriptors is a promising encoding strategy for performing visual recognition in this speci c context.
The large availability of digital cameras, especially embedded in smartphones and tablets, allows nal users to make photos of their objects of interest at almost no cost. On one side, there are users making thousands of photos. On the other side, cultural heritage institutions typically have photos and metadata, both in digital form, related to the objects they preserve. In this context, there is a growing demand of technologies for content-based multimedia information retrieval.
In the last few years, research on object recognition has focused on local
features [
Visual objects recognition has also been studied in the context of cultural
heritage and computing. As an example, the VISITO Tuscany1 project has
investigated the visual recognition of cultural heritage objects (such as monuments,
landmarks, etc.) [
The research reported in this paper summarizes the results presented in [
The dataset we used consists of 17,155 photos related to 14,560 ancient
inscriptions that were made available by Sapienza University of Rome, within the
EAGLE project. In order to visual recognizing inscriptions, we selected and tested
the most promising approaches from the recent literature. As local feature we
used the well known Scale Invariant Feature Transform (SIFT) [
To recognize the actual object in a query image, we perform a visual similarity search between all the images in the dataset. The optimum would be to have an image of the same inscription as rst result. Whenever this is not the case, it is interesting to understand at which position in the result list the most visually similar photo of the same object appears. In fact, traditional computer vision techniques could be applied on the results in order to achieve better e ectiveness. Thus, we use the probability p of nding an image of the same object within the rst r results, as quality measure. For r = 1, p equals the accuracy of a classi er that classify the query inscription as the most similar that has been found (i.e., a 1-NN classi er). A common measure of e ectiveness in similarity search applications is the mean-Average Precision (mAP) which e ectively summarizes the average and precision curves.
In Table 1, we report the best results obtained ordered with respect to the
mAP. In the rst column, we report a brief text about the approach. In the
second column, the average number of SIFT considered is shown (i.e., 235 when
local features selection was applied and 1,591 otherwise). The third column
reports the number of words used in the aggregation phase. While the words have
been selected both for BoF and VLAD using k-means, their use is very di
erent. Thus, in the bytes column, we computed the average size in bytes of the
resulting representation. As quality measures, we used the probability p of
having at least one relevant image between the rst r results for r =1,10,100 and
the mAP. In case we use these approaches to recognize the query image relying
Visual Recognition in the EAGLE Project
on the nearest image in the dataset, the best approach is the VLAD that
obtained an accuracy of 0:69 for a codebook size of 256 and selecting the 235 most
relevant local features. The second best is the BoF in conjunction with
geometry consistency checks performed using RANSAC [
In this work, we tested state-of-the-art object recognition techniques on a dataset of 17,155 photos related to 14,560 inscriptions. The best accuracy was obtained by using the VLAD approach that has been recently proposed for performing object recognition on a large scale. Surprisingly, even the BoF approach in conjunction with geometry consistency checks was not able to outperform the VLAD representation, that can be also more e ciently indexed than BoF. The obtained accuracy was of 0.69, which is good considering the di culties of the task and the few images available for each inscription in the dataset. However, we plan to improve this results by performing re-ranking and direct local features matching. To this goal, we also reported the probability of having a relevant images between the retrieved images. The results show that it is possible to have a relevant image between 100 retrieved ones with probability 0.90 using the VLAD approach with a visual vocabulary of size 256 and ltering the SIFT. Thus, we plan to try binary local features and other techniques in order to improve the obtained 0.69 accuracy up to the 0.90 obtainable, in theory, by re-ranking the rst 100 image retrieved using VLAD.
This work was partially supported by EAGLE (Europeana network of Ancient Greek and Latin Epigraphy, co-founded by the European Commision, CIP-ICTPSP.2012.2.1 - Europeana and creativity, Project Reference 325122).