=Paper=
{{Paper
|id=Vol-2235/paper7
|storemode=property
|title=ArchAIDE: Archaeological Automatic Interpretation and Documentation of cEramics
|pdfUrl=https://ceur-ws.org/Vol-2235/paper7.pdf
|volume=Vol-2235
|authors=Holly Wright,Gabriele Gattiglia
|dblpUrl=https://dblp.org/rec/conf/euromed/WrightG18
}}
==ArchAIDE: Archaeological Automatic Interpretation and Documentation of cEramics==
https://ceur-ws.org/Vol-2235/paper7.pdf
ArchAIDE: Archaeological Automatic Interpretation and
Documentation of cEramics
Holly Wright1[0000-0002-3403-4159] and Gabriele Gattiglia2[0000-0002-4245-8939]
1
University of York, The King’s Manor, York, YO1 7EP, UK
2
University of Pisa,Via Trieste 38, 56126 Pisa, Italy
holly.wright@york.ac.uk
Abstract. The ArchAIDE project (archaide.eu) is funded by the European
Union’s Horizon 2020 research and innovation programme and has developed a
new app that aims to improve the practice of pottery recognition in archaeology,
using the latest automatic image recognition technology. Every day,
archaeologists are working to discover and tell stories around objects from the
past, investing considerable time, effort and funding to identify and characterise
individual finds. Pottery is of fundamental importance for the comprehension
and dating of archaeological contexts, and for understanding the dynamics of
production, trade flows, and social interactions. Today, this characterisation and
classification of ceramics is carried out manually, through the expertise of
specialists and the use of analogue catalogues held in archives and libraries. The
goal of ArchAIDE is to optimise and economise this process, making
knowledge accessible wherever archaeologists are working. ArchAIDE
supports the classification and interpretation work of archaeologists (during
both fieldwork and post-excavation analysis) with an innovative app for tablets
and smartphones, designed to be an essential tool for archaeologists. Pottery
fragments are photographed, their characteristics sent to a comparative
collection, which activates the image recognition system, resulting in a response
with all relevant information linked, and ultimately stored, within a database
that allows sharing online. The system currently supports shape-based
recognition of Terra Sigillata and Roman Amphorae, and decoration-based
recognition of Majolica of Montelupo, as proof-of-concept.
Keywords: Archaeology, Pottery, Ceramics, Image Recognition, 3D Modelling
1 Overview
The ArchAIDE project [1] is funded by the European Union’s Horizon 2020 research
and innovation programme and has developed a new app that aims to improve the
practice of pottery recognition in archaeology, using the latest automatic image
recognition technology [2]. Every day, archaeologists are working to discover and tell
stories around objects from the past, investing considerable time, effort and funding to
identify and characterise individual finds. Pottery is of fundamental importance for the
comprehension and dating of archaeological contexts, and for understanding the
dynamics of production, trade flows, and social interactions. Today, this
characterisation and classification of ceramics is carried out manually, through the
Cultural Informatics 2018, November 3, 2018, Nicosia, Cyprus. Copyright held by
the author(s).
60
expertise of specialists and the use of analogue catalogues held in archives and
libraries. The goal of ArchAIDE is to optimise and economise this process, making
knowledge accessible wherever archaeologists are working. ArchAIDE supports the
classification and interpretation work of archaeologists (during both fieldwork and
post-excavation analysis) with an innovative app for tablets and smartphones,
designed to be an essential tool for archaeologists. Pottery fragments are
photographed, their characteristics sent to a comparative collection (which is meant to
show typical pottery types and characteristics, against which pottery to be identified
by the user is compared), which activates the image recognition system, resulting in a
response with all relevant information linked, and ultimately stored, within a database
that allows sharing online. This goal has been implemented through the following
practical elements:
● a digital comparative collection for multiple pottery types has been
created, incorporating existing digital collections, digitised paper
catalogues and multiple photography campaigns;
● an automatic-as-possible workflow has been built to accurately digitise
paper catalogues and improving the search and retrieval process;
● a multilingual thesaurus of descriptive pottery terms has been created,
mapped to the Getty Art and Architecture Thesaurus and including French,
German, Spanish, Catalan, English and Italian;
● an app has been created using the digital comparative collections to
support archaeologists in recognising potsherds during excavation and
post-excavation analysis, with an easy-to-use interface and efficient
image recognition algorithms for search and retrieval based on either
characteristics of shape or decoration;
● the app can also be used as a tool for learning about pottery
identification, either for students or when specialists are not available;
● once a sherd has been recognised, the app can be used to automatically
populate information about the sherd into a virtual assemblage for a
site, including the generation of an identity card as a formatted, digital
or printable document;
● the underlying technologies developed for the app have also been
implemented as a desktop application, which is a web-based, real-time
data visualization resource, to improve access to archaeological heritage
and generate new understanding;
● the comparative data will be available from an open access archive
ensuring it is available for re-use beyond the ArchAIDE project,
contributing to our sustainable, common heritage.
ArchAIDE has entered its third and final year, and has successfully
implemented nearly all of the practical elements. A beta version of the app is currently
being presented and promoted to stakeholders, allowing feedback to be incorporated
into its ongoing development.
61
2 Partnership
The ArchAIDE partnership has representation from the academic and industry-led
ICT domains, and from the academic and development-led archaeology domains. The
archaeological partners of the consortium are the MAPPA Lab at the University of
Pisa (coordinator) which has relevant experience in mathematical and digital
applications in Archaeology, and archaeological communication; the Material Culture
and Archaeometry research unit at the University of Barcelona, which is focused on
promoting studies of material culture, especially on archaeological ceramics, and
archaeometric approaches; the Digital Archaeology Laboratory at the University of
Cologne, which manages ARACHNE, a highly structured object database in
partnership with the German Archaeological Institute (DAI); and the Archaeology
Data Service (ADS) at the University of York, which is a world-leading digital data
archive for archaeology.
The consortium also includes two companies carrying out preventive and
development-led archaeological investigations: Baraka Arqueólogos S.L., which has
particular expertise in the study of archaeological ceramics, and Elements S.L which
is experienced in the application of digital technologies related to ceramic studies.
Finally, the consortium’s technical ICT partners are the Visual Computing Lab at
CNR-ISTI, an institute of Italian CNR devoted to research on Visual Media and
Cultural Heritage; the Deep Learning Lab at the School of Computer Science at Tel
Aviv University, which focuses on document analysis, image textual description, and
action recognition; and the private software company, Inera s.r.l, which has
experience in the field of protocols and web apps.
3 Lessons Learned
It quickly became evident that the quantity and quality of comparative data required
to create the image recognition algorithm was far greater than estimated during the
planning of the ArchAIDE project. This meant that together with the digitisation of
paper catalogues, whose acquisition was at an advanced stage, photos would need to
be taken of many more sherds. Specifically, developing the appearance based
similarity training was more complex than expected, and to ensure accurate results
were returned, recognition based on appearance (decoration) had to be separated from
recognition based on shape (profile), requiring the building of two different
algorithms. As a result, the archaeological partners invested considerable effort in
photographing new pottery sherds to train the algorithms. The system currently
supports shape-based recognition of Terra Sigillata and Roman Amphorae, and
decoration-based recognition of Majolica of Montelupo, as proof-of-concept for the
two main diagnostic criteria used by archaeologists.
In order to gather sufficient examples to train the algorithms, partners
undertook photographic campaigns of collections in a variety of locations in Europe,
along with help from volunteers and students at additional sites. This included more
than 10,000 images of Majolica of Montelupo collected by partners at the University
of Pisa, a large photo campaign and the creation of a typology of medieval and post-
medieval pottery by the partners at the University of Barcelona, ongoing work by
Baraka and Elements in locating sherds of Roman Amphorae and Terra Sigillata from
62
a range of their archaeological excavation work. Partners from the University of
Cologne are also currently undertaking a very large photo campaign of Roman
Amphorae and Terra Sigillata in Austria to improve the shape-based algorithm.
There was much discussion with partners at Tel Aviv University about how
best to approach training the algorithms. For the appearance-based recognition
(decoration), it was decided to create an algorithm based on combining classic
machine learning tools with neural networks that were trained on general image
classification tasks. Following a testing phase on a huge dataset of images, the
functionality was incorporated into the ArchAIDE app, and classification is now
available to archaeologists. Early testing indicated that (a) the performance of the
classification differs with different lighting conditions, and this can be improved by
simulating further lighting conditions while training the system; (b) the recognition
achieves better accuracy on the “common” classes, compared to more “rare” classes,
and this can be fixed by giving a weight to each input in the training, to simulate an
equal number of inputs from each class.
As for shape-based recognition, the system was designed to produce
“synthetic sherds” (3D shapes available on the computer) to train the system, starting
with the pottery profiles that are extracted from the catalogues, in combination with
work carried out by partners at CNR-ISTI. Early in the project, CNR-ISTI carried out
a variety of experiments around automated generation of 3D models from traditional
archaeological pottery profile drawings, which proved very successful. Not only was
it possible to generate a 3D model, it was possible to automatically identify the
different diagnostic parts of ceramic object, allowing objects to be ‘virtually broken’
to create a wide range of synthetic sherds [3]. After being trained and test on
Amphorae synthetic sherds, an algorithm was developed by partners at Tel Aviv
University based on a standard convolutional neural network (CNN). As ArchAIDE is
training on large numbers of classes, there was also experimentation with curriculum
training (gradually introducing more classes during the training process) and custom
loss functions, to make the network converge. Presentation of the difficulties
encountered, results achieved and opportunities for feedback have been carried out
through organising workshops and training days in the UK, Italy, Germany, Spain and
participating in national and international conferences.
During the first two years of the project exploitation strategies were analysed
and discussed, resulting in a better understanding of the potential market has led the
project to a promising approach: a free ArchAIDE mobile app as a vehicle to
commercialise digitised versions of the pottery catalogues. The idea is to show
copyright holders the added value of digitising paper catalogues, as they can then be
used dynamically in a digital environment like the ArchAIDE app. The work of the
copyright holders can then be shown how their work becomes more accessible, and
more useful than in a traditional paper publication. While the long-term goal for
archaeological data is to be open access, for those copyright holders who are not able
to do so, a commercial exploitation model will be developed. ArchAIDE app users
will be able to buy a specific catalogue (i.e. Conspectus) from the app itself as an “in-
app purchase”, the proceeds of which are paid to the copyright holder for use of their
resource. For a user, buying a catalogue means having the possibility to browse and
search the types contained in it and display all the available information, including
multimedia object and eventually 3D models generated by ArchAIDE team. Whether
63
any copyright holders choose to participate or not, this exploitation model will serve
as a strong proof-of-concept for making paper catalogues more useful and accessible,
within a commercial environment, but copyright remains an open challenge.
4 Open Challenges
As expected, when the shape-based algorithm was first tested with real archaeological
pottery sherds for the first time, the system did not operate well. Furthermore, as the
network was trained on sherds generated from profile drawings, the classification was
not robust enough to handle small variations that can be seen in sherds observed in
practice. Both problems are currently being addressed in the research carried out by
partners at Tel Aviv University. Currently, the process for appearance-based
recognition is totally integrated in the mobile app, whereas the process for shape-
based recognition is still undergoing improvement, but already permits the automated
extraction of a profile from the image of a sherd.
As mentioned in the previous section, there are considerable challenges with
regard to open licensing and intellectual property rights. As the project has progressed
it has become evident that the comparative data necessary to the implement the
ArchAIDE app must be derived from a variety of sources, each with different
advantages and restrictions. For example, the online comparative collection Roman
Amphorae: A digital resource, held by the Archaeology Data Service, or an analogue
equivalent might be a particular comparative paper catalogue for Majolica of
Montelupo. In the first example, while the data creators retain copyright, the
comparative collection is already freely and openly disseminated online via a deposit
agreement between copyright holder and the Archaeology Data Service, and can
therefore be incorporated into the ArchAIDE app without needing to derive further
permissions from the copyright holders. This is not the case for the paper catalogue
described in the second example, where conversion into a dynamic digital resource
was never envisioned.
While useful tools to help digitise the paper catalogues necessary to show the
technical proof of concept of the ArchAIDE app have been developed by CNR, this
does not mean the ArchAIDE project necessarily now holds copyright to the newly
digitised, remixed data (although the metadata created as part of this process by the
ArchAIDE project can be argued to be new data, for which the project can claim
copyright). Whether this data can be made available outside the proof of concept
would need to be negotiated with each copyright holder, which represents a major
logistical and (potentially) financial difficulty. This becomes even more complicated
if the ArchAIDE app is monetised in any way. The issue cannot of course be solved
by ArchAIDE, but instead provides another important proof of concept opportunity
by the project. By showing the potential of digitising paper catalogues in a way that
demonstrates how their content can be actively re-used, allows ArchAIDE to now
open discussions with publishers and other data providers around the importance of
making their resources available in new ways with a concrete example (seeing their
data in use within the app), furthering the long-term discourse around making
research data open and accessible.
64
5 Tools
While the full range of tools described in this paper will be freely and openly
available by the completion of the ArchAIDE project (subject to copyright), the
following aspects are already completed:
Mobile App: The following features are currently functional within the mobile app:
● the appearance-based recognition workflow (i.e. the automatic recognition of
pottery through the decoration) is complete
● the shape-based recognition system allows for the correction of the white
balance (automatically and manually), scaling of the image using a ruler
(automatically and manually), the extraction of the profile of a potsherd
(semi-automatically)
● the information contained into the reference database can be searched and
visualised
● a personal profile can be created within which archaeologists can store data
related to assemblages of sherds from their own excavations
Desktop App: The following features are currently functional within the web-based,
desktop app:
● the information contained into the reference database can be searched and
visualised, in particular the geolocation of the origin and the occurrences of
each pottery type, 3D models of each pottery type that can be interrogated,
the relationships between different sherds (for instance) types and stamps
● the appearance--based recognition tool (i.e. the automatic recognition of
pottery through the decoration)
Data: The following data is currently available by request from the authors:
● the multilingual thesaurus of descriptive pottery terms is available in JSON
format. The terms are mapped to the Getty Art and Architecture Thesaurus
and include French, German, Spanish, Catalan, English and Italian.
References
1. Home - ArchAIDE. http://www.archaide.eu/.
2. Gualandi ML, Scopigno R, Wolf L, Richards J, Buxeda i Garrigos J, Heinzelmann M,
Hervas MA, Vila L, Zallocco M (2016) ArchAIDE-Archaeological Automatic
Interpretation and Documentation of cEramics. In: EUROGRAPHICS Workshop on
Graphics and Cultural Heritage. The Eurographics Association. doi:10.2312/gch.20161408
3. Banterle F, Dellepiane M, Evans T, Gattiglia G, Itkin B, Zallocco M (2017) The
ArchAIDE Project: results and perspectives after the first year. In: EUROGRAPHICS
Workshop on Graphics and Cultural Heritage. doi:10.2312/gch.20171308
65