=Paper=
{{Paper
|id=Vol-1472/IESD_2015_paper_8
|storemode=property
|title=DAICA - Digital Assistant Investigating Cultural Assets
|pdfUrl=https://ceur-ws.org/Vol-1472/IESD_2015_paper_8.pdf
|volume=Vol-1472
|dblpUrl=https://dblp.org/rec/conf/semweb/HotzCPPRB15
}}
==DAICA - Digital Assistant Investigating Cultural Assets==
https://ceur-ws.org/Vol-1472/IESD_2015_paper_8.pdf
DAICA - Digital Assistant Investigating
Cultural Assets
Lothar Hotz1 , Dan Cristea2 , Justyna Pietrzak3 , Martin Povazay4 , Brigitte
Rauter4 , and Daniela Buleandra5
1
HITeC e.V. c/o University of Hamburg, Germany,
hotz@informatik.uni-hamburg.de
2
University Alexandru Ioan Cuza and Romanian Academy, Romania,
dcristea@info.uaic.ro
3
Eleka Ingeniaritza Linguistikoa S.L., Spain, justyna@eleka.net
4
P.Solutions Informationstechnologien GmbH, Austria,
{martin.povazay,brigitte.rauter}@psolutions.at
5
SIVECO Romania SA, Romania, Daniela.Buleandra@siveco.ro
Abstract. Besides web pages, the web offers access to an immense va-
riety of digitized source material, inventories and catalogues hosted by
libraries and archives relevant for humanities and social sciences (HSS)
studies. In practice, a remote access to HSS information is consider-
ably hampered by several barriers: Researchers interested in a specific
topic do not know which institution harbors information related to a
specific topic; Data collections are equipped with unique user interfaces
and offer different data structures; Language barriers impede informa-
tion exploitation; Retrieval mechanisms do not provide intelligent access
to semantically related information. In this paper, we describe an Dig-
ital Assistant Investigating Cultural Assets (DAICA) for research and
information procurement in HSS, guided by the vision of a digital in-
formation space of cultures. The DAICA will support HSS studies by
autonomously identifying appropriate resources and presenting topical
investigation results. In particular, the DAICA will integrate technol-
ogy and provide a solution for analysing historical digitized documents,
performing semantical search in deep data structures, automatic transla-
tion, extending a search by meaningful relations, creating summaries of
identified resources, and providing user interactions for complex search
results.
Keywords: Semantic search, machine translation, summarization, op-
tical character recognition, cultural heritage
1 Introduction
The web offers an immense variety of digital or digitized source material, in-
ventories and catalogues relevant for studies in humanities and social sciences
(HSS). Institutions such as libraries, museums, archives, local and regional au-
thorities, parliamentary and media documentation services provide a wealth of
2
information, organized on local websites and in principle accessible for inquiries
and research from anywhere in the Internet, however not by ordinary search
engines - they reside in the ”deep web” and require special access methods.
Resources exist in a multitude of media types: unstructured and structured
text, with and without metadata, full text accessible or not, images, and videos.
Media comprise historical or cultural texts, biographical information, newspa-
pers articles, maps of cities, regions, and countries, paintings, photographs, and
much more. In addition to varying data formats, the user interfaces to these re-
sources differ widely, both regarding language and functionality. Hence, research
is difficult, and the outcomes are all too often incomplete and insufficient. Typ-
ically, tedious manual investigations are needed to study a specific topic: one
has to get into contact with many possible sources, get acquainted with access
modalities, retrieve data, commonly by language-specific key-word search, deter-
mine relevance across language barriers, and summarise the resulting material.
In HSS studies, the task is further aggravated because of diverse cultural re-
gions, each with its own history and tradition, and hence, often with a different
understanding of seemingly identical terms.
This situation is faced by scholars, students, journalists, public, and media
if they want to investigate abroad data sources, not only data of their local
libraries. In principle the same applies to investigations by business and com-
mercial aggregators. There is clearly the need to pave the way for barrier-free
access, i.e., support in accessing distributed repositories, in translation services,
in fast interpretation of new unknown resources, and collaboration with other
interested users about this matter to cultural data.
Fortunately, basic technology for multilingual and semantically enhanced
search in multimedia databases is available. But in order to effectively sup-
port HSS studies, techniques have to be adapted and integrated. For example,
relevance criteria including temporal and geographical proximity, cultural vicin-
ity, or taxonomical distance of terms must be taken into account for a semantic
search to be effective. Optical character recognition (OCR) must be invoked
for searching scanned documents; Machine translation must be used for iden-
tifying, linking and presenting relevant multilingual data, summarisation and
visualization and user-device interaction facilities must be provided for complex
and multifaceted data. Altogether, these methods can allow effective retrieval of
historical and cultural data from the deep web.
As a technological innovation, this paper presents a concept of an intelligent
HSS research assistant called Digital Assistant Investigating Cultural Assets,
DAICA, which will be used by scholars of the humanities and by the interested
public or the industry having the need for investigation a specific topic. Sum-
marisations will be returned of semantically related documents, articles, texts
not only of web-pages but also of other data sources.
This paper describes the use cases of a DAICA (Section 2), a new search
procedure in Section 3 integrating semantic search, machine translation, OCR,
entity discorvery, and summarization. These technologies are integrated in a
configurable framework (Section 4 and Section 5).
3
2 A Use Case of DAICA
Figure 1 presents a use case of a digital assistant presented in this paper with
user and background interactions processed by DAICA. Main features are the
proactive discovering of the user’s interest topics (“A1”), the background actions
based on localization if the users is moving (“A2”) and on OCR for interpreting
original ancient documents. By further observing the user’s writing activities,
DAICA computes related sources in the web and, hence, supports the user by
his investigation activities (“A3”).
Fig. 1. Use Case “New Type of Assisted HSS Study”
4
3 A New Integrated Search Procedure
A main task of DAICA is the transparent integration of semantic search, machine
translation, and all other capabilities such as summarization, OCR, and entity
discovery (see Figure 2 which illustrates the complete process). A user spells out
implicitly or explicitly the search specification (query) in his/her language, the
source language, in the example German. DAICA translates this query and its
ontological enhancements into the target language (here Romanian) of a data
source and starts the search. The results and their enhancements through links
and summarisation are again translated back into the source language. In the
following section, the details about the involved components are given.
Fig. 2. Process of a search session with ontological enhancement, DAICA instantiation
(see next section) retrieval, and machine translation
5
4 The DAICA Framework
In order to obtain sustainable, reusable results the objective of DAICA is to build
a general framework which will be used for the development of complex spe-
cialised architectures, each accommodating different capabilities, selected dur-
ing configuration sessions. These capabilities realize the basic technologies for
investigation tasks, i.e., optical character recognition (OCR) for enabling the
translation of text images into words, search which takes the meaning of queries
and documents into account, machine translation for interpreting documents in
foreign languages, summarisation for getting a quick overview of a document or
article, entity and link discovery for identifying important persons and subjects
in a text. However, it is not evident which capabilities to use at which time, or,
if capabilities come in variants, which version is best for a given investigation
task.
Therefore, DAICA is defined as a framework, its infrastructure, and a suitable
interface technology which can be used to interactively assemble architectures
for selecting suitable components which implement the capabilities for a given
investigation task. Various kinds of users, ”aggregators”, will use this kit to
build DAICA configurations that best support their own needs or meet the
investigation requirements of others.
DAICA instantiations will constitute another layer of outputs resulting
from DAICA interactions. A DAICA instantiation (or instance) represents the
combination between a DAICA configuration and a specific set of acquired re-
sources from different data sources (usually referring to a specific topic a certain
user has worked on). These resources will be accumulated by a user (or a commu-
nity of users) during a series of work sessions with DAICA. Hence, all interactions
typically with specific investigation goals and directed to specific resources, are
stored, catalogued and offered for post-research re-use, for the benefit of their
creators or future users. Hence, DAICA instantiations are collections of resources
with respect to a topic.
Examples of DAICA configurations can be:
– DAICA-1: Capability to process contemporary German, Spanish, English
and Romanian, with OCR, indexing, external linking of name entities, sum-
marisation, and translation between these four languages;
– DAICA-2: Processing German and Romanian texts from 1850 to present
date, OCR including the Gothic German and the transitional Cyrillic alpha-
bet used in Romania in the middle of the XIX-th century, indexing and exter-
nal linking of name entities, time expressions, summarisation, and translation
between these two languages.
Examples of DAICA instantiations can be:
– Based on DAICA-2: Links to the bibliographical sources in the Library of
Hamburg and Academy Library of Bucharest, knowledge-base with dated
entries related to the migration in Germany and Romania in the XIX-th
century;
6
– Based on DAICA-1: Links of German academic libraries with information
from Basque archives, in relation to investigations accomplished by German
scholars in the Basque Country in XIX century.
Hence, an instantiation summarizes all information about a specific topic,
e.g., content identified in some libraries, notes made by the user. Furthermore,
if made public, other users can make use of and refine such previously created
instantiations through the DAICA instantiation retrieval mechanism. As such,
the DAICA instantiations are the base for building a community discussing and
further developing cultural topics.
DAICA uses a number of already existent technologies, which will be adapted
to comply with the actual requirements. The DAICA capabilities include the
following features:
– Specification and customization of the investigation task (proactive and trig-
gered search specifications);
– Specification of data sources, their access, and their content in the form
of metadata schemas, languages, or ontologies and used terminology, hence
enabling access to foreign data and content without the need of manually
traversing user interfaces or interpreting a library structure (by data source
profiles);
– Analysis of ancient digitized documents (by pattern recognition and OCR,
word spotting);
– Deep semantic search through data sources (by indexing and semantic search);
– Automatic translation of queries and resources (by machine translation);
– Linking of resources with expressive relationships on the basis of semantic
entities (by entity identification, reference resolution, detection of temporal
and spatial relations);
– Creating summaries of the identified resources (by automatic summarisa-
tion);
– Friendly end-user interfaces for the visualization of complex search results
and dependencies in the Web for different types of devices: laptop, tablet,
smart phone (by innovative visualization and user-device interaction);
– Easy configuration of new processing architectures to support a wide range
of thematic investigations (by configuration facilities);
– Projects profiling for storing, retrieving and sharing of resources as DAICA
instances (by instantiation facilities).
In summary, these features will be integrated in a generally applicable and cus-
tomizable technological framework that will allow easy configuration of new ar-
chitectures in order to help researchers and other categories of users to perform
assisted cultural HSS investigations. Once installed in a DAICA platform, the
framework can be used by aggregators (libraries, research institutes, administra-
tion) to configure new applications that will allow public users to get access to
new data or to administer previously curated DAICA instantiations.
7
5 Technologies for DAICA
5.1 Existing investigation tools
The widely used Aleph integrated library system provides academic, research,
and national libraries with the efficient, user-friendly tools and workflow sup-
port they need to meet the increasing requirements of the industry today and
in the future. Built on an Oracle database, Aleph runs on a range of operat-
ing systems. Employing system-wide XML technology, Aleph offers third-party
integration through an XML gateway. The product is based on industry stan-
dards, offering the ultimate in resource-sharing capabilities, full connectivity,
and seamless interaction with other systems and databases.
Another solution used in libraries is DigiTool, which enables academic li-
braries and library consortia to manage and provide access to digital resources,
both those that are created for use within the institution and those that are
collected and maintained by the library for the benefit of the public.
Since many resources have a public exposure on the Web, other existing inves-
tigation tools or techniques which can be used for searching are the Web search
engines and crawlers. Some open source or commercial tools (which can influ-
ence the solution) are: Datapark, ebhath, Eureka, Indri, ISearch, IXE, Lucene,
Managing Gigabytes (MG), MG4J, mnoGoSearch, MPS Information Server, Na-
mazu, Nutch, Omega, OmniFind IBM Yahoo! Ed., OpenFTS, PLWeb, SWISH-
E, SWISH++, Terrier, WAIS/ freeWAIS, WebGlimpse, XML Query Engine,
XMLSearch, Zebra, BBDBot and Zettair.
Besides search itself, one technique to be used when combining HSS data
from multiple sources is data integration. State of the art approaches for data
integration have adopted a schema-first (e.g., ETL, enterprise integration), a
schema-never (e.g., search engines), or a schema-later (e.g., dataspaces) method-
ology.
Such tools provide the basic search and data access interfaces to library
content. For DAICA, libraries operating those tools can and will be integrated
through data source profiles. Furthermore, the provided search facilities of the
tools will be used by the semantic search capability to perform keyword-based
search.
A lot cultural assets are currently published through EUROPEANA. EURO-
PEANA bases its search functionalities on who, what, where, when and corre-
sponding restrictions for media type, language, country, and provider. DAICA
will base the search on semantic ontologies and multilingual access, thus, fa-
cilitating the document access for users. However, through the envisioned data
source profiles, EUROPEANA can be integrated in the DAICA framework and,
thus, be part of a DAICA investigation.
5.2 OCR, pattern recognition
For identification and retrieval of digitzed but not yet recognized documents,
DAICA includes OCR (Optical Character Recognition) tools. The main chal-
lenges which have to be faced are the following:
8
– OCR must be based on a variety of historical fonts and spellings.
– Document images may have poor quality and may require image enhance-
ment.
– Character and word recognition may be ambiguous due to noise.
– Word, sentence and semantic context must be exploited for disambiguation.
There exist several commercial and open source OCR tools, which perform high-
quality OCR (up to 99%) for standard fonts and low-noise conditions [1]. On
the other hand, character and word recognition results may be quite poor (be-
low 80%) without prior knowledge of the font and without exploiting context
information.
Hence, DAICA applies several innovative techniques to achieve high-quality
OCR. First, OCR tasks are supported by their semantic context using meta-data
and ontologies. Hence, ambiguities can be significantly reduced. For example,
ambiguous readings can be refuted if the semantic distance (computed from
an ontology such as WordNet) to the investigation topic exceeds a threshold.
As a second innovative technique, applicable to manuscripts or unusual fonts,
DAICA will allow word spotting based on patterns supplied by the investigator.
This way, occurrences of similar patterns can be retrieved from a document. A
third technique, mainly applicable to handwritten documents, will be the use of
an advanced text-line finder which can cope with varying line orientations.
Thus, the approach for DAICA will be mainly based on existing OCR tools
of the partners and open source tools, as well as low-level and context-supported
computer vision and manuscript analysis [2,3,4].
5.3 Semantic search
A central goal of the DAICA is to provide support for studies of cultural heritage
by extending keyword-based search to a much broader search based on semantic
relations. A semantic search has the advantage of narrowing down ambiguous
word meanings, especially across language barriers, and allowing proactive back-
ground search for related information. This goal can be achieved by a variety
of techniques which try to take the intention of the user and the meaning of a
query into account when searching in data sources.
There exist several approaches for semantic search as documented, for ex-
ample, in the surveys, [5,6,7]. DAICA lays the focus on exploiting ontological
information which are used in two fundamental ways: (i) to define, refine and
expand the query topic; (ii) to find semantically related information in data
sources.
Several publicly accessible implementations of semantic search approaches
exist, including QWant, GoPubMed, Swoogle, and Google’s Knowledge Graph,
which deal with specific kinds of ontological representations. These techniques,
however, do not meet the requirements for the intelligent agent conceived in
this work: (i) DAICA will have to access a large number of heterogeneous con-
tent structures used in the archiving institutions for cultural heritage or similar
9
data aggregations. Some may be supported by full-fledged Semantic Web on-
tologies, others by customized categorization schemes. In consequence it will be
necessary to invoke ontology alignment in some form; (ii) Search will be mul-
tilingual, crossing language barriers between the user and information sources;
(iii) In DAICA, the user can define a query by several kinds of topic descriptions,
ranging from keywords, annotated images, graphic patterns, to coherent texts.
Hence several heterogeneous measures for semantical distance will play a part,
for example taxonomical distance, relatedness by names, time or geographical
location, or chains of ontological structures; (iv) The user will be supported by
proactive search, i.e., by autonomous background explorations through entity
and link discovery in user’s text writing; (v) Access to DAICA will be possible
via mobile devices, and rendition of results will include summarisation.
The software for individual techniques is mostly available either as open
source or detained by the authors. The main task for the DAICA is to conceive
and integrate a tool combining the techniques in a user-friendly way.
5.4 Ontology management
In our approach, ontologies play an important role for obtaining meaningful
search results in support of a user’s investigation. All essential DAICA function-
alities resort to ontologies, in particular semantic search, language translation,
interpreted OCR, entity discovery, topical linking, and summarisation. Ontolo-
gies may provide concept names and definitions in terms of relations to other
concepts, for example generalization, specialization, synonyms and antonyms.
Standardized properties relate entities to important search criteria, such as lo-
cation and time.
Due to the highly heterogeneous data sources of the cultural heritage and
diverse evolved standards, investigations with DAICA have to cope with mul-
tiple ontologies in different languages, ranging from carefully designed OWL
ontologies to simple databases characterized by metadata schemes. In order to
determine the relevance of resources for a user query, DAICA must be able
to align these ontologies with the semantics of the query. Several methods for
query answering based on multiple and multilingual ontologies have been devel-
oped in the past decade, see [8,9,10] for surveys. Typically, there is a matching
(or alignment) step where correspondences between heterogeneous ontologies are
determined, and an interpretation step, where information relevant for a query
is extracted.
In the DAICA infrastructure, ontology matching and interpretation will be
performed for ontologies based on standards such as Dublin Core or Schema.org,
on controlled vocabularies (WordNet and thesaurus vocabularies), and on exist-
ing biographical data standards and classifications. In several countries data
sources are described by authority files of standardized metadata, in Germany:
Gemeinsame Normdatei, beacon files, gazetteer data and links, files for Common
public corporation data (GKD, Gemeinsame Körperschaftsdatei) with company
and institution names, registers with personal names (PND), and Common norm
10
data file (SWD, Schlagwortnormdatei) with commonly used tag words, cate-
gories, and subject headings. Mass data with named entity tagging and recogni-
tion data will enhance the scope of results and open up semantic relations and
links to more resources.
5.5 DAICA instantiation retrieval
A DAICA instantiation represents a DAICA configuration and the resources
acquired by a user or a community of users having close scientific interests for a
specific topic using this configuration. As such, a DAICA instantiation represents
a complete investigation case which is both, a useful documentation for the
investigator and a valuable resource for similar investigations of other users. It
is the objective of DAICA to support all users of the DAICA community by a
case base of instantiations and case-based retrieval mechanisms.
Case-based information retrieval is a well-established technology, see [11,12]
for surveys. While case-based retrieval has been originally conceived for feature-
based object representations, applications to relational structures have proved
quite successful [13]. More recently, case-based retrieval was further enhanced
by ontology-based representations and corresponding similarity measures [14].
During the development of DAICA, a special theoretical attention will be given
to an ontological organisation of the collections of DAICA instantiations. For
example, issues of interest here are: demarcation strategies (when is it that two
instances have to be considered as identical or distinct?), inheritance (is it that
an instantiation A inherits parts of descriptions, sources, links, etc. from an
instance B?).
5.6 Machine translation, multilingual processing in combination
with semantic search and summarisation
Developing of efficient machine translation is a long-lasting and multi-level pro-
cess. DAICA uses a mixture of the mature technologies of statistical, example-
based and and rule-based machine translation (SMT and RBMT). As basic fea-
tures, DAICA includes:
– Resource collection (semi-automatic parallel corpora extraction and dictio-
nary building, with special emphasis on lesser-resourced languages and in-
domain registers). These data will be used for training and tuning machine
translation modules.
– Development of the query translation module. Previous experience and ex-
pertise of the partners will be used for adapting existing methods to language
pairs of DAICA. SMT is language-independent, and the same toolkit can be
used for any pair of languages provided specific single language texts and
parallel texts for all language translation pairs exist. But the state of the
development rule-based machine translation (RBMT) varies, depending on
the language pair.
11
– DAICA will use the Apertium software [15]. Apertium is a classical shallow-
transfer or transformer system, released under GNU Licence. Apertium in-
cludes dictionaries for language pairs involving Spanish. The Apertium MT
engine consists of the pipelined modules for morphological anlaysis, part-of-
speech tagger, and text generators as well as Statistical Machine Translation
(SMT) based on the Moses toolkit [16].
Hence, our summariser is multilingual at the architectural level, meaning that
it incorporates a pipeline of modules which has the same structure irrespective of
the language of the processed document. However, initial elements of this chain
(among which, the tokeniser, the POS-tagger, the lemmatiser, the NP-chunker,
the clause splitter, the name entity recogniser, and the anaphora resolver) are
strongly language dependent.
In the former ATLAS project6 , summarisers for Bulgarian, German, Greek,
Polish and Romanian have been built, meaning that our general summarisation
architecture has been adapted for all these languages by assembling basic-levels
NLP modules supplied by partners. For DAICA, we will build summarisers for
German, English, and Romanian, by re-using (and, where necessary, also enhanc-
ing) the German and Romanian basic components and including open-source
modules for English.
The situation is somehow different when the language of the documents is
old. Romanian or, for instance, has changed dramatically over time. Not only the
lexica, grammar and syntax have evolved, but also the alphabet has changed from
Old Cyrillic to Latin, with a mixture of the two, called the Cyrillic Transition
Alphabet, used for a period in the middle of the XIX-th century. Based on
previous work [17,18] we will study on diachronic Romanian morphology.
5.7 Entity and link discovery
Recognition of entity mentions in texts (names of people, moments of time,
countries, locations, events, organizations) and their correct interpretation in
context is an issue of primary importance in DAICA. These mentions should
open access gates to entries in the collection of accessible resources. Examples
for points of interests are:
– Identify entity mentions in metadata field values and full texts and, if neces-
sary, do their ontological interpretations, e.g., identify temporal entities and
historical dates and events;
– Identify relevant relations between entities such as relations between in-
stances: is-in (at