Besides web pages, the web o ers access to an immense variety of digitized source material, inventories and catalogues hosted by libraries and archives relevant for humanities and social sciences (HSS) studies. In practice, a remote access to HSS information is considerably hampered by several barriers: Researchers interested in a speci c topic do not know which institution harbors information related to a speci c topic; Data collections are equipped with unique user interfaces and o er di erent data structures; Language barriers impede information exploitation; Retrieval mechanisms do not provide intelligent access to semantically related information. In this paper, we describe an Digital Assistant Investigating Cultural Assets (DAICA) for research and information procurement in HSS, guided by the vision of a digital information space of cultures. The DAICA will support HSS studies by autonomously identifying appropriate resources and presenting topical investigation results. In particular, the DAICA will integrate technology and provide a solution for analysing historical digitized documents, performing semantical search in deep data structures, automatic translation, extending a search by meaningful relations, creating summaries of identi ed resources, and providing user interactions for complex search results.
information, organized on local websites and in principle accessible for inquiries and research from anywhere in the Internet, however not by ordinary search engines - they reside in the "deep web" and require special access methods.
Resources exist in a multitude of media types: unstructured and structured text, with and without metadata, full text accessible or not, images, and videos. Media comprise historical or cultural texts, biographical information, newspapers articles, maps of cities, regions, and countries, paintings, photographs, and much more. In addition to varying data formats, the user interfaces to these resources di er widely, both regarding language and functionality. Hence, research is di cult, and the outcomes are all too often incomplete and insu cient. Typically, tedious manual investigations are needed to study a speci c topic: one has to get into contact with many possible sources, get acquainted with access modalities, retrieve data, commonly by language-speci c key-word search, determine relevance across language barriers, and summarise the resulting material. In HSS studies, the task is further aggravated because of diverse cultural regions, each with its own history and tradition, and hence, often with a di erent understanding of seemingly identical terms.
This situation is faced by scholars, students, journalists, public, and media if they want to investigate abroad data sources, not only data of their local libraries. In principle the same applies to investigations by business and commercial aggregators. There is clearly the need to pave the way for barrier-free access, i.e., support in accessing distributed repositories, in translation services, in fast interpretation of new unknown resources, and collaboration with other interested users about this matter to cultural data.
Fortunately, basic technology for multilingual and semantically enhanced search in multimedia databases is available. But in order to e ectively support HSS studies, techniques have to be adapted and integrated. For example, relevance criteria including temporal and geographical proximity, cultural vicinity, or taxonomical distance of terms must be taken into account for a semantic search to be e ective. Optical character recognition (OCR) must be invoked for searching scanned documents; Machine translation must be used for identifying, linking and presenting relevant multilingual data, summarisation and visualization and user-device interaction facilities must be provided for complex and multifaceted data. Altogether, these methods can allow e ective retrieval of historical and cultural data from the deep web.
As a technological innovation, this paper presents a concept of an intelligent HSS research assistant called Digital Assistant Investigating Cultural Assets, DAICA, which will be used by scholars of the humanities and by the interested public or the industry having the need for investigation a speci c topic. Summarisations will be returned of semantically related documents, articles, texts not only of web-pages but also of other data sources.
This paper describes the use cases of a DAICA (Section 2), a new search procedure in Section 3 integrating semantic search, machine translation, OCR, entity discorvery, and summarization. These technologies are integrated in a con gurable framework (Section 4 and Section 5).
A main task of DAICA is the transparent integration of semantic search, machine translation, and all other capabilities such as summarization, OCR, and entity discovery (see Figure 2 which illustrates the complete process). A user spells out implicitly or explicitly the search speci cation (query) in his/her language, the source language, in the example German. DAICA translates this query and its ontological enhancements into the target language (here Romanian) of a data source and starts the search. The results and their enhancements through links and summarisation are again translated back into the source language. In the following section, the details about the involved components are given.
In order to obtain sustainable, reusable results the objective of DAICA is to build a general framework which will be used for the development of complex specialised architectures, each accommodating di erent capabilities, selected during con guration sessions. These capabilities realize the basic technologies for investigation tasks, i.e., optical character recognition (OCR) for enabling the translation of text images into words, search which takes the meaning of queries and documents into account, machine translation for interpreting documents in foreign languages, summarisation for getting a quick overview of a document or article, entity and link discovery for identifying important persons and subjects in a text. However, it is not evident which capabilities to use at which time, or, if capabilities come in variants, which version is best for a given investigation task.
Therefore, DAICA is de ned as a framework, its infrastructure, and a suitable interface technology which can be used to interactively assemble architectures for selecting suitable components which implement the capabilities for a given investigation task. Various kinds of users, "aggregators", will use this kit to build DAICA con gurations that best support their own needs or meet the investigation requirements of others.
DAICA instantiations will constitute another layer of outputs resulting from DAICA interactions. A DAICA instantiation (or instance) represents the combination between a DAICA con guration and a speci c set of acquired resources from di erent data sources (usually referring to a speci c topic a certain user has worked on). These resources will be accumulated by a user (or a community of users) during a series of work sessions with DAICA. Hence, all interactions typically with speci c investigation goals and directed to speci c resources, are stored, catalogued and o ered for post-research re-use, for the bene t of their creators or future users. Hence, DAICA instantiations are collections of resources with respect to a topic.
Examples of DAICA con gurations can be: { DAICA-1: Capability to process contemporary German, Spanish, English and Romanian, with OCR, indexing, external linking of name entities, summarisation, and translation between these four languages; { DAICA-2: Processing German and Romanian texts from 1850 to present date, OCR including the Gothic German and the transitional Cyrillic alphabet used in Romania in the middle of the XIX-th century, indexing and external linking of name entities, time expressions, summarisation, and translation between these two languages.
Examples of DAICA instantiations can be: { Based on DAICA-2: Links to the bibliographical sources in the Library of Hamburg and Academy Library of Bucharest, knowledge-base with dated entries related to the migration in Germany and Romania in the XIX-th century; { Based on DAICA-1: Links of German academic libraries with information from Basque archives, in relation to investigations accomplished by German scholars in the Basque Country in XIX century.
Hence, an instantiation summarizes all information about a speci c topic, e.g., content identi ed in some libraries, notes made by the user. Furthermore, if made public, other users can make use of and re ne such previously created instantiations through the DAICA instantiation retrieval mechanism. As such, the DAICA instantiations are the base for building a community discussing and further developing cultural topics.
DAICA uses a number of already existent technologies, which will be adapted to comply with the actual requirements. The DAICA capabilities include the following features: { Speci cation and customization of the investigation task (proactive and triggered search speci cations); { Speci cation of data sources, their access, and their content in the form of metadata schemas, languages, or ontologies and used terminology, hence enabling access to foreign data and content without the need of manually traversing user interfaces or interpreting a library structure (by data source pro les); { Analysis of ancient digitized documents (by pattern recognition and OCR, word spotting); { Deep semantic search through data sources (by indexing and semantic search); { Automatic translation of queries and resources (by machine translation); { Linking of resources with expressive relationships on the basis of semantic entities (by entity identi cation, reference resolution, detection of temporal and spatial relations); { Creating summaries of the identi ed resources (by automatic summarisation); { Friendly end-user interfaces for the visualization of complex search results and dependencies in the Web for di erent types of devices: laptop, tablet, smart phone (by innovative visualization and user-device interaction); { Easy con guration of new processing architectures to support a wide range of thematic investigations (by con guration facilities); { Projects pro ling for storing, retrieving and sharing of resources as DAICA instances (by instantiation facilities).
In summary, these features will be integrated in a generally applicable and customizable technological framework that will allow easy con guration of new architectures in order to help researchers and other categories of users to perform assisted cultural HSS investigations. Once installed in a DAICA platform, the framework can be used by aggregators (libraries, research institutes, administration) to con gure new applications that will allow public users to get access to new data or to administer previously curated DAICA instantiations. 5.1
The widely used Aleph integrated library system provides academic, research, and national libraries with the e cient, user-friendly tools and work ow support they need to meet the increasing requirements of the industry today and in the future. Built on an Oracle database, Aleph runs on a range of operating systems. Employing system-wide XML technology, Aleph o ers third-party integration through an XML gateway. The product is based on industry standards, o ering the ultimate in resource-sharing capabilities, full connectivity, and seamless interaction with other systems and databases.
Another solution used in libraries is DigiTool, which enables academic libraries and library consortia to manage and provide access to digital resources, both those that are created for use within the institution and those that are collected and maintained by the library for the bene t of the public.
Since many resources have a public exposure on the Web, other existing investigation tools or techniques which can be used for searching are the Web search engines and crawlers. Some open source or commercial tools (which can in uence the solution) are: Datapark, ebhath, Eureka, Indri, ISearch, IXE, Lucene, Managing Gigabytes (MG), MG4J, mnoGoSearch, MPS Information Server, Namazu, Nutch, Omega, OmniFind IBM Yahoo! Ed., OpenFTS, PLWeb, SWISHE, SWISH++, Terrier, WAIS/ freeWAIS, WebGlimpse, XML Query Engine, XMLSearch, Zebra, BBDBot and Zettair.
Besides search itself, one technique to be used when combining HSS data from multiple sources is data integration. State of the art approaches for data integration have adopted a schema- rst (e.g., ETL, enterprise integration), a schema-never (e.g., search engines), or a schema-later (e.g., dataspaces) methodology.
Such tools provide the basic search and data access interfaces to library content. For DAICA, libraries operating those tools can and will be integrated through data source pro les. Furthermore, the provided search facilities of the tools will be used by the semantic search capability to perform keyword-based search.
A lot cultural assets are currently published through EUROPEANA. EUROPEANA bases its search functionalities on who, what, where, when and corresponding restrictions for media type, language, country, and provider. DAICA will base the search on semantic ontologies and multilingual access, thus, facilitating the document access for users. However, through the envisioned data source pro les, EUROPEANA can be integrated in the DAICA framework and, thus, be part of a DAICA investigation. 5.2
For identi cation and retrieval of digitzed but not yet recognized documents,
DAICA includes OCR (Optical Character Recognition) tools. The main
challenges which have to be faced are the following:
{ OCR must be based on a variety of historical fonts and spellings.
{ Document images may have poor quality and may require image
enhancement.
{ Character and word recognition may be ambiguous due to noise.
{ Word, sentence and semantic context must be exploited for disambiguation.
There exist several commercial and open source OCR tools, which perform
highquality OCR (up to 99%) for standard fonts and low-noise conditions [
Hence, DAICA applies several innovative techniques to achieve high-quality OCR. First, OCR tasks are supported by their semantic context using meta-data and ontologies. Hence, ambiguities can be signi cantly reduced. For example, ambiguous readings can be refuted if the semantic distance (computed from an ontology such as WordNet) to the investigation topic exceeds a threshold. As a second innovative technique, applicable to manuscripts or unusual fonts, DAICA will allow word spotting based on patterns supplied by the investigator. This way, occurrences of similar patterns can be retrieved from a document. A third technique, mainly applicable to handwritten documents, will be the use of an advanced text-line nder which can cope with varying line orientations.
Thus, the approach for DAICA will be mainly based on existing OCR tools
of the partners and open source tools, as well as low-level and context-supported
computer vision and manuscript analysis [
A central goal of the DAICA is to provide support for studies of cultural heritage by extending keyword-based search to a much broader search based on semantic relations. A semantic search has the advantage of narrowing down ambiguous word meanings, especially across language barriers, and allowing proactive background search for related information. This goal can be achieved by a variety of techniques which try to take the intention of the user and the meaning of a query into account when searching in data sources.
There exist several approaches for semantic search as documented, for
example, in the surveys, [
Several publicly accessible implementations of semantic search approaches exist, including QWant, GoPubMed, Swoogle, and Google's Knowledge Graph, which deal with speci c kinds of ontological representations. These techniques, however, do not meet the requirements for the intelligent agent conceived in this work: (i) DAICA will have to access a large number of heterogeneous content structures used in the archiving institutions for cultural heritage or similar data aggregations. Some may be supported by full- edged Semantic Web ontologies, others by customized categorization schemes. In consequence it will be necessary to invoke ontology alignment in some form; (ii) Search will be multilingual, crossing language barriers between the user and information sources; (iii) In DAICA, the user can de ne a query by several kinds of topic descriptions, ranging from keywords, annotated images, graphic patterns, to coherent texts. Hence several heterogeneous measures for semantical distance will play a part, for example taxonomical distance, relatedness by names, time or geographical location, or chains of ontological structures; (iv) The user will be supported by proactive search, i.e., by autonomous background explorations through entity and link discovery in user's text writing; (v) Access to DAICA will be possible via mobile devices, and rendition of results will include summarisation.
The software for individual techniques is mostly available either as open source or detained by the authors. The main task for the DAICA is to conceive and integrate a tool combining the techniques in a user-friendly way. 5.4
In our approach, ontologies play an important role for obtaining meaningful search results in support of a user's investigation. All essential DAICA functionalities resort to ontologies, in particular semantic search, language translation, interpreted OCR, entity discovery, topical linking, and summarisation. Ontologies may provide concept names and de nitions in terms of relations to other concepts, for example generalization, specialization, synonyms and antonyms. Standardized properties relate entities to important search criteria, such as location and time.
Due to the highly heterogeneous data sources of the cultural heritage and
diverse evolved standards, investigations with DAICA have to cope with
multiple ontologies in di erent languages, ranging from carefully designed OWL
ontologies to simple databases characterized by metadata schemes. In order to
determine the relevance of resources for a user query, DAICA must be able
to align these ontologies with the semantics of the query. Several methods for
query answering based on multiple and multilingual ontologies have been
developed in the past decade, see [
In the DAICA infrastructure, ontology matching and interpretation will be performed for ontologies based on standards such as Dublin Core or Schema.org, on controlled vocabularies (WordNet and thesaurus vocabularies), and on existing biographical data standards and classi cations. In several countries data sources are described by authority les of standardized metadata, in Germany: Gemeinsame Normdatei, beacon les, gazetteer data and links, les for Common public corporation data (GKD, Gemeinsame Korperschaftsdatei) with company and institution names, registers with personal names (PND), and Common norm data le (SWD, Schlagwortnormdatei) with commonly used tag words, categories, and subject headings. Mass data with named entity tagging and recognition data will enhance the scope of results and open up semantic relations and links to more resources. 5.5
A DAICA instantiation represents a DAICA con guration and the resources acquired by a user or a community of users having close scienti c interests for a speci c topic using this con guration. As such, a DAICA instantiation represents a complete investigation case which is both, a useful documentation for the investigator and a valuable resource for similar investigations of other users. It is the objective of DAICA to support all users of the DAICA community by a case base of instantiations and case-based retrieval mechanisms.
Case-based information retrieval is a well-established technology, see [
Developing of e cient machine translation is a long-lasting and multi-level
process. DAICA uses a mixture of the mature technologies of statistical,
examplebased and and rule-based machine translation (SMT and RBMT). As basic
features, DAICA includes:
{ Resource collection (semi-automatic parallel corpora extraction and
dictionary building, with special emphasis on lesser-resourced languages and
indomain registers). These data will be used for training and tuning machine
translation modules.
{ Development of the query translation module. Previous experience and
expertise of the partners will be used for adapting existing methods to language
pairs of DAICA. SMT is language-independent, and the same toolkit can be
used for any pair of languages provided speci c single language texts and
parallel texts for all language translation pairs exist. But the state of the
development rule-based machine translation (RBMT) varies, depending on
the language pair.
{ DAICA will use the Apertium software [
Hence, our summariser is multilingual at the architectural level, meaning that it incorporates a pipeline of modules which has the same structure irrespective of the language of the processed document. However, initial elements of this chain (among which, the tokeniser, the POS-tagger, the lemmatiser, the NP-chunker, the clause splitter, the name entity recogniser, and the anaphora resolver) are strongly language dependent.
In the former ATLAS project6, summarisers for Bulgarian, German, Greek, Polish and Romanian have been built, meaning that our general summarisation architecture has been adapted for all these languages by assembling basic-levels NLP modules supplied by partners. For DAICA, we will build summarisers for German, English, and Romanian, by re-using (and, where necessary, also enhancing) the German and Romanian basic components and including open-source modules for English.
The situation is somehow di erent when the language of the documents is
old. Romanian or, for instance, has changed dramatically over time. Not only the
lexica, grammar and syntax have evolved, but also the alphabet has changed from
Old Cyrillic to Latin, with a mixture of the two, called the Cyrillic Transition
Alphabet, used for a period in the middle of the XIX-th century. Based on
previous work [
Recognition of entity mentions in texts (names of people, moments of time, countries, locations, events, organizations) and their correct interpretation in context is an issue of primary importance in DAICA. These mentions should open access gates to entries in the collection of accessible resources. Examples for points of interests are: { Identify entity mentions in metadata eld values and full texts and, if necessary, do their ontological interpretations, e.g., identify temporal entities and historical dates and events; { Identify relevant relations between entities such as relations between instances: <person> is-in <place> (at <time>), <country> invades <country>, <person> signs <treaty>, etc. { Identify collections of documents having contingent content such as <document> is-primary-source-for <event>, <document> in-relation-with <event>, <document> mentions <person> (at <time>); 6 www.atlasproject.eu
One approach for entity discovery is the use of large repositories of entity names (gazetteers), such as person names, topics, locations, or temporal mentions. Ontologies and terminological databases can equally be used. This approach might look brute force, however, because of the existence of authority les and terminologies in library research, there is a huge amount of such entity storages and ontologies which can be used, similar to those mentioned in Section 5.4. Furthermore, we have recently built a large collection of regular expressions for the identi cation of geographical locations in free texts. As other means, larger contexts and syntactic analysis can be used to identify relations between entities. Once such relations are detected, the documents containing them can be tagged and indexed accordingly, providing information that can be used for intelligent retrieval. 5.8
Nowadays the quantity and diversity of data in the internet on whatever subject
is extremely vast, which makes it more and more di cult to enquire on speci c
subjects. The needed information is usually hidden in an ocean of garbage data.
This is one aspect of the well-known problem of information overload. One way
to deal with it is to use summarisation techniques. In [
Text summarisation methods can be classi ed into extractive and abstractive. An extractive summary includes sequences of words taken from the original document, which could be clauses, sentences or paragraphs. An abstractive summary does not reproduce sequences from the original document, but rather includes paraphrases of sections that mention important facts, events, and entities.
Many systems are known which perform automatic text summarisation, applying di erent techniques. Some use surface methods (involving no linguistic analysis but exploiting instead the format of the document), some take name entities from the original text as pivot elements and assume that the texts surrounding them is important and should stay in the summary (involving some kind of lexical analysis and classi cation methods); some relate signi cant features in the text and the summary and try to copy the ability to produce summaries from human-produced ones (involving learning and statistical methods); and some are based on discovering the discourse structure (involving processing at linguistic, syntactic and discourse level). The summarisation systems can also be considered from the point of view of the number of the processed texts, as single and multi-document, by the languages processed, as monolingual or multilingual, as well as by the genre of the processed texts.
The summarisation approach in DAICA is an extractive single-document
process producing general or focussed summaries. It will enhance the approach
described in [
In this paper, we presented a concept for a digital assistant which integrates and combines semantic technologies such as interpreted OCR, semantic search, summarization, entity and link discovery, machine translation, and case-based retrieval for supporting users in investigation and research tasks. As a main focus, the assistant consideres resources of cultural heritage data sources such as libraries. However, the underlying technologies allow the application of the DAICA concept to arbitray Internet sources such as web pages or social media data. This paper represents a preliminary step of re ning the conceptual and design principals before starting the actual development process of a new technology. However, the basic technologies which will be used for a complete DAICA system have been applied by us in similar approaches. We believe that the present day technologies, belonging to the domain of Arti cial Intelligence, that have attained a theoretical and applicational maturity can be combined in DAICA in a very creative way.