This discussion paper represents an extended abstract of a recent publication where we presented KIRA (Knowledge-based Information Retrieval from Art collections ), a system to query, browse and analyze cultural digital contents from a set of distributed and heterogeneous repositories. KIRA relies on a Big Data infrastructure with the following features: capability to gather information from di erent data sources; advanced data management techniques and technologies; ability to provide useful and personalized data to users based on their preferences and context. KIRA thus provides retrieval and presentation functionalities to search information of interest and present it to the users in a suitable format and according to their needs. Using ad-hoc APIs, our system can also support several applications: mobile multimedia guides, web portals to promote the Cultural Heritage, multimedia recommender and storytelling systems and so on. We discuss the main ideas that characterize the system, showing its use for several applications.
The enhancement and promotion of worldwide Cultural Heritage (CH) using
Information and Communication Technologies (ICT) represents nowadays an
important research issue, with a variety of potential applications. ICT have
radically changed the modern CH scenery: simple traditional Information Systems
for the management of cultural artifacts have left the place to complex systems
that expose rich information extracted from heterogeneous data sources (e.g.
Digital Libraries and Open Archives, Multimedia Art Collections, Social Media,
Web Encyclopedias, etc.). A large number of proposals, which focus on how ICT
solutions should be applied to the CH domain for di erent purposes, has been
presented in the literature [
In spite of the great e ort, some research problems have to be still faced
during the design of a modern Cultural Heritage Information System, especially if we
consider high change rate, large volume, and intrinsic heterogeneity of cultural
data: i) the adoption of architectural models for Big Data management [
This paper represents an extended abstract of the work [
The paper is organized as follows. Section 2 describes the proposed data model. Section 3 presents a system description with several implementation details. Section 4 reports a possible application of our system and discusses some conclusions and the future work. 2
The introduced data model relies on the concept of \Cultural Item" (CI): examples of CIs are speci c ruins of an archaeological site, sculptures and/or pictures exhibited within a museum, historical buildings in an old town center and so on. 1 The High Technology District for Cultural Heritage (DATABENC) management of the Campania Region, in Italy (www.databenc.it).
In the CH domain, a CI can be opportunely described with respect to a
variety of annotation schemata, for example the archaeological view, the
architectural perspective, the archivist vision, the historical background, etc., which
usually exploit di erent sets of \metadata" and possibly domain taxonomies or
ontologies [
The attributes A1; : : : ; An are Ontological Attributes (OAs) and correspond to concepts that are relevant for the speci c domain(s) being modeled. In turn, Non-Ontological Attributes B1; : : : ; Bm (NOAs) can contain other useful information, such as multimedia items (e.g. audio, video, images, texts and 3D models, etc.) characterized by a set of low-level features and other metadata. In particular, we can adopt both \literals" or a set of URIs (Uniform Resource Identi ers ), which allow to access the related cultural information according to the LD/LOD paradigms, as values of the annotation attributes. In addition, a CI may be associated with a speci c \Point of Interest" (POI), de ned by a set of geographic coordinates, and corresponding either to a single point or to a set of lines and more complex polygons of the considered environment. De nition 2 (Semantic Annotation). Given a set of ontologies O, an annotation schema O and cultural item CI, a Semantic Annotation of CI is a tuple O(CI) = (a1; : : : ; an; b1; : : : ; bm), where 8i 2 [1; n]; ai 2 dom(Ai) and 8j 2 [1; m]; bi 2 dom(Bi)
Using various sets of ontologies and semantic annotations [
Leveraging di erent annotation schemes and ontologies, our model allows achieving interoperability goals. The KB content can be easily exported in the most used formats (e.g. XML, RDF, OWL) and according to the most di used harvesting standards for CH applications (e.g., EDM, Italian ICCD, etc.). On the other hand, the LD/LOD paradigm permits us to deal with several problems related to data consistency and copyright constraints3.
KIRA has to deal with the large and heterogeneous amount of information:
annotations and descriptions provided by cultural heritage foundations, web
encyclopedias and open archives, multimedia contents coming from social media
networks and digital libraries, opinions and comments of users from common
online social networks, etc. For this reason, KIRA presents a layered architecture
typical of a Big Data platform [
Functionalities and Implementation Details
One of the most important functionalities provided by KIRA consists in the capability of gathering the di erent kinds of data from di erent sources: User Data, Social Data, Digital Repository and Multimedia Data.
User Data basically include preferences and needs that are useful to de ne
the related pro les: data on users (e.g. favorite artistic genre and artists)
constitutes the Personally Identi able Information (PII) that is stored in the
Knowledge Base and can be used as additional lter in the retrieval. As to Social Data,
the current prototype only considers information coming from Twitter. In
particular, KIRA retrieves user comments and posts information about a given CI
by exploiting the related APIs. Social data can be used in several applications
requiring a \social vision" of cultural items. We collect Digital Repositories' Data,
information describing cultural items from on-line digital repositories (e.g.
museums, libraries, open archives, multimedia collections, etc.). The wrapper for
this kind of sources can import such data descriptions and convert them into a
JSON format. Multimedia data (e.g. images, texts, video, etc.) related to a given
cultural item can be similarly collected using the wrapper facilities. In
particular, the descriptions in terms of basic metadata are captured and stored within
4 Data integration problems for heterogeneous data sources are addressed by means of
classical schema mapping techniques, record linkage and data fusion techniques [
KIRA, while raw multimedia data can be opportunely linked and, in other cases,
temporarily imported into the system for content-based analysis [
The data gathered by the Wrappers are then stored and managed by the
Knowledge Base. One of the basic functionalities of the KB is to export the
related content into the Europeana Data Model 6 (EDM) format (see [
The data describing basic properties of CIs (e.g. name, short description, etc.)
and basic information on users pro les are stored into a key-value data store (i.e.
Redis). The complete description in terms of all the metadata of CIs using the
di erent annotation schemes are in turn saved using a wide column data store
(i.e. Cassandra). We use a table for each kind of CIs having a column for each
\metadata family"; column values can be literals or URIs. The document store
technology (i.e. MongoDB ) is used to deal with JSON messages, complete user
pro les and descriptions of internal resources (multimedia data and textual
documents, etc.) associated with a cultural item. All the relationships among cultural
items within a cultural environment and interactions with users (behaviors) are
managed by means of a graph database (i.e. Titan). The entire cartography
related to a cultural environment together with POIs is managed by a GIS (i.e.
PostGIS ), which provides the functionalities to lter and visualize on a map the
geographic area around a given PoI. Multimedia data management is realized
using the Windsurf library [
This heterogeneous Knowledge Base provides basic Restful APIs to
read/write data and further functionalities for importing/exporting data in the most
common di used Web standards. The search of data useful for the applications
can be eased by using di erent information lters that implement the right
queries to the various databases. The implementation of such lters is based
5 Note that multimedia data that are managed by the system are suitably ltered
before the storing process. The number and kinds of multimedia data required by
the application are tuned by means of con guration parameters.
6 http://pro.europeana.eu/edm-documentation
7 We chose to adopt such heterogeneous technologies in order to meet the speci c
requirements of the applications dealing with the huge amount of data at stake.
For example, Social Networking applications typically bene t of graph database
technologies because of their focus on data relationships. In turn, more e cient
technologies (key-value ore wide-column stores) are required by Tourism applications
to quickly and easily access the data of interest.
on Apache Spark, and we distinguish four kinds of query on the KB: i) query
by keywords/tags : through such a query a user/application can search a set of
CIs using keywords (as in Google search engine) or speci c tags (the query is
then \expanded" with similar search terms leveraging the system thesauri); ii)
query by metadata: by this query a user/application can search a set of CIs
using speci c metadata of internal or external annotation schemes (the query
for OAs is semantically \expanded" with the concepts of managed ontologies
that are similar to the target one); iii) query by example: through such a query
a user/application can search a set of multimedia contents { related to CIs {
that are similar to a target one (the query processing is base on the Windsurf
multimedia libraries ); iv) query by user preferences : user pro les are exploited
to nd the set of cultural items that are more similar to user preferences using
co-clustering techniques [
We describe a possible application of our system to support the development of a multimedia guide for the Paestum archeological site. The ancient buildings, together with the museum and its main artifacts, constitute the set of cultural items for our case study. Tourists, both from their places and while visiting ruins, can browse these cultural items and enjoy a useful multimedia guide describing them, or be recommended other nearby places, comments of other users and other information of interest. When users search a speci c cultural item, as an example the Temple of Neptune, our system provides a basic description with the related multimedia objects (i.e. audio, images, video and texts) and detailed users' comments. The list of proposed cultural items with descriptions and multimedia objects depends on the user's preferences and system settings: images rather than voice, or expert-level rather than layman-level descriptions of the art pieces; speci c metadata and annotation schemes. In addition, query by example facilities can be exploited to determine other images that are similar to a given multimedia object. In addition, a semantic search can be performed on speci c ontological attributes to nd other cultural items of the same type. At the same time, users can choose to retrieve some interesting information, to read comments, opinions and ratings about the visited cultural items and to express their own ratings and opinions. Figure 2 shows a running example (obtained by assembling di erent screenshots) concerning the search of CIs related to the Paestum ruins. Users can browse the data by means of an appropriate GUI; they can lter objects belonging to a given CI using di erent criteria: type of multimedia data, language, etc. Future work will be devoted to collect the huge amount of data related to all the di erent cultural objects of the Campania region and to experiment our system from the e ciency and e ectiveness points of view with respect to the information retrieval and ltering tasks providing a comparison with other systems.