Cognitive-based Visualization of Semantically
Structured Cultural Heritage Data
Kalliopi Kontiza, Antonis Bikakis, Rob Miller
Department of Information Studies, University College London, UK
{k.kontiza.12,a.bikakis,rsm}@ucl.ac.uk
Abstract. We present preliminary findings regarding the increasing use of Info-
Vis tools and semantically structured data by cultural heritage institutions. This
sector faces a number of challenges in developing best practices for publishing
Linked Data, including the presentation of their digital cultural heritage collec-
tions and the visualization of their multidimensional hidden histories. We suggest
that, as these institutions’ interest in Semantic Web technologies grows and as-
sociated applications are more widely adopted, the need to provide InfoVis tools
for efficient overview and exploration of cultural data increases. We postulate that
changes in the paradigms for interaction with cultural datasets are also needed,
with more focus on users’ needs and cognitive processes. We suggest that by tak-
ing into account human information processes, better cognitive support can be
introduced via InfoVis tools for Linked Data, thus reducing the cognitive load
experienced by users.
1 Introduction
The aim of this position paper is to give a brief summary of the current state-of-the-art
as regards the use of Information Visualisation (InfoVis) for semantically structured cul-
tural heritage (CH) data, and to suggest some directions for research and development
in this area. The Semantic Web (SW), also known as Web 3.0, is the new environment
in which cultural digital resources will be exploited. The Resource Description Frame-
work (RDF), which integrates a variety of applications using XML or other machine-
readable formats for syntax and URIs for naming, has become the standard data model
for describing semantically structured data using statements in the form of triples, which
describe resources and can be considered as metadata. Linked Data (LD) is a way of
publishing structured data that allows metadata to be connected and enriched, so that
different representations of the same content can be identified, and links between related
resources can be made. Although the term LD is often used as if it is a specific, well
defined technology, it could be better understood as a set of best practices [8]. These
aim to get data published on the web in a way that is readable, interpretable and usable
by machines, by ensuring that their meaning is explicitly defined by a string of words
and markers [6].
LD practices are starting to be introduced as novel and promising approaches to
address the specific challenges that the CH sector encounters when publishing “col-
lection” data on the Web [7]. Museums as memory organizations are key players in
61
Cognitive-based Visualization of Semantically Structured Cultural Heritage Data
preserving CH Tangible1 objects by storing them with attached metadata. Despite ef-
forts amongst the CH community to analyze the needs of their online ‘audiences’, no
clear understanding about these users and their expectations has yet been gained [8].
Of course, publishing CH contents on the Web cannot replace the physical experience
of visiting a museum or an exhibition in reality. But the key question is whether online
content, when combined with appropriate Visualization tools, can act as a complemen-
tary alternative for access and exploration of cultural data. The hope is that the structure
and semantics of RDF representations and standard ontologies, combined with InfoVis
techniques, can lead to new ways of thinking about aspects of cognition as emergent
properties of the interaction of people with cultural artifacts [16].
We take as a starting point the definition of InfoVis as “the use of computer sup-
ported interactive visual representation of data to amplify cognition” [3]. Our long term
aim is to investigate why and how InfoVis can assist people in understanding CH infor-
mation, in particular by exploiting the structure and the semantics of RDF representa-
tions and relevant ontologies. More specifically we aim to develop a cognitively based
InfoVis model that is geared towards revealing the various temporal, spatial, contex-
tual, conceptual links between different cultural artifacts, their creators and associated
events, building upon standard ontologies and vocabularies. From this we hope to de-
velop a set of principles for the design of interactive visual interfaces for exploring and
understanding the semantically encoded data used in museum collections.
The rest of the paper is structured as follows. Section 2 provides an overview of
some relevant SW terminology and then describes a number of current initiatives that
use InfoVis Tools to present semantically structured CH data. Section 3 provides argu-
ments in favor of taking greater account of theories of human cognition when designing
CH InfoVis Tools, in order to provide efficient exploration of semantically encoded cul-
tural datasets. Finally, Section 4 summarizes the preceding discussion, and sets out an
agenda for future research.
2 Semantic Web and Linked Cultural Data
2.1 Background
Tim Berners-Lee has re-introduced the SW as “a new layer of metadata being build
inside the Web” [7]. One of the main principles behind its architecture is that it of-
fers users the ability to share knowledge by constructing meaningful representations
on the Web [1]. RDF was published as the first SW standard by the World Wide Web
Consortium (W3C) in 19992 . But it was only around 2005 when the ideas of SW and
LD started to gain momentum and the SW and LD research communities focused on
enhancing existing large datasets using simple RDF and lightweight ontologies.
The use of lightweight ontologies based on RDF is one of the reasons for the success
of LD in the CH sector. The Dublin Core vocabulary3 for example, which became pop-
ular for expressing metadata as RDF, provides the basis for the Europeana Data Model
1
Tangible cultural heritage consists of concrete cultural objects, such as artifacts, works of art,
buildings and books
2
http://www.w3.org/RDF/
3
http://dublincore.org/documents/dcmi-terms/
62
Cognitive-based Visualization of Semantically Structured Cultural Heritage Data
(EDM)4 used by Europeana5 for gathering and publishing metadata for thousands of
digitized cultural collections [10]. But even more sophisticated models are provided
as simple RDF Schema ontologies. One example is the CIDOC Conceptual Reference
Model (CIDOC-CRM)6 , a core ontology for describing the semantics of schema and
data structure elements in CH documentation [7]. This was developed as an ambitious
attempt to link museum objects across collections ontologically [4]. The CIDOC-CRM
aims to facilitate “the integration, mediation and interchange of heterogeneous cultural
information, by capturing their richness” [4].
Heath [7] enumerates some of the content related challenges that CH institutions are
faced with in order to publish their data collections: their content can be multi-format
(text, images, audio, video, collection items, learning objects), multi-topical (art, his-
tory, artifacts, traditions, etc.), multi-lingual, multi-cultural and multi-targeted (targeted
to both laymen and experts, various ages, etc.). SW standards and LD as best practice
could provide a shared basis with which to facilitate content-related and cross-domain
semantic interoperability.
Progressive steps have been taken towards LD online publishing. Following the LD
principles7 Tim Berners-Lee also introduced a five-star rating scheme for LD at increas-
ing levels of openness and linkage8 . For a linked dataset to be accepted by the Open
Linked Data Project9 it must not be subject to commercial licenses or use restrictions.
2.2 Current Initiatives on Linked Data
Figure 1 (larger version at lod-cloud.net) gives an overview of the datasets that have
been published online following the Linked Data format and principles. It shows the
growing number of CH institutions, web services and museums that have adopted LD
standards as a way to turn the rich descriptions of their digital collections into a web
of related data. Examples of such institutions include The Getty Research Institute10
and the Yale Center for British Art11 . Examples of web services are the Swedish Open
4
http://www.europeana.eu/schemas/edm/
5
http://europeana.eu
6
http://www.cidoc-crm.org
7
The LD principles formulated by Tim Berners-Lee are: 1) use URIs as names for things, 2) use
HTTP URIs, 3) when someone looks up a URI, provide useful information using the standards
(RDF, SPARQL), 4) include links to other URIs, so that they can discover more things.
8
http://www.w3.org/DesignIssues/LinkedData.html
9
http://linkeddata.org/
10
http://www.getty.edu/research/tools/vocabularies/lod/
11
http://britishart.yale.edu/collections/using-collections/
technology/linked-open-data
63
Cognitive-based Visualization of Semantically Structured Cultural Heritage Data
Debian Linked
Prospects Package
and
Prefix.cc Jugem Tracking
Mark
Athelia Trends Mail Bootsnall
System
RFID GNOSS Cornetto
Courts Isocat
Thesaurus
Web
Gem. Nmasuno Artenue Vulnera- Linked
IDS
Red Uno
Thesaurus Traveler Vosmedios pedia User
Audiovisuele
Internacional DBnary Ietflang
Archieven
Elviajero GNOSS GNOSS Feedback Yso.fi
GovUK
Societal Flickr WALS Allars
Wellbeing
Deprivation Imd
Environment
DBTune
Proyecto Nextweb Wrappr Tags2con Olia
Rank 2010 Magnatune Delicious
DBTune Apadrina GNOSS Pdev Yso.fi
Zaragoza Museos Dewey
Turruta GovUK artists Espania Datahub.io Lemon YSA Lista
Decimal
Imd Crime Semantic Encabeza
Rank 2010 last.fm GNOSS Classification
ineverycrea Lemonuby Quran Mientos
Opendata
Scotland Simd DBTune Deusto WOLD
Materia
Wiktionary Shoah
Geographic John Peel Tech DBpedia
Access Rank
GovUK Societal
Wellbeing
GovUK
Wellbeing lsoa Acorn Indymedia Sessions Lotico Didactalia Berlios Glottolog JITA Victims
Deprivation Imd
Crime Rank 2010
Happy GNOSS Names
GovUK Yesterday Sat
Transparency Mean RKB Reload Sudoc.fr
Impact Indicators
Energy Efficiency Interactive Explorer ZDB
new Builds GovUK
German Maps Green Wordnet UTPL
imd env. Miguiad
Zaragoza Labor Law GNOSS Competitive- LOD
rank 2010 GovUK
Datos Enakting
Pokepedia
Thesaurus Eviajes ness Wordnet Lexinfo Muninn Theses.fr
Opendata Homelessness
Crime GNOSS Garnica Data World War I Universidad
Scotland
Households
Abiertos GNOSS (VU) Open Gesis Pub
Simd Health
Accommodated
Temporary Plywood de Cuenca
Opendata Greek Ac Uk Thesoz Aspire Bielefeld
Rank
Scotland GovUK
Housing Types
Open Social-
Apache Linkeddata
Households Wordnet Project
Simd Social Lettings
Umthes
Data semweb Qmul
Crime Rank General Needs GovUK Chronicling Gutenberg
Lettings Prp DBTropes Thesaurus
Aspire
Opendata
Household Housing
Semantic America RISP MSC RKB
FU-Berlin
ODCL Composition Market
RKB Brunel
Scotland XBRL My Worldcat Data Explorer
SOA Simd Housing Experiment
ISO 639 RKB Explorer ERA
Rank
GovUK
Impact Eurovoc Oasis Explorer Bnf.fr unlocode Radatana
Indicators Enakting BBC Linked
CE4R Webscience Princeton Ciard
in Openly MyOpenlink
Affordable
Housing Starts SKOS
NHS Jamendo
Program-
Crunchbase Library
Findingaids
VIVO Ring Aspire
Opendata Local Brazilian DBTune Dataspaces
GovUK Indiana Keele
Scotland impact Politicians OCD mes STW Bible
Graph indicators energy GovUK Amsterdam Thesaurus
Bluk University
Simd Rank efficiency new Wellbeing
OpenlinkSW Museum for BNB Ontology
builds
Worthwhile RKB Semantic- RKB Aspire
Mean Explorer RDFize Dataspaces
web.org Wordnet
AS EDN LOD Economics Datos Explorer
GovUK Crime Eurostat
Open FAO MMU Aspire
Households
FU-Berlin last.fm Typepad (W3C) Idref.fr Bne.es OS Dev8d
Opendata Social lettings
SORS Linked Geopolitical NTU
Scotland Simd General Needs GovUK
Icane Calais Ontology
Mis
Income Rank
Lettings Prp
Number
Transparency
Impact Indicators Enakting Edgar Museos B3Kat Semantic
Bedrooms Neighbourhood
Plans Energy Currency
GovUK
DNB GNOSS Web
Designators
Revyu Semanlink Colinda Grundlagen lobid Nottingham
GovUK
GovUK
Transparency
Transparency
Impact ind. BBC DCS Resources Aspire Trent
Societal Impact Indicators
Households DBTune Lingvoj Agrovoc Bibbase Resource
GovUK
Transparency
Wellbeing
Deprivation Imd
Affordable
Housing Starts
Loius Open In temp. Music RDF DWS Sheffield UCL Lists
Ctic Music-
Impact Indicators Housing Rank la GovUK
Impact Energy Accom.
Ohloh Skos Southampton Aspire
Planning 2010 Public Indicators Eurostat Camera brainz LCSH Group ECS Eprints
Applications
Granted Dataset Planning
ABS Info Wiki Sussex
Applications Linked Deputati BBC Aspire Gutenberg
GovUK
Granted 270a.info Data Linked NYTimes Lexvo Manchester Sztaki
GovUK Societal Wildlife LOD2 L3S
Opendata Wellbeing Linked Data
Hellenic
Linked Open
Finder Project O'Reilly LOD Core
Scotland Graph
Education
Households
2008
Deprivation imd
Employment Stock Data Wiki Bibsonomy DBLP RKB
Pupils by Rank La 2010
Linked Police Europeana Wordpress Southampton RKB Explorer
School and NUTS BFS Index GovUK ac.uk Viaf Explorer
Datazone Transport RKB Courseware Verrijktkoninkrijk
270a.info PlanetData Eprints
GovUK GovUK Data Explorer
Societal societal
Linked NHS Clean Linked Project Harvest
wellbeing Wiki RKB JISC
Wellbeing
Deprivation Imd deprivation imd Statistics Eurostat Jargon Energy
MDB Explorer
RKB Libris Aspire
GovUK Health Score employment
Transparency Data W3C RKB Explorer DOI
Societal
Wellbeing
2010 score 2010 data.gov.uk UIS Reference 270a.info Reegle Semantic Web
Deepblue RKB
Explorer Dotac Uclan
Deprivation Imd Explorer
270a.info data.gov.uk DogFood RKB
Education Rank La
2010 Morelab RKB RAE2001 Resex
Explorer
GovUK
GovUK Explorer Citeseer
transparency Transparency GEMET RKB Hellenic Archiveshub RKB Curriculum
Opendata
impact
indicators
Impact
Indicators
UK Eurostat Explorer Fire Brigade
Geo KUPKB Linked Explorer RKB Open Multimedia
Lab University Ariadne
Scotland Simd
Education
tr. families Housing Starts Legislation
API
RDF
FAO
cordis
Names DBpedia LOV
Data NSF
RKB
RKB
Explorer RKB
Explorer
OAI
Library Ghent
Rank Eprints Explorer Aspire
270a.info WWW Explorer
GovUK GovUK GovUK Worldbank Drugbank RKB
Southampton RKB IEEE Portsmouth
Impact
Societal Societal 270a.info GovUK Foundation CKAN Explorer Explorer
Indicators
Wellbeing Wellbeing Education NALT FU-Berlin ACM
Deprivation Imd
Deprv. Imd IMF EUNIS Newcastle RKB
Housing Starts Health Rank la ECB Data
Opendata
2010 Empl. Rank
270a.info 270a.info NUTS UMBEL RKB RKB Explorer Aspire
La 2010 OECD Explorer Explorer Ulm Roehampton
Scotland Simd Geo- VIVO RKB
270a.info GovUK
EEARod Risks Pisa
Employment vocab
Rank GovUK GovUK Environment
Transparency
Input ind. FOAF- Linked Geospecies
University Explorer Deutsche
Biographie
societal Societal
Open
Local auth.
Geo Uniprot of Florida RKB LAAS
wellbeing Wellbeing
Deprivation Imd
Agency
Bathing Water Data
Funding f.
Gvmnt. Grant
Enakting
Mortality Linked
Profiles Open Data
Metadata RKB Explorer RKB
deprv. imd Income Rank La Quality Freebase YAGO of DBLP Explorer
rank la '10 Euskadi BIS Data Linked Explorer
GovUK
2010
Ecology kisti
lobid
Transparency 270a.info Geo Irit RKB NBN
Input indicators Opencyc Organizations RKB
Local authorities EEA Environmental Data Explorer Aspire Resolving
Working w. tr. GovUK GovUK Applications Taxon RKB Explorer Deploy Plymouth
Families
societal Input ind. Govtrack Reference Sider
Nomenclator FRB linkedct Explorer RKB IBM
wellbeing Local Authority
Asturias 270a.info
Thesaurus
Ordnance StatusNet DBpedia concept FU-Berlin
deprv. imd Funding From Wiki Explorer
Government ESD Linked Survey NL RKB
rank '07 Enakting Open
Progval Code Budapest RKB RKB
Grant Standards Linked Explorer
CO2Emission Piracy DBpedia Explorer Explorer RKB Dutch
GovUK Enakting Data StatusNet Haus Lisbon
Imd Income Population Dtdns EL DBpedia URI Taxon- Darmstadt Eurecom Explorer RKB Ships and
Rank La GovUK Government Testee Explorer
2010
GovUK Societal
Web Integration Linked live Burner Aves3D concept FT
Epsrc
Sailors
Households Wellbeing
for Railway Assets
Accommodated
Deprv. imd
Linked EPO IATI as City Data StatusNet StatusNet RKB
Data
per 1000
Score '10 Linked Data Lichfield Project GADM 1w6 Lydiastench StatusNet Bio2RDF
Explorer TWC
Geovocab chickenkiller Drug Dataset British
DBpedia Dailymed Organic Roma
Arago-
Interaction
Edunet
IEEEvis Museum
GovUK
Societal Open
OSM StatusNet StatusNet DBpedia FR Linklion Knowledge FU-Berlin Collection RKB JudaicaLink
dbpedia EU Agencies Data for Samnoble
Wellbeing
Bodies Tourists in Election Open Data Open Thelovebug DE Base Explorer
Deprivation
Imd Rank 2010 Castilla y Leon Mobile StatusNet StatusNet Serendipity Italy
Data Ecuador Uniprot
Project Network Qdnx StatusNet Fcac Biomodels RKB
2001 Geo StatusNet Ssweeny LOD RDF Asn.us Explorer
GovUK GovUK Geological David DBpedia Disgenet ACBDLS Kaunas
Spanish
Imd GovUK Homeless-
Ecuador Survey of ES GNU GNI National
Census Net Add. ESD Haberthuer
to RDF Rank 2010
Dwellings
ness Austria StatusNet StatusNet DBpedia Licenses Aspire Diet Library Getty DM2E
Accept. per Toolkit Randomness Thesaurus StatusNet chromic Harper WEB NDL AAT
1000 StatusNet Russwurm KO Bio2RDF Identifiers Authorities
Guide Bka StatusNet Taxon- Taxon Org Adams
London Somsants concept Publications
GovUK GovUK Timttmy Linked Linked Bio2RDF
UK DBpedia Occurences Swedish
Households wellb. happy World Postcodes
StatusNet StatusNet DBpedia TCGA Life GeneID Semantic
Projections yesterday StatusNet Opensimchat EU Open Cultural Web
Factbook Gomertronic
BPR Population std. dev. GovUK FU-Berlin
Maymay StatusNet JA Data Bio2RDF
Irefindex
Heritage Journal
Morphtown Identifiers
Dev Local StatusNet StatusNet DNB Bio2RDF Bio2RDF Life Sciences
Authority shnoulle NDC Drugbank
Services Thist Greek Mrblog GND Gene
GovUK
Eionet
Administrative StatusNet Bio2RDF chem2 Expression
Biosamples
service GovUK Geography StatusNet DBpedia Bio2RDF bio2rdf Atlas RDF
RDF StatusNet StatusNet StatusNet Fragdev Mesh RDF
expenditure Imd Score Imirhil IT
2010 GovAgriBus Kaimi Soucy Quitter StatusNet DBpedia Bio2RDF
Bio2RDF
HGNC
Cross-Domain
Denmark
Jonkman IServe CS Uniprot Clinicaltrials
GovUK
StatusNet StatusNet Taxonomy
Households Charging StatusNet Enipedia Bio2RDF Bio2RDF
Projections Stations
Oceandrilling
Orangeseeds gegeweb Postblue Bio2RDF
Orphanet Sabiork
Borehole Bio2RDF
total StatusNet Reactome Ncbigene Social Networking
Houseolds StatusNet
Bizkai Macno DBpedia RDF SGD
Atari StatusNet Diseasome
Sense StatusNet StatusNet StatusNet linuxwrangling Austrian
Bio2RDF
StatusNet Frosch Rainbowdash Lite FU-Berlin
Bio2RDF SGD
CIPFA NVS Skilledtests Legadolibre Ski
Bio2RDF Pubmed Resources
mkuttner Bio2RDF
Alexandria
Digital Library StatusNet
Racers
RDF DBSNP Iproclass Geographic
Gazetteer StatusNet StatusNet Alpino License
Lebsanft StatusNet Freelish Hackerposse RDF Chembl
Geo schiessle StatusNet StatusNet RDF
AEMET Wordnet StatusNet Cooleysekula Uniprot Bio2RDF
Status.net StatusNet DBpedia Product Bio2RDF Taxonomy
StatusNet StatusNet Ourcoffs Ilikefreedom PT DB
KB
OMIM Bio2RDF Bio2RDF
Government
StatusNet Spip piana Pharmgkb LSR
StatusNet
Enel Coreyavis Datenfahrt
StatusNet StatusNet
Shops StatusNet StatusNet RKB Bio2RDF
StatusNet Belfalas Scoffoni StatusNet StatusNet Explorer Bio2RDF Media
Tekk Uni Bio2RDF GOA
Exdc Tschlotfeldt
StatusNet StatusNet Mamalibre Siegen Product Open ECS Affymetrix CTD
StatusNet StatusNet
thornton2 Glou Ontology Bio2RDF
Hiico Deuxpi Food Bio2RDF
ECO Homologene
StatusNet StatusNet StatusNet Facts
StatusNet StatusNet User-Generated Content
StatusNet Recit Planetlibre Sweetie
Qth Spraci
Integralblue
StatusNet Belle Open Linked Bio2RDF
StatusNet Data Wormbase
Alexandre Fcestrada Ontos Food
StatusNet StatusNet News KDATA Thesaurus Bio2RDF
Mystatus Keuser StatusNet Franke Portal Biomodels
Bio2RDF
StatusNet
StatusNet StatusNet Otbm OMIM Linguistics
20100 Mulestable StatusNet StatusNet Resources
ldnfai StatusNet StatusNet Ced117 Status Nobel
StatusNet Sebseb01 Pandaid
Tekord Prizes
Ludost
StatusNet
tl1n Livejournal
SISVU Linked Datasets as of August 2014
StatusNet StatusNet StatusNet
StatusNet
Kenzoid doomicile StatusNet StatusNet Johndrink Equestriarp
Bonifaz Kathryl Water
Fig. 1. The Linked Open Data cloud (August 2014) consists of datasets (shown as bubbles) in
different domains (colour identified) and mappings between similar sources in the dataset (shown
as arcs between bubbles). Source: Linking Open Data cloud diagram 2014, Schmachtenberg,
Bizer, Jentzsch and Cyganiak. http://lod-cloud.net/.
Cultural Heritage (SOCH)12 , Europeana13 , and Museos de Espan̈a14 . Specific museums
include various Italian Museums15 , The Smithsonian American Art Museum16 , The
British Museum17 , The Victoria and Albert Museum18 , and The Rijksmuseum19 .
The integration of semantics-based interaction paradigms showcases the benefits
of semantically marked up information, delivered to the end user through a semantic
browsing facility, and highlights an important feature of the SW for the CH domain; the
ability to explore and navigate relationships. The SCULPTEUR project focused initially
on the semantic interoperability of LD and further developed the mSpace interaction
framework20 , resulting in an interface that provides the users with the ability to organize
their information space. The interactive interface aimed to allow users unfamiliar with
12
A web service used to search and retrieve data from any organization holding information or
media relating to Swedish cultural heritage, http://www.ksamsok.se/in-english/
13
Aims to support research of the digitised content of Europe’s galleries, museums, libraries and
archives by addressing issues such as licencing, interoperability and access, http://labs.
europeana.eu/api/linked-open-data/introduction/
14
a directory that allows facet-based searches on a collection with more than 1,500 dy-
namic pages of public and private Spanish museums, http://museos.gnoss.com/
comunidad/mismuseos/
15
http://www.linkedopendata.it/datasets/musei
16
http://americanart.si.edu/collections/search/lod/about/
17
http://collection.britishmuseum.org/
18
http://www.vam.ac.uk/api/
19
https://www.rijksmuseum.nl/en/api
20
http://www.w3.org/2001/sw/wiki/MSpace
64
Cognitive-based Visualization of Semantically Structured Cultural Heritage Data
museum collections to navigate, visualize and explore rich sources of cultural heritage
information [19]. In particular, the application of InfoVis tools in mSpace helps people
to develop knowledge by exploring relationships in data, and to customise access to the
content to suit their individual interests by “slicing, sorting, swaping, information views
and multimedia preview cues” [18].
MUSEUMFINLAND (Finnish Museums on the Semantic Web)21 is a semantic por-
tal for publishing heterogeneous museum collections on the Semantic Web in order to
provide the museum visitors with intelligent content-based search and browsing ser-
vices [9]. The view-based multi-facet search engine exploits the semantic link recom-
mendation system to reveal the underlying semantic context of the collection items and
their mutual relation [9]. CultureSampo22 is a continuation of this work. The Culture-
Sampo interface has been enhanced with InfoVis tools that allow the user to explore
cultural heritage through nine semantic perspectives/thematic views: Maps and histori-
cal places, relational search, faceted domain-centric browsing, collections, Finnish his-
tory, cultural processes and skills, biographies, semantic Kalevala, and Carelia23 . The
main concept behind the interface is “to let users create virtual exhibitions that mimic
the way real museums are organized, containing themed exhibition rooms of items and
displays that together, through the objects, tell the story of a particular subject” [14].
Through the CHIP (Cultural Heritage Information Personalization) project24 , the
Rijksmuseum focused on delivering novel personalization functions for the visitors on
the museum’s website. The CHIP demonstrator included three innovative components;
the Art Recommender, the Tour Wizard and the Mobile Tour Guide, where the SW
was deployed to support “the presentation of recommendations by combining different
views like a historical timeline, a museum map and a faceted browser”[20]. The eval-
uations of the CHIP demonstrator provided critical insight in how to further adapt the
user interaction facilities and interface to suit user’s needs and preferences.
The ResearchSpace project25 is a contextual search system that allows searching
against objects, people, places, events, periods and concepts, providing context by mak-
ing use of semantically enriched cultural data. The new design of the ResearchSpace
search interface is a radical departure from traditional keyword and advanced searching,
and can be customized to suit different online audiences.
Finally, the Russian Linked Culture Cloud26 project is a collaboration between The
Russian Museum and ITMO University, and is based on open data related to Russian
Culture heritage. SW technologies, enabling enrichment of initial data with other facts
of Russian culture, provide an advanced user experience with the help of visualization
and navigation through enriched text, interactive timelines and an interactive influence
graph [15].
21
http://www.museosuomi.fi/
22
http://www.kulttuurisampo.fi/
23
http://www.kulttuurisampo.fi/about.shtml?lang=en
24
http://www.chip-project.org
25
http://www.researchspace.org/home/project-information/design
26
http://culturecloud.ru/
65
Cognitive-based Visualization of Semantically Structured Cultural Heritage Data
3 Cognition-based InfoVis Tools for Linked Datasets
As industry’s and society’s interest in SW technologies grows and the number of widely
adopted SW applications increases, there is an opportunity to focus on the properties
and characteristics of semantically encoded data that can most readily be used to en-
hance InfoVis tools. In the case of online museum collections, SW technologies can
have a beneficial impact on (a) exploring and navigating relationships, as richer se-
mantics highlight the conceptual relationships between artifacts, and (b) presentation-
interaction, as they offer richer presentation possibilities in terms of browsing and nav-
igation.
Linked data interoperability in the semantic web has recently received much re-
search attention. However, the emphasis has largely been in automating the mapping
process to standards, even though the creation of mappings often involves the user. The
main Linked Data users are technology experienced, and one reason for this is the lack
of appropriate user interfaces and visualizations for non-expert users. Visual approaches
are needed to assist various kinds of users, who pursue diverse goals and have individ-
ual requirements. InfoVis tools developed using a human-cognition-based model will
cetainly improve users’ engagement with and understanding of semantically encoded
cultural data.
Case studies of museums that have implemented personalization facilities to their
Web sites show that understanding is stimulated when the systems use concepts familiar
to the user [2]. Visuals help understanding by acting as a frame of reference or as a
temporary storage area for human cognitive processes. By providing a larger working
set for thinking and analysis they become external cognition aids [12]. Card, Mackinlay
and Shneiderman [3] list some key ways in which visuals can amplify cognition:
(i) Increasing memory and processing resources available
(ii) Reducing search for information
(iii) Enhancing the recognition of patterns
(iv) Enabling perceptual inference operations
(v) Using perceptual attention mechanisms for monitoring
(vi) Encoding information in a manipulable medium
One of the main aims of InfoVis is to amplify and augment the cognition of users.
But a key challenge of the field is to measure its effectiveness in this respect. The ab-
sence of a cognitive-based framework for the evaluation of InfoVis systems makes the
significance of achievements in this area difficult to describe, validate and defend. Ac-
cording to [13] even though InfoVis research has matured technically in recent years,
an important problem for the field remains the lack of an underlying theory or even
a systematic framework for guiding design and investigation. InfoVis classifications
have focused on the process of synthesising and displaying data, and how to standard-
ise this from a computational point of view. Considerations of how users interact with
the resulting interfaces have only recently been integrated into this area of research. A
greater emphasis on human cognition has the potential to uncover new ways of present-
ing, searching, exploring and visualizing the available semantic data in order to enhance
human understanding. For example, the very recent work of Patterson et al [16] links
66
Cognitive-based Visualization of Semantically Structured Cultural Heritage Data
InfoVis with high-level cognitive processes such as reasoning and thinking, and Liu et
al [13] argue that the use of “distributed cognition ... has the potential to serve as a
theoretical framework for InfoVis”. Other approaches that have revealed potential re-
search opportunities in this area include Greene and Petre’s Cognitive Dimensions [5],
Johnson-Laird’s work on model theory and reasoning using visual notations [11] and
Peirce’s systems of diagrammatic logic[17].
In the particular context of cultural heritage, a cognitive-based framework for the
design of InfoVis interactive systems can improve visitors understanding of collections
and their ability to explore the cultural information according to their needs. This will
help visitors to discover the interconnectedness of digital cultural collections, enable
information to be presented attuned to their interests and background, and therefore
increase users interest and engagement with both digital and physical collections.
4 Summary and Conclusions
This paper presented some preliminary findings from an ongoing investigation into the
use of InfoVis for semantically structured cultural heritage data. We gave a summary of
the current state-of-the-art in this area, followed by an argument for more user-focused
research and development, which will draw on models and theories of human cogni-
tion. Since the ultimate purpose of InfoVis in this context is to enhance users’ ability
to explore and understand (cultural) data, its effectiveness must be measured in these
terms, and attention must therefore be paid to ordinary users’ cognitive processes. Our
future research plan is to use data from observational studies together with ideas from
cognitive psychology in order to develop a theoretical framework for guiding the design
and evaluation of cultural heritage InfoVis tools. Such tools will allow users to explore
and interact with cultural data with only minimal additional cognitive load. An addi-
tional longer term aim is to apply this framework to produce an actual exemplar for a
new generation of online InfoVis tools for exploration of museum data collections.
References
1. Alesso, H.P., Smith, C.F.: Thinking on the Web: Berners-Lee, Goedel, and Turing. Wiley,
Hoboken, N.J. (2008)
2. Bowen, J., Fantoni Filippini, S.: Personalization and the Web from a Museum Perspective.
In: Museums and the Web 2004: Proceedings. Archives & Museum Informatics, Washington
DC / Arlington VA, USA (2004)
3. Card Stuart K. : Readings in information visualization : using vision to think / written and
edited by Stuart K. Card, Jock D. Mackinlay, Ben Shneiderman. Morgan Kaufmann series in
interactive technologies Y, Morgan Kaufmann Publishers (1999)
4. Eide, O., Felicetti, A., Ore, C.E., DAndrea, A., Holmen, J.: Encoding cultural heritage in-
formation for the semantic web. procedures for data integration through cidoc-crm mapping.
In: Open Digital Cultural Heritage Systems Conference. p. 47 (2008)
5. Green, T.R.G., Petre, M.: Usability analysis of visual programming environments: a cogni-
tive dimensions framework. Journal of Visual Languages & Computing 7(2), 131–174 (1996)
6. Guerrini, M., Possemato, T.: Linked data: a new alphabet for the semantic web. JLIS. it 4(1),
67 (2013)
67
Cognitive-based Visualization of Semantically Structured Cultural Heritage Data
7. Heath, T., Bizer, C.: Linked Data: Evolving the Web into a Global Data Space, vol. 1 (2011)
8. Hooland, S.v.: Linked data for libraries, archives and museums: how to clean, link and pub-
lish your metadata. Facet Publishing, London (2014)
9. Hyvonen, E., Makela, E., Salminen, M., Valo, A., Viljanen, K., Saarela, S., Junnila, M.,
Kettula, S.: MuseumFinlandFinnish museums on the semantic web. Web Semantics: Science,
Services and Agents on the World Wide Web 3(2-3), 224–241 (2005)
10. Isaac, A., Baker, T.: Linked data practice at different levels of semantic precision: The per-
spective of libraries, archives and museums. Bulletin of the American Society for Informa-
tion Science and Technology 41(4), 34–39 (2015)
11. Johnson-Laird, P.: 8 the history of mental models. Psychology of reasoning: Theoretical and
historical perspectives p. 179 (2004)
12. Kerren, A.: Information visualization human-centered Issues and perspectives. Springer
(2008)
13. Liu, Z., Stasko, J.T.: Mental models, visual reasoning and interaction in information visual-
ization: A top-down perspective 16(6), 999–1008 (2010)
14. Makela, E., Hyvonen, E., Ruotsalo, T.: How to deal with massively heterogeneous cultural
heritage data lessons learned in culturesampo. Semantic Web Interoperability, Usability,
Applicability 3(1) (2012)
15. Mouromtsev, D., Haase, P., Cherny, E., Pavlov, D., Andreev, A., Spiridonova, A.: Towards
the Russian Linked Culture Cloud: Data Enrichment and Publishing. In: The Semantic Web.
Latest Advances and New Domains, pp. 637–651. Springer (2015)
16. Patterson, R.E., Blaha, L.M., Grinstein, G.G., Liggett, K.K., Kaveney, D.E., Sheldon, K.C.,
Havig, P.R., Moore, J.A.: A human cognition framework for information visualization. Com-
puters & Graphics 42, 42–58 (2014)
17. Pietarinen, A.V.: Peirce and the logic of image. Semiotica 2012(192) (2012)
18. Pitzalis, D., Lahanier, C., Pillay, R., Aitken, G., Russell, A., Smith, D.A., Sinclair, P.A.S.,
Addis, M.J., Lowe, R., Hafeez, S., others: Semantically exposing existing knowledge repos-
itories: a case study in cultural heritage (2006)
19. Sinclair, P.A.S., Goodall, S., Lewis, P.H., Martinez, K., Addis, M.J.: Concept browsing for
multimedia retrieval in the SCULPTEUR project (2005)
20. Wang, Y., Stash, N., Aroyo, L., Gorgels, P., Rutledge, L., Schreiber, G.: Recommendations
based on semantically enriched museum collections. Web Semantics: Science, Services and
Agents on the World Wide Web 6(4), 283–290 (2008)
68