Personalized Artistic Tour using Semantic Web Technologies

Abstract

Based on current trends in the domain of Cultural Heritage promotion, visitors seek to engage in exceptional and unique experiences beyond established visiting practices. Meanwhile, the latest developments in information and communication facilitate access to cultural databases and repositories, bringing out the potential of new cultural products and services. In this direction, a variety of typologies and visitor categorizations have been developed that however do not take into account the complexity of visitors' demands and motivations, or that visitors tend to experience a journey based on new technologies and social media. Semantic Web technologies could be the key for designing personalized services, among other things, facilitating data interoperability in different repositories, making possible the correlation of data with different visitor profiles. In this context, Intelligent Interaction research group works towards an innovative approach that will enrich and enhance the experience of the modern cultural visitor.

Keywords

• Personalization
• Semantic Web
• Linked Open Data
• User eXperience (UX)

-

Preservation and promotion of Cultural Heritage (CH) can be greatly enhanced by implementing efficient methods for collection, storage and processing of cultural data. Remarkably increasing amounts of CH data regarding artefacts and collections hosted in GLAMs (Galleries, Libraries, Archives and Museums) are stored in online repositories around the world, while a big part of this knowledge is open and accessible to researchers, visitors and developers of cultural applications. However, information directly or indirectly related with a cultural artefact often lies scattered in multiple repositories. Therefore, it is important, not only for the scientific community but also for the general public, to share knowledge, which eventually will benefit researchers, professionals, as well as visitors, while offering a more complete understanding of the artefact/collection in question.

In recent years, significant efforts have been made to integrate CH knowledge stored in different repositories using Semantic Web technologies. In this direction, structured data and knowledge models have been used to define rules for storing, querying and analyzing data. The term Linked Open Data (LOD), proposed by the inventor of the World Wide Web Tim Berners-Lee in 2006, refers to data that are published and linked according to specific rules for linking structured metadata on the Web, in a way that their meaning is explicitly defined through formal semantic models in order to become machine-processable. Through LOD, datasets are linked to external data sets, and can in turn be linked to other external data sets (Bizer et al., 2011). Interlinked datasets (in other words, the LOD cloud) integrate, complement and expand the scattered information, thereby bridging repositories of different organizations and possibly of different geographic location. By adopting LOD techniques, a significant amount of remote knowledge is now available in a structured form allowing full and global access to valuable information.

Intelligent Interaction (II - http://ii.aegean.gr/) research group, established in 2016, is active in the areas of Semantic Web technologies, User eXperience and Personalization methods, Intelligent Systems and Cultural Heritage Management, and it has participated in national and european conferences with publications in reputable scientific journals in the respective research fields. Taking into consideration the urgent need for a common interpretation and management of CH knowledge and data, the II research group aims to propose new methods and techniques for classifying, preserving and promoting the CH knowledge domain, by exploiting Semantic Web technologies and Linked Open Data techniques. A milestone step towards this direction, which becomes possible by adopting these technologies, is disambiguating the definitions of concepts and relationships belonging to different sub-fields and activities of the broader CH domain. A conceptual knowledge model, integrated with reasoning mechanisms, can capture and highlight significant correlations between semantically represented data, thereby contributing to information retrieval efficiency, optimal presentation of retrieved information, decision-making support, and eventually to a common understanding of the underlying knowledge on behalf of scientists, researchers as well as common users.

Drawing on the above and being motivated by the broader vision of interconnected global knowledge, the II research group investigates methods for efficient dissemination of information and knowledge by addressing individual user needs, eventually leading to a better overall Cultural User eXperience (CUX). As such, this work focuses on investigating noel methods for presenting personalized information to the visitor, taking into account individual characteristics such as artistic background, artistic but also wider interests, as well as environmental elements describing the context of interaction (context awareness) (Konstantakis, 2018; Antoniou, 2016) . More specifically, the presented research investigates ways of making personalized recommendations to the visitor in the form of cultural paths, i.e., sets of visiting points of cultural interest (PoCI) that in conjunction formulate a cultural narrative, tailored to the visitor’s cultural background and interests. Examples of PoCIs include (but are not limited to) a particular visual artwork in a museum, an art collection or an entire cultural venue.

Technological issues

As our work so far indicates, the idea that Semantic Web technologies can be vital in defining personalized cultural paths is based on two factors: i) semantic technologies ensure data interoperability and interconnection between different online data sources (repositories), ii) semantic technologies facilitate the correlation of data with different visitor profiles. Combining personalization techniques with semantic technologies in the context of Cultural Heritage (CH) can lead to more effective presentation of cultural content, through semantic modeling of user profiles and correlation with semanticallyenabled cultural data, using a reasoning mechanism. In this respect, our main goal is to study and analyze the different issues that this approach may address in order to maximize CUX experience through semantic technologies combined with LOD and personalization methods (Deladiennee, 2017).

To begin with, personalization is the ability of a system to adapt its interface to different user profiles and requirements in order to satisfy particular needs, based on personal information. As depicted in Figure 1, the information may be provided either explicitly by the user, or implicitly by monitoring user actions (Antoniou, 2016; Bowen, 2004). In case a system requires explicitly provided information, users have to submit information about personal interests and preferences, usually by filling in surveys. On the other hand, implicit data collection doesn’t require interaction with users, who often do not realize that the displayed content is tailored to their interests, since the system extracts their preferences from monitored interaction (e.g., web usage mining, cookies, collaborative filtering, accessing by search) (Kuusik, 2009) .

Explicit provision of user profiles may be achieved with the use of predefined user profiles. According to (Falk, 2006; Morris, Hargreaves & McIntyre, 2004) , there are four different modes of visitor behavior in CHI, especially when engaging with the exhibits: ‘browsers’, ‘followers’, ‘searchers’ and ‘researchers’. The different visitor types may prefer different types of information presentation; as such, different technologies may have to be adopted to accommodate their preferences. Additionally, (Walsh, Clough & Foster, 2016) have identified different categories of users of Digital Cultural Heritage (DCH) systems and services. The authors suggest that it may be more efficient to categorize users by expertise, rather than by label or user type. Alternatively, some combination of multiple criteria could be applied. However, predefined user profiles may not correspond well to every visitor, failing to capture current user needs and expectations. Furthermore, user profiles are usually created at the beginning of a visit when visitors are usually more reluctant to carry out form-filling activities (Konstantakis, 2017) . Therefore, those methods are helpful though not always effective and applicable.

Indicative methods of capturing visitor behavior may include recording i) visiting history at different cultural spaces and GLAM’s (Konstantakis, 2017) , ii) visitor’s behavior and preferences based on user-generated content in social media (social data mining analysis), iii) visitor’s activity while moving within a particular cultural space. These methods could be expanded to include a wider frame of interaction that is inherent to the user’s cultural experience. Behavior recording can then be utilized for recommending exhibits that correspond to the visitor’s interests, experiences and knowledge background. However, defining accurate visitor personas remains a challenging issue, since it requires to rely on alternative sources for retrieving user information in order to limit the visitor’s distraction from their cultural experience to the minimum.

A similar issue emerges when a visitor uses a system for the first time. In such cases, the system will most likely fail to effectively recommend content to the user. This problem is commonly known as cold start and is a common issue in recommendation systems. Many solutions and methods have been proposed to address the cold start issue. Common recommendation strategies are based on association rules and clustering techniques (Sobhanam, 2013) , social information (Zhang et al., 2010; Noor & Martinez, 2009) , ontological classification of knowledge (Noor & Martinez, 2009) and hybrid user modelling (Wang et al., 2008) . Particularly in the CH domain, multiple methods have been proposed to address the user profiling and classification task, aiming to improve the overall UX. A rather common technique is classifying users under persona profiles based on replies to multiple-choice queries.

Regarding the utilization of LOD techniques, there are also some challenging issues that require attention. Information exchange between different data aggregators still suffers from significant flows, often due to lack of heterogeneity/interoperability in data mapping, as well as redundancy of cataloging rules and standards. In the CH domain, the significant heterogeneity degree of cultural information makes it challenging to achieve syntactic, structural and, more importantly, semantic interoperability between remote datasets and databases. Another relevant issue is the one widely known as the semantic gap (Freitas et al., 2012) . Often, critical differences can be found between the user’s informational needs expressed in a natural-language query and the underlying data representation of the targeted dataset. Therefore, creating a unifying global knowledge model of the broader CH domain, by creating and exploiting as many linked data as possible, remains an issue to date.

Defining effective user profiles is a complicated and dynamic process. In this respect, semantic modeling of user profiles and requirements can offer valuable aid. Data provided within the LOD cloud are structured using standard rules and common semantics. By utilizing formal ontologies to describe concepts related to user profiling, we can achieve a deeper, complete and more structured representation of user features, which in turn can lead to a more efficient interpretation of the user’s informational needs (Di Noia & Ostuni, 2015), and by extension to effective recommendations. Bibliographic research has shown that there have been several attempts to conceptually model the broader knowledge set that synthesizes the concept of user profile (Niaraki, 2009; Pretschner, 1999; Sieg, 2007; Skillen, 2012; Trajkova, 2004; Weißenberg, 2006; Zhou, 2006; He, 2016) , although relevant research in the context of identifying the user as a visitor remains limited. Nonetheless, there are still issues to be addressed when combining the aforementioned technologies and techniques (Semantic Web, personalization, LOD) in order to offer personalized services to groups of users, for example, performance issues when required to efficiently and timely handle large volumes of users and content, let alone if the user information (e.g., preferences, needs, requirements) is constantly changing. 3

Open Challenges

Taking into account the aforementioned issues, we aim to provide the visitor with rich and personalized cultural information, towards optimizing their overall cultural experience. In particular, we propose the implementation and recommendation of personalized cultural paths, as described in Introduction. Selecting cultural points of interest in order to form a cultural path can be based on thematic, conceptual or spatial relevance, or some combination of the above, and always in juxtaposition with the visitor’s cultural profile.

While both explicit and implicit methods for providing user information focus on user profile modeling for presenting data in a personalized fashion, it is not yet clear which method is bound to provide the most satisfying results. Based on our previous work (Konstantakis, 2018) , we argue that some combination of implicit and explicit collection of data can be proved more efficient, especially when intending to minimize distraction in UX. Additionally, we recommend the use of formal ontologies and related semantic technologies for creating conceptual schemata that incorporate semantic reasoning mechanisms, through which novel information and conceptual interrelations can be generated (Kadima, 2010; Golemati, 2007) . Finally, integrating LOD in personalized cultural paths will greatly enhance access to available cultural information from multiple data repositories, enabling useful semantic interrelations between information hosted in remote repositories and aggregators, such as EUROPEANA Digital Library (Europeana Foundation, 2018), SEARCH CULTURE Greek aggregator (Georgiadis, 2016) .

In conclusion, given that personalized cultural and art services are far from being characterized as saturated, as well as there is no online service offering personalized recommendations to visitors of a cultural site based on personal preferences in the form of a single narrative, our research aims to analyze, comprehend and eventually match the visitor’s profile and preferences with available cultural content, thereby improving their overall UX.

Acknowledgements

The research and writing of this paper were financially supported by the General Secretariat for Research and Technology (GSRT) and the Hellenic Foundation for Research and Innovation (HFRI). John Aliprantis has been awarded with a scholarship for his PhD research from the “1st Call for PhD Scholarships by HFRI” – “Grant Code 234”.

Bizer, C., Heath, T., & Berners-Lee, T.: Linked data: The story so far. In Semantic services, interoperability and web applications: emerging concepts (pp. 205-227). IGI Global (2011).

Bowen, J. P., & Filippini-Fantoni, S.: Personalization and the web from a museum perspective. In Museums and the Web (Vol. 4). Archives & Museum Informatics, Toronto, Canada (2004).

Deladiennee, L., Naudet, Y.: A graph-based semantic recommender system for a reflective and personalised museum visit: Extended abstract. SMAP 2017: 88-89 (2017).

Di Noia, T., & Ostuni, V. C.: Recommender systems and linked open data. In Reasoning Web International Summer School (pp. 88-113). Springer, Cham (2015).

Europeana Foundation: Europeana Annual Report and Accounts: A decade of democratising culture (2018). DE HAUTES ETUDES INTERNATIONALES D'INFORMATIQUE DOCUMENTAIRE (2004).

References

Falk , J. H. : An Identity-Centered Approach to Understanding Museum Learning . Curator: The museum journal , 49 ( 2 ), pp. 151 - 166 ( 2006 ).

Freitas , A. , Curry , E. , Oliveira , J. G. , & O'Riain , S. : Querying heterogeneous datasets on the linked data web: challenges, approaches, and trends . IEEE Internet Computing , 16 ( 1 ), 24 - 33 ( 2012 ).

Georgiadis , H.: SearchCulture.gr: Open access in Cultural Heritage Digital Content . National Documentation Center, presentation ( 2016 ).

Golemati , M. , Katifori , A. , Vassilakis , C. , Lepouras , G. , & Halatsis , C. : Creating an ontology for the user profile: Method and applications . In Proceedings of the first RCIS conference , pp. 407 - 412 ( 2007 ).

He , Ruining: Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation . Proceedings of the 10th ACM Conference on Recommender Systems, Pages 309 - 316 ( 2016 ).

Kadima , H. , & Malek , M. : Toward ontology-based personalization of a recommender system in social network . In Soft Computing and Pattern Recognition (SoCPaR) , International Conference of (pp. 119 - 122 ). IEEE ( 2010 ).

Konstantakis , M. , Aliprantis , J. , Teneketzis , Al. and Caridakis , G.: Understanding User eXperience in Human-Computer Interaction: UX aspects in Cultural Heritage . 2nd Pan- Hellenic Conference on Informatics (PCI'18) , Athens ( 2018 ).

Konstantakis , M. , Michalakis , K. , Aliprantis , J. and Caridakis , G.: Recommending user Experiences based on extracted cultural PErsonas for mobile ApplicaTions - REPEAT methodology . Mobile HCI 2018 , 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, Barcelona ( 2018 ).

Konstantakis M. , Michalakis M. , Aliprantis J. , Kalatha E. , Moraitou E. , Caridakis G. : A Methodology for Optimised Cultural User peRsonas Experience - CURE Architecture , 32nd Human Computer Interaction Conference, Belfast, Northern Ireland ( 2018 ).

Konstantakis M. , Aliprantis J. , Caridakis G. , Michalakis M. , Kalatha E. : Formalising and evaluating Cultural User Experience, SMAP Special Session Personalized delivery of cultural heritage content ( 2017 ).

Kuusik , A. , Roche , S. , & Weis , F. : SMARTMUSEUM: Cultural content recommendation system for mobile users . In Computer Sciences and Convergence Information Technology , 2009 . ICCIT' 09 . Fourth International Conference on (pp. 477 - 482 ). IEEE ( 2009 ).

Moraitou E. , Konstantakis M. , Kontaki C. , Aliprantis J. , Kalatha E. , Kalavrytinos P. , Tsigris A. , Tsougkrianis P. , Anagnostopoulos C. , Caridakis G. : Travelogue with Augmented Cultural and Contemporary Experience , International Conference on Digital Heritage, Cyprus ( 2018 ).

Morris , G. , Hargreaves , J. & McIntyre , A. : Learning journeys: Using technology to connect the four stages of meaning making . Birmingham: Morris , Hargreaves, McIntyre Website ( 2004 ).

Noor , S. & Martinez K. : Using social data as context for making recommendations: an ontology-based approach . In: Proceedings of the 1st Workshop on Context, Information and Ontologies , p. 7 . ACM, New York ( 2009 ).

Niaraki , A. S. , & Kim , K. : Ontology based personalized route planning system using a multicriteria decision-making approach . Expert Systems with Applications , 36 ( 2 ), 2250 - 2259 ( 2009 ).

Pretschner , A. , & Gauch , S. : Ontology based personalized search . In Tools with artificial intelligence , 1999 . proceedings. 11th ieee international conference on (pp. 391 - 398 ). IEEE ( 1999 ).

Sobhanam , H. : Addressing cold start problem in recommender systems using association rules and clustering technique . International Conference on Computer Communication and Informatics (ICCCI) , Coimbatore, INDIA ( 2013 ).

Sieg , A. , Mobasher , B. , & Burke , R. D.: Learning ontology-based user profiles: A semantic approach to personalized web search . IEEE Intelligent Informatics Bulletin , 8 ( 1 ), 7 - 18 ( 2007 ).

Skillen , K. L. , Chen , L. , Nugent , C. D. , Donnelly , M. P. , Burns , W. , & Solheim , I. : Ontological user profile modeling for context-aware application personalization . In International Conference on Ubiquitous Computing and Ambient Intelligence (pp. 261 - 268 ). Springer, Berlin, Heidelberg ( 2012 ).

Trajkova , J. , & Gauch , S. : Improving ontology-based user profiles. In Coupling approaches, coupling media and coupling languages for information retrieval (pp. 380 - 390 ). LE CENTRE Walsh , D. , Clough , P. & Foster , J.: User Categories for Digital Cultural Heritage . 1st International Workshop on Accessing Cultural Heritage at Scale, USA, pp. 3 - 9 ( 2016 ).

Wang , Y. , Stash , N. , Aroyo , L. , Gorgels , P. , Rutledge , L. & Schreiber , G. : Recommendations based on semantically enriched museum collections . Web Semantics: Science, Services and Agents on the World Wide Web , 6 ( 4 ), pp. 283 - 290 ( 2008 ).

Weißenberg , N. , Gartmann , R. , & Voisard , A. : An ontology-based approach to personalized situation-aware mobile service supply . GeoInformatica , 10 ( 1 ), 55 - 90 ( 2006 ).

Zhang , Z. K. , Liu , C. , Zhang , Y. C. & Zhou , T. : Solving the cold-start problem in recommender systems with social tags . EPL Journal , 92 ( 2 ) ( 2010 ).

Zhou , X. , Wu , S. T. , Li , Y. , Xu , Y. , Lau , R. Y. , & Bruza , P. D.: Utilizing search intent in topic ontology-based user profile for web mining . In Proceedings of the 2006 IEEE/WIC/ACM International conference on web intelligence (pp. 558 - 564 ). IEEE Computer Society ( 2006 ).