Visualise Semantic Story Maps

Abstract

This paper presents the Story Map Building and Visualizing Tool (SMBVT), a software that allows users to create and visualise semantic story maps using a user-friendly web interface. The tool uses Wikidata as external reference knowledge base and exploits Semantic Web technologies in the back-end system to represent stories modelled on the Narrative ontology, a CRM-based vocabulary for representing narratives. SMBVT is entirely open-source and accessible after free registration.

-

Ontology

1. Introduction

In this paper, we describe the Story Map Building and Visualizing Tool (SMBVT) that allows users to create semantic stories in the form of story maps and timelines using a user-friendly web interface. SMBVT exploits Semantic Web technologies in the back-end system to represent stories modelled on the Narrative Ontology [ 1 ]. It builds up a user-shared semantic knowledge base that automatically interconnects all stories. SMBVT is an extension of a previous version named Narrative Building and Visualising Tool [ 2 ] (NBVT). NBVT only supported narrative representation as a timeline and did not support story maps; it required story events always to have temporal information associated and did not support spatial information. This limitation allowed us to improve the software and create SMBVT, which supports both timeline and story map representations. Overall, the newly functionalities of SMBVT can be summarised as follows: (i) timeline and story map visualizations are supported as story representations; (ii) temporal information is no longer mandatory for the story events; (iii) spatial coordinates can be indicated for each event; (iv) media objects IRIs (images and videos) can be attached to the events; (v) the event sequence can be ordered either by date (fabula) or user-defined order (plot); (vi) an event ”category” can be indicated, which corresponds to distinct icons and colours on (N. Pratelli) the story map; (vii) a story map preview can be visualized before publication; (viii) the stories can be directly published online.

A more thorough description of SMBVT that includes the technical requirements and functionalities, along with the assessment of both narrative building and visualisation interfaces, is presented in [ 3 ]. SMBVT represents narratives as a network of spatiotemporal events related by semantic relations (part-of, temporal and causal relations). In particular, a narrative consists of three main elements: 1. the fabula, i.e., the story itself as it happened, in reality or in fiction, as defined by Russian formalism [ 4 ]; 2. the narrations, i.e., one or more expressions, each in its own language and medium, which narrate the fabula. Each narration corresponds to Bal’s definition of presentation [ 5 ]; 3. the reference, i.e., a relation that connects (fragments of) the narrations to (fragments of) the fabula, allowing the derivation of the plot (or syuzhet ) of the narrative, as defined by Russian formalism [ 6 ].

A narrative can be visualised as a timeline (if event dates are provided) or a story map. A timeline shows the events temporally ordered (the fabula). A story map is instead a sequence of geospatial events ordered by a narrator (the plot ). SMBVT standardizes the event descriptions by assigning internationalized resource identifiers (IRIs) to the event components, i.e., the entities that take part in the event (e.g., persons, objects, places, and concepts). The tool automatically saves the collected knowledge as a Web Ontology Language (OWL) [ 7 ] graph and openly publishes it as Linked Open Data [ 8 ]. The story maps created through SMBVT greatly support storytelling about territories, describing a territory beyond its map [9] by enriching the map with text, pictures, videos, and other multimedia information. SMBVT is entirely open-source, the code is available on GitHub [10], and accessible after free registration 1 or in a demo version (with a limited number of functionalities) without credentials2. To demonstrate the potential of our tool to convey information going beyond the map for scientists, stakeholders, and the general public, SMBVT was used to create diferent stories in diferent scientific domains. These stories are freely accessible on-line3. The examples shown in this paper are retrieved from the story of the medieval journey of Leonardo Bruni, an Italian humanist who travelled from Rome (Italy) to Konstanz (Germany) in 1414. Bruni describes the journey in a letter to his friend Nicolò Niccoli. The letter is in Latin and titled Epistola de peregrinatione ad urbem Constantiam.

2. Related Works

This Section presents an overview of the main software for creating and visualizing story maps. We focused on software that represents maps as sequences of events with associated multimedia content. The first software we report is ArcGIS Storymap. It is a widely used software that allows creating, publishing and sharing interactive map-based narratives [11]. This software allows 1https://tool.dlnarratives.eu/ 2https://dlnarratives.eu/demo.html 3The stories are published on https://dlnarratives.eu/narratives.html and https://moving.d4science.org/group/ moving_storymaps (accessible after free registration to the D4Science platform) customizing maps by adding text, photos, audio, and video information. Several commercial versions are available, but ArcGIS also ofers a free version 4. This version has advanced features disabled, e.g., external Web page embedding, advanced map-interaction facilities, theme customization, and controlled accessibility. Moreover, it does not allow importing story entities from existing knowledge bases (e.g., Wikidata).

Timescape5 is a map-based storytelling platform that allows the enrichment of events with text, images, and external links. Timescape is commercial software but ofers a free version that allows creating up to five story maps. StoryMapJS 6 is a free online tool to build story maps, also available as a JavaScript library. The online tool ofers an editor accessible through a personal Google account. StoryMapJS story is organized as a sequence of slides, where each slide represents a story event with a specific place associated. Furthermore, media contents can be associated with the events. A widely used approach to story map building is to modify a preexisting HTML template that internally uses libraries for map visualization. For example, story maps based on the Leaflet Storymaps template 7. The Leaflet Storymaps template is designed to guide readers along a point-by-point tour, with a scrolling narrative to display text, images, audio, video, and scanned map backgrounds. Users can create their story maps by entering the map data into a linked Google Sheet (or CSV file).

A hybrid approach between HTML coding and story maps building is ofered by two opensource software: TimeMapper [ 12] and Odyssey.js8. TimeMapper produces story maps taking as input an external CSV template file. Odyssey.js allows users to create story maps using the Markdown language. It includes a small number of webpage templates to help users quickly create their stories.

In comparison with SMBVT, the main limitation we identified in the tools reported in this Section is that they do not support story inter-linking and information storage on semantic knowledge bases. In summary, SMBVT introduces new features for story map building, such as (i) Semantic Web technologies, (ii) a completely free-to-use Web tool, and (iii) automatic connection to Wikidata. These features make SMBVT overcome the common drawbacks of other commercial and open-source solutions.

3. The Story Map Building and Visualising Tool

3.1. Web Interface for Story Map Building

SMVBT ofers a web graphical interface to create stories. At the story creation time, the user writes a title corresponding to the subject of the narrative. The system automatically associates this title to a Wikipedia page (subject entity). The tool then queries Wikipedia (through Wikipedia APIs) to extract all Wikipedia pages directly linked to the subject entity. Finally, the tool queries Wikidata (through its SPARQL Query Service) to associate names, descriptions, and images to the entities included in the Wikipedia pages. Since Wikidata is 4https://doc.arcgis.com/en/arcgis-storymaps/reference/licensing.htm 5https://www.recombine.net/projects/timescape.html 6https://storymap.knightlab.com 7https://github.com/atlefren/storymap 8http://cartodb.github.io/odyssey.js/ an ontological knowledge base [13], this operation also retrieves Internationalized Resource Identifiers (IRIs) for the extracted entities. The user can also define new entities with associated names, descriptions, images, and Wikidata IRIs. After the initialisation phase, the tool displays the extracted Wikidata entities related to the subject of the narrative on the left-hand side of the screen. These entities are classified into seven categories: person, organisation, object, concept, work, and other (e.g., animals, fictional characters). They correspond to the top classes of the Narrative Ontology. The right-hand side panel allows the creation of the story events. Each event can involve a subset of the entities from the left-hand side part of the interface. Additionally, the user can complete the entity-definition form by adding the following information: The event title, the temporal information (start and end dates), the geographical coordinates (latitude and longitude), an event categorization, a descriptive text, the principal media object IRI (e.g., a YouTube video), links to digital objects collected in external knowledge bases. Figure 1 shows the tool interface reporting an event from the story map about Leonardo Bruni’s medieval journey.

3.2. Back-end Services

SMBVT retrieves events, entities and all associated data from a PostgreSQL-JSON database for visualisation. It synchronises this database with an OWL-graph representation of the stories stored on an Apache Jena Fuseki server [14]. The Fuseki server provides a SPARQL endpoint to query the complete graph of collected stories. This server organises the stories as the sub-graphs of one overall story graph. The server allows executing SPARQL data-extraction queries [15] on the entire story graph within or across the stories. In particular, based on the SPARQL server, SMBVT ofers a search functionality (Figure 2) that allows querying the entire knowledge base in a user-friendly way. The web interface allows users to search for an entity (helped by automatic completion) and retrieve the following information based on predefined SPARQL queries: (i) all stories in which the entity appears; (ii) all the events of the stories in which the entity appears; (iii) the number of entity occurrences across all stories; (iv) the co-occurring entities across all events of all stories.

3.3. Story Map Visualisation

SMBVT visualises the produced stories as story maps placing the narrative events on an interactive map that respects an event browsing order based on the user-defined plot. Each story event is associated with one positional pin, one image/video, a colour and pin style (depending on the event type), one title and descriptive text, several Wikidata entries (representing persons, locations, organisations, and other entities that occur in the event), and external digital objects (e.g., Web pages and Europeana objects). The software uses a customised version of the StoryMapJS library9 for map interaction, event browsing, and visualisation. StoryMapJS allows managing large background maps and images associated with the events and can visualise stories represented as JSON documents. At the story-map loading time, SMBVT on the fly translates its PostgreSQL-JSON story representation into a StoryMapJS JSON-compliant document. Figure 3 shows the event regarding the departure of Leonardo Bruni from Verona as represented on the story map. The pin highlighted in red represents the current event on the map. On the right side of the map, the title of the event (i.e. Verona, which is the place of departure), an image of Verona, the dates of the event, its description and the related Wikidata entities are reported. Together with the story map visualisation, SMBVT also supports story visualisation as an event timeline, when temporal information is available for the events. Timeline visualisation is based on a customised version of the TimelineJS library10. The SMBVT story map publication process automatically generates a web application that embeds all required JavaScript libraries, instructions and styles, story-related JSON documents, and event images. The publication process returns a public link to the web application. Each publication operation overwrites the 9https://storymap.knightlab.com 10https://timeline.knightlab.com/ previously published application so that the public link always points to the latest story-map version. This operation is necessary to support the continuous updating and enrichment of the story while ofering the users always the latest version. Therefore it guarantees a long-term story’s maintenance, usability, and accessibility. At the moment, one user at a time can review and modify a story map. [9] A. Korzybski, A non-aristotelian system and its necessity for rigour in mathematics and physics, in: Science and sanity: an introduction to non-Aristotelian systems and general semantics, Lancaster, 1933. [10] ISTI-CNR, The SMBVT source code, on-line, 2022. Available at https://github.com/

EmanueleLenzi92/SMBVT. [11] N. Walshe, Using arcgis online story maps, Teaching Geography 41 (2016) 115–117. [12] T. Becker, B. Köbben, C. Blok, Timemapper : visualizing moving object data using wms time and svg smil interactive animations, in: Proceedings SVGOpen 2009, SVGOpen, 2009. 7th international conference on scalable vector graphics (SVGOpen 2009), 02-04-10-2009, Mountain View, United States. [13] D. Vrandecic, The rise of wikidata, IEEE Intelligent Systems 28 (2013) 90–95.

10.1109/MIS.2013.119. [14] A. Jena, Apache jena fuseki, The Apache Software Foundation 18 (2014). [15] M. J. O’Connor, A. K. Das, Sqwrl: a query language for owl., in: OWLED, volume 529, 2009.

References

[1] C. Meghini , V. Bartalesi , D. Metilli , Representing narratives in digital libraries: The narrative ontology , Semantic Web ( 2021 ) 1 - 24 . 10 .3233/SW-200421.

[2] D. Metilli , V. Bartalesi , C. Meghini , A wikidata-based tool for building and visualising narratives , International Journal on Digital Libraries 20 ( 2019 ) 417 - 432 . Https://- doi.org/10.1007/s00799-019-00266-3.

[3] V. Bartalesi , G. Coro , E. Lenzi , N. Pratelli , P. Pagano, MOVING D3 . 3 - Tools for sciencesociety-policy interfaces. Using semantic story maps to describe a territory beyond its map , Technical Report, ISTI Research Report, MOVING, D3.3 , 2022 .

[4] V. Shklovsky , et al., Art as technique , Literary theory: An anthology 3 ( 1917 ).

[5] M. Bal , Narratology: Introduction to the theory of narrative , University of Toronto Press, Toronto, 1997 . ISBN: 978 - 0802096319 .

[6] P. Petrov , Fabula/sjuzhet, The Encyclopedia of Literary and Cultural Theory ( 2010 ).

[7] D. L. McGuinness , F. Van Harmelen , et al., Owl web ontology language overview , W3C recommendation 10 ( 2004 ).

[8] C. Bizer , T. Heath , T. Berners-Lee , Linked data: The story so far, in: Semantic services, interoperability and web applications: emerging concepts , IGI global , 2011 , pp. 205 - 227 . 10 .4018/jswis.2009081901.