The ever increasing amount of information available online is posing an enormous challenge for information and content curation professionals whose professional job profile includes analysing or making use of the information including understanding the, in a wider sense, storyline of an event or series of events. The problem of identifying or generating narrative structures in a robust way is yet to be solved. There is a big need for tools that support users to automatically identify, interpret and relate the elements of a (fictional or real) narrative.
In that sense, many professionals have to cope with huge amounts of incoming information and content that need to be processed (scanned, skimmed, contextualised, evaluated and, eventually, further processed) in a short amount of time in order to produce a new piece of information, for example a news article, a social media post, a longread or a press statement -let us call this group "knowledge workers" or "content curators". Generally, they still need to do a great amount of work intellectually. If automatic tools are available, these are often restricted to specific tasks, for example, keyword-based alerting or named entity tagging.
This position paper describes the conceptual design and partial implementation of a Semantic Storytelling prototype [RMSB + 18]. Our aim is that the system will be able to process an incoming set of textual data (e. g., a document collection) using several semantic analysis technologies in order to generate a large variety of semantic annotations that can be exploited for the purpose of semantic storytelling. This involves supporting users through various interactive and dynamic data and content exploration methods that rely on abstract story knowledge.
This article is structured as follows. Section 2 covers related work. In Section 3 we describe our vision regarding Semantic Storytelling and Section 4 concludes the article and discusses the most relevant conceptual, scientific and technical gaps.
Several approaches are closely related to our Semantic Storytelling concept and vision, all of them concentrating on their own specific objectives and providing solutions for their respective challenges. Ours is to enable, ideally with limited or no human intervention at all, the identification of plots or storylines based on text collections for which we generate rich and deep semantic annotations.
Some approaches focus on extracting information using NLP techniques in order to use it, in later stages, for generating stories. An unsupervised approach for clustering news articles based on identified event instances is presented by Ribeiro et al. [RFT17], while Li et al. [LZY17] present a supervised prediction model to analyse different strength levels of claims in science news as a fact-checker. The NewSum Toolkit [VM15] is a combination of NLP and Machine Learning technologies supporting a number of steps for news article writing like gathering data, automatic classification and summarisation of large amounts of incoming articles. More complex approaches are used by Yarlott et al. [YCGF18] based on the hierarchical theory of discourse by van Dijk [vD88], or by Dai et al. [DTH18], where a content representation structure of the documents is used to build a first predictive model using these indicative structures as features. News recommendation has gained attention in works such as Cucchiarelli et al. [CMSV18], where journalists get recommendations by taking an event and checking if they got a greater echo in Twitter or Wikipedia postings, or in Bois et al. [BGJ + 17], where newspaper articles are recommended based on lexical similarity, linked through a graph representation of relations.
A different class of systems is mainly oriented on providing content or applications for entertainment purposes. For example, Wood [Woo08] uses a collection of pictures and other media to generate albums, Gervás [Ger13] focuses on gaming. Other groups use story structuring methods as part of therapy programs [KBE14], while other approaches focus on "storytelling" or, rather, text generation, in a particular domain, typically recipes [CLNU13,Dal89] or weather reports [Bel08, RSHD05, TSRD06], requiring knowledge about characters, actions, locations, events, or objects [GDAPH05,RY10,Tur14]. A notable exception to this approach, where domain knowledge is a prerequisite, is [LLUJR13], who attempt to construct plot graphs from a set of stories annotated using crowd sourcing. Some authors include the order of events [Cha11].
For the demanding question of how to generate a story grammar which orders events detected in a previous step into storylines, many approaches are based on theories taken from literature studies, more precisely, narratology. For example, Caselli and Vossen [CV17] use the plot structure as described by Bal [Bal97] for a chronological and logical ordering of events, Yarlott et al. [YF16] and others used Propps morphology of Russian hero tales [Pro68] as theoretical background for story detection and generation systems. We plan to experiment with the concept of text genres, specifically text-structural conventions, to get a better understanding of structure in texts and to better extract the main events inside these structures that often exhibit specific communicative functions. One approach describing text genres according to their communicative functions can be found in [Sha18].
Another important component of our Semantic Storytelling vision is the graphical user interface, which will enable users to interact not only with the information that has been analysed but also with the generated storylines. Examples of final story visualisations are Ma et al. [MLF + 12] or Segel and Heer [SH10], who present several options. Novel visual interactive methods are presented in Kybartas and Bidarra [KB15]. In contrast, an approach focused on the content management is presented by Mulholland et al. [MWC12]. Storyteller visualises complex relations between events found in newspaper articles [vMVvdZ + 17], while user interactions are limited to filtering the data set.
This section presents our Semantic Storytelling vision including the concept, an indicative use case and the different components needed for the development of a complete system.
Storytelling is a human technique to order a series of events in the world and find meaningful patterns in it [Bru91]. By telling a story we relate events into a schematic structure, for example, in terms of topic, locality or causal relationships, and construct explanatory models of the world and events. Semantic Storytelling can be seen as the attempt to translate the theories of storytelling into a formal, and machine-processable scheme.
Storytellers dynamically adjust their narratives and tell their stories differently depending on who the listener is [RLEW13]. The most simplistic goal of storytelling is the automatic (or semi-automatic) generation of stories, where a story is considered a natural language text containing a complete, correct and unambiguous story. The definition by Rishes et al. [RLEW13] splits storytelling into semantic content generation and natural language generation.
We rather see a storyline as a set of building blocks, which depending on their combination (temporal, geographical, semantic, causal) form a story, which allows us to provide a wider range and more flexibility for suggesting storylines. Our goal is to develop a suggestion-or recommender system that allows the, ideally, automatic arrangement of (named) entities, i. e., conceptual instances, and events, within a storyline, where users benefit from a recommender system and a controlled context and navigation tool.
The following example is meant to illustrate the functionality of an "ideal implementation" of the Semantic Storytelling system we have in mind. Tens of thousands of books on the Second World War provide detailed information on events that involve different persons, places, alliances etc. A historian, journalist or author working on the topic needs to be able to order and arrange this vast amount of content in an intelligent way to create new content. An ideal system can support the understanding of historical interactions and relationships. The goal is to identify all persons, places and events, to position events on a timeline, also to identify the causal, temporal etc. relationship between different events. While Natural Language Processing is not yet able to perform these tasks without any errors, we firmly believe that the application of state of the art methods can provide a benefit to the user, for example, by following the storylines of individual persons, exploring their relationships with others, focusing upon specific events, scrolling backward or forward in time.
The abstract architecture of our system is composed of three main building blocks: Semantic Analysis, Text Genre-specific Story Knowledge and Semantic Generation (Figure 1). In the following, we briefly describe the three sets of components, especially concentrating on the conceptual and technological gaps.
This building block involves various processing steps that relate to the annotation, extraction and classification of certain parts of the incoming content in order to enrich the documents, for example, by adding semantics and information taken from external sources. Named Entity Recognition, Named Entity Linking and Time Expression Analysis are needed to identify named entities of various types and classes (Persons, Locations, Organization, Others) and to anchor the content to a timeline. Extracted entities, topics etc. will be linked to external knowledge graphs (e. g., DBPedia,1 Wikidata,2 Geonames,3 ). A robust approach at Topic Detection is needed in order to assign abstract topics to, say, individual sentences, paragraphs, chapters and documents. Annotated topics will enable yet another different layer of accessing and recombining the processed content. Managing the linguistic annotations in a Linked Data format (we use NIF in our current prototype [HLAB13]) allows the exploitation of Linked Open Data for storyline generation. While robust Event Detection with a high coverage, carried out at the same level of semantic abstraction, is still beyond the state of the art of Natural Language Processing, such a module is crucial to enable the re-composition of storylines out of a large and heterogeneous set of identified events. Automatically anchoring events to a timeline is also beyond what is possible right now fully automatically. To analyse a wide variety of incoming documents, we need to be able to process different classes or genres of documents, we need to identify and work with Discourse Structure, we need to identify the genre or type of a document, we need to be able to distinguish fact from fiction. While
As previously mentioned, Semantic Generation involves the dynamic and interactive recomposition and visualisation of extracted information based on the information extracted from the Semantic Analysis step. This especially involves arranging content elements (documents, paragraphs, sentences, claims or events) on a dynamic timeline. Summarisation techniques can be used to compress larger pieces of content into bites that can be easily digested, moved around on the screen and maybe expanded back into longer or their original versions. The principles by which the actual construction of storylines based on the recomposition of previously extracted information will be performed, is still an open question. In contrast to template-filling approaches [MSBR17], we will focus on approaches that are based on computational narratology to generate narrative structures of story lines, while using automatically extracted information and external knowledge provided as Linked Open Data [RMSB + 18].
The most crucial missing conceptual piece of our Semantic Storytelling vision is, critically, what we call Text Genre-specific Story Knowledge for advanced text and discourse-informed document processing. This includes technologies and approaches for representing and identifying the structure, patterns, sequences and abstract entities of different types of stories and to make this explicit knowledge available to corresponding analysis and, later, generation components. Therefore, we will build generic processing pipelines for different text types, which will allow us to handle typical features found within these. The idea is to apply a generic processing pipeline optimised by the characteristics of the respective text genre. While for news, for example, a timeline-based event ordering would be useful, for journal articles discourse structure aspects like discourse parsing and claim extraction would be the key aspects a knowledge worker would be interested in.
Text genre-informed information retrieval will make it possible to process each text according to its typical text genre-specific structure and communicative function. Using this knowledge we can more precisely extract the most important lexicogrammatical realisations which express these main commmunicative aims, for example to inform about an event, to discuss a scientific claim or to present a story etc. (cf. Sharoff's classification of web text genres [Reh02], where text genres are defined by generalised communicative aims and not predetermined by lexicogrammatical realisations [Sha18]). This approach will allow us to extract events and order them in a flexible way, addressing the needs of the respective use case and document collection.
Semantic Storytelling can be conceptualised as the automatic (or semi-automatic) generation of different storylines based on information extracted, classified and annotated within extensive textual data sets or document collections [BMSN + 16]. We have developed a number of initial prototypes that demonstrate part of the functionality needed [RHS + 17, MSBR17, SBN + 16, RMSB + 18].
Our goal is the development of a prototype platform that will support knowledge workers in the complex and time-consuming task of handling, evaluating, processing, sorting and processing document collections in order to generate new pieces of content. One goal of the platform is to enable users to identify interesting stories as efficiently as possible based on the (extracted) information available.
We are trying to pinpoint the key open questions in order to suggest a roadmap for Semantic Storytelling for the next years. While technologies such as Named Entity Recognition and Linking, Time Expression Analysis, Topic Detection and Text Classification have been in production use in many different applications for years, important components such as Event Detection but especially more advanced discourse analysis tools including Rhetorical Structure Analysis and Text Genre detection must still be considered avantgarde and not ready for production use yet. While research on such technologies is making progress, the wider field of Natural Language Understanding and Language Technology still needs to fully discover and embrace the relevance and importance of what we call Text Genre-specific Story Knowledge for truly advanced text and discourse-informed document processing. In our future work we will concentrate on the development of corresponding technologies and knowledge representation approaches.
The project QURATOR is supported by the German Federal Ministry of Education and Research (BMBF), "Unternehmen Region", instrument "Wachstumskern" (grant no. 03WKDA1A).