Writing is a communication stream through which authors enrich, entertain and educate the audience about a specific topic. For ascertaining facts in the pre-writing researching phase, various search engines provide relevant results with titles, descriptions, links and tags along with a range of statistical charts and snippets. But these results do not comprehend underlying resources to aid individuals' metacognition. Metacognition (individual's reflections about their own knowledge [1]) can be improved by cognitive control which can be introduced by purposeful goal-oriented behaviour and decision-making process [2]. The "Terms Board" (TB) demo presented in this paper accommodates cognitive control from two perspectives: story planning and timeline. While story planning templates like Joseph Campbell's "The Hero's Journey" guide the authors in organizing their plans [3] for better writing, the timeline In: R. Campos, A. Jorge, A. Jatowt, S. Bhatia, M. Litvak (eds.): Proceedings of the Text2Story'23 Workshop, Dublin (Republic of Ireland), 2-April-2023 Envelope isami001@gold.ac.uk (I. R. Sami); t.russell-rose@gold.ac.uk (D. T. Russell-Rose); l.soldatova@gold.ac.uk (Prof. L. Soldatova) relativity of the terms plays a major role in understanding their factual relativity. As the use of a storyboard improves individual/collective engagement and expressing ability [4], TB visualizes the terms as a board. Based on the "The Hero's Journey" template, TB displays six cards to reflect story planning aspects: what the story is about, who are the characters of the story, where the story is located, why there are challenges, what has been done to address the challenges and why the actions were effective. Each card contains three packed bubbles of three top terms in that category to reflect the timeline aspect: historical, consistent and latest. By doing this, TB is visualizing a palette of thoughts for metacognition during news analysis and editorial writing. Therefore, the major contributions of this article are: the introduction of a concept of comprehensive Terms Board (TB) visualization as a cognitive guide and reporting an experimental evaluation of TB via cognitive reading and writing user experiment tasks.
Applications can facilitate information seekers by providing information through search facilities or automatically generated comprehensive categorization/visualization [5]. Global and professional domain-specific search engines play a vital role in knowledge-seeking by providing various advanced search options, categorical filters and statistical information [5]. On the other hand systems like Open Knowledge Maps (OKM) [6][7], WordStream [8] and VISTopic [9] comprehend metadata of the search result categorically using comprehensive visualization. Existing search and comprehensive technologies are focused on knowledge discovery, while TB is focused on analyzing and planning knowledge delivery, stimulating creativity and improving the engagement of the audience by aiding individuals to create a guided story plan.
TB categorizes terms into the following story planning aspects. Who: role players in the documents are identified by the category "Person" or "Organization". Where: key locations in the documents are identified by the category "Location". They are represented in proper nouns. What (topics): main topics are identified by emphasized nouns and proper nouns. Why (negatives): challenges are identified by a static list for the prototype system. What (actions): verbs that reflect the actions in the documents. Why (positives): words identified by a static list for the prototype system. Each story planning category forms a card in TB. A card is further separated into three timeline-based aspects. Historical: if a term appears early in the given date range, it gets a higher weight. Latest: if a term appears later in the given date range, it gets a higher weight. Consistent: these are the terms that appear consistently both historically and in the latest documents. The weight is the average of historical and latest weight.
For displaying terms in TB, emphasized terms are extracted from a topic-based collection of documents and displayed as shown in Figure 1. Each document is analyzed in the archive using the topological approach presented in the paper "A simplified topological representation of text for local and global context" [10] to identify topics and actions (verbs) from a document. The identified terms are categorized into six story-planning aspects. For an extracted term, the classification type (Person/Organization/Location) of the term is identified by the categorical information provided by Google Knowledge Graph API [11]. The extracted emphasized term are combined as a list of highly frequent related terms using frequency analysis. The algorithm uses the category of terms to place them in appropriate cards and weights each term on the timeline-based aspects to select the top terms of that aspect. TB visualization is generated using the D3 library's packed bubble chart [12]. On hovering over each timeline bubble, full terms are displayed. Clicking a term opens a relational view modal with a related set of documents (title, meta-description and link to the main document). We applied the scanning approach to display related terms. Scanning helps to locate specific facts rapidly [13]. For a given set of terms, all the sentences in the relative context are scanned based on word co-occurrence, top 5 relations for each given term are displayed using the D3 library's Sankey [13] diagram when a timeline bubble is clicked. On clicking a relation, the set of documents (title, meta-description) that reflects the relation is displayed. When clicking a document, a user navigates to the original document/news.
We evaluated TB via an online participation-based experiment. We used a pharmaceutical news website to build a collection and produce the visualizations. Each participant from a homogeneous group was given two story writing tasks on pre-decided topics. The selected two topics were Pfizer and AstraZeneca. For both of the tasks, the participants were given a list of documents (title, introduction and link to full document text) to plan a story and write a minimum of 500 characters about the topic. But for one of the tasks, we presented TB visualization generated based on the documents having the topic before the list. The topics were randomly assigned to the control condition. The order of the tasks was generated using "Latin Square Design" to balance the systematic difference between successive conditions. We invited three academic reviewers to blindly score the writing tasks. They scored the quality of the story on a scale from 1 to 5, where 1 stands for poor and 5 stands for very good. We took the average score given by the reviewers for each task. We performed a paired t-test on the average scores of quality of writing, ease of use and completion time. We recruited 32 participants for this experiment.
The result is reported in Table 1. As shown in Table 1, we have achieved 95% confidence in the reported result on all criteria apart from "Quality of Story". According to the evaluation, the participants spent more time writing their stories and achieved a better quality of writing when they are assisted with TB visualization compared to the tasks where they were given only a list of documents. They found the task assisted by TB comparatively difficult as they had to process additional information than only scanning the documents.
Clearly, participants' thoughts can't be directly observed. Therefore, information about metacognition must be collected in indirect ways [1]. We used Google Analytics to record participants' activity while doing the task with their permission. We collected 17 participants' activities.
We tracked clicks and hovers on cards, timeline bubbles and documents during reading and selecting documents in the story planning phase. We also tracked how long the selected document was open during the session. Using this information, we generated directed graphs. The red nodes represent the topic of the story (Pfizer or AstraZeneca)which is the root node. The amber nodes represent story planning aspects (whatTopicBox, whatActionBox, whoBox, whereBox, whyNegativeBox, whyPositive Box). The yellow nodes represent the timeline aspects (consistent, old_to_new/historical, new_to_old/latest). The green nodes represent documents. The size of the nodes represents the number of events/time associated with the node during the pre-writing story planning phase. Figure 2 (a) displays the story planning directed graph of a participant while writing a story using only a list of documents. The participant selected 6 documents from the list to plan their story and then gradually increased their focus on 2 documents to base the writing. participant while writing a story using TB. The figure shows that the participant investigated five cards to start with and then selected the Who card's consistent terms to select a document for reading in the planning phase. All collected observations from analytics show that the participants selected fewer documents via TB than the list of documents which refers to the fact that they selected the documents using a strategic planning process. Therefore, combining this observation along with the results reported in Table 1, we can state that for a group of participants, TB contributed to more metacognition (increased time for finishing the task) and better quality of writing by aiding planning before writing a story.
TB is designed to comprehend facts and aid in improving an individual's metacognition to create a plan for a designated task based on a strategy. TB has the potential to contribute to news analysis and editorial writing. TB can expose the metacognition process while writing. In this paper, we demonstrated TB using a news corpus and showed that TB contributed to metacognition and quality of writing by aiding planning before writing a story. TB can also be used for other text corpora such as articles and books as an aid for brainstorming in learning and teaching.