The comment sections of online news platforms have shaped the way in which people express their opinion online. However, due to the overwhelming number of comments, no in-depth discussions emerge. To foster more interactive and engaging discussions, we propose our ComEx interface for the exploration of reader comments on online news platforms. Potential discussion participants can get a quick overview and are not discouraged by an abundance of comments. It is our goal to represent the discussion in a graph of comments that can be used in an interactive user interface for exploration. To this end, a processing pipeline fetches comments from several diferent platforms and adds edges in the graph based on topical similarity or meta-data and ranks nodes on metrics such as controversy or toxicity. By interacting with the graph, users can explore and react to single comments or entire threads they are interested in.
In the past, newspaper readers could only interact with and express their opinion on an article by writing a letter to the editor. The editor could then decide to publish and/or to reply to the letter in the next issue of the newspaper. The considerable efort of writing and mailing such letters, was a natural limiting factor for the number of interactions. Now, users of online news platforms can easily post comments and discuss article topics with others. The simplicity and ubiquity of expressing one’s opinion online was therefore termed as democratization of opinion. On the downside, readers can be overwhelmed by the volume of comments. Repeated arguments, Troll comments, or attention-seeking unrelated opinions hinder the emergence of meaningful discussions. Long discussions across multiple pages may discourage readers from scrolling through more than the top ten comments.
We envision a platform that focuses on providing a space for discussions where people listen to and refer to each other’s comments. To this end, we part from a traditional “linear” list to a two-dimensional canvas that groups comments using diferent features for a better overview. This allows for new interaction paradigms that could inspire readers of news comments to engage in an already ongoing discussion.
visualizing of and interacting with online discussions. We present a novel concept of stipulating engagement through improved information visualization. In this regard, we identified three components that are crucial to reach this goal: 1. More engagement: more users who were passive in the past should become active contributors in discussions. 2. More in-depth: more comments should refer to one another and more dialogues should emerge. 3. More insights: users should read more relevant and less redundant or of-topic comments.
are also currently preventing a better user experience. Online discussion spaces are fragmented across various individual platforms. Although the topics discussed are typically similar, e.g. daily news events. We are the first to introduce the idea of a common, shared discussion platform with the goal of increasing engagement of discussion participants and facilitating interaction. To this end, we present a novel interface for exploring large amounts of reader comments across diferent news platforms. The core of the visualization is based on a graph representation of comments, where nodes are sentences and edges describe how they relate to one another. This graph allows us to incorporate several views on the data and enrich the comments with syntactic and semantic features, such as topical similarity or temporal proximity. It is our goal, to find a graph representation that captures arguments and the evolution of the discourse. By clustering, filtering, and merging, the interface enables users to reduce the complexity by exploring the comments at diferent levels or granularity.
The following sections provide an overview of the system architecture of ComEx and describe the visualization paradigms behind it. Furthermore, we discuss our work in progress towards a meaningful graph representation and showcase initial results applied in a case study on reader comments about bushfires in Australia. 1
1 Comment • Comment text, user Metadata • Votes, replies, timestamp • Semantic (textual, …) • Syntactic (temporal, …) Filtering & Merging • Emphasize latent structure Clustering & Classification • Semantic clustering • Toxicity classification • Engagement prediction • Ranking
of written discourse and relevant text mining methods. 2 4 The goal for our interactive visualization is to form visual clusters of comments that make it easy to comprehend the inherent semantic structure of a large set of 3 comments. To achieve this goal, the underlying layout model reflects not only the structural information, e.g., 5 sentences belonging to the same comment, but also the key topics and arguments made. Appropriately mapping Figure 1: The ComEx system transforms comments into a the nuances of a discussion, the size of the dataset, and graph structure and allows their exploration through a web the text lengths pose as hard problems for language interface. models. Attempts using topic models have been made to visualize political speeches [1] as moving particles or text collections as glyphs symbolizing topic distributions [2]. additional functionality in the form of data processing Others use document embeddings and dimensionality modules. As an example, we incorporate a comment reduction to create a partial map of Wikipedia articles [3] classification approach that identifies main causes for or scatterplots of forum posts [4]. Both examples are increased user engagement based on comment texts [12]. not applicable here, as they rely on a large, manually Thereby, ComEx can highlight engaging points of a labeled dataset. We propose to use pre-trained sentence discussion that are likely to trigger many user reactions. embeddings. All comments of one story are clustered More distantly related is work that supports exploratory into key discussion points. The layout within each cluster search through scientific articles [ 13]. A comprehensive is done using attracting and repelling forces between overview of the characteristics of exploratory search has particles based on sentiment or keywords. Thereby, we been published by Palagi et al. [14]. benefit from sentence embeddings to get a global layout and achieve a nuanced local layout by using mined metadata (clusters, keywords, sentiment, etc.). 3. System Overview and
Related work in the area of text mining forms clusters Paradigms of comments mentioning the same entities [5] or and visualizes discussions with pie charts [6] or topic-model- The ComEx system implements a novel concept of based graphs [7]. Zhang et al. [8] focus on summarizing interacting with and getting an overview of the growing social media posts to provide aggregates of all reposts and number of reader comments in online news discussions. replies in a conversation. They form pseudo-documents Figure 1 shows the system architecture. The data as context used in an encoder of a recurrent neural net- ingestion pipeline scrapes comments from diferent news work from which the summary is generated. Leveraging platforms (1). The comment texts and their metadata, sentiment analysis and stance detection, there is also such as upvotes, references to other comments, or timesrelated work on allowing users to search for diverse tamps are then stored in a relational format (2), which perspectives on the same topic [9, 10]. In their analysis is cached (3) to reduce the load on the news platforms. of millions of comments, Ambroselli et al. [11] identified This data is transformed into a graph structure, where three main causes for increased user engagement: reac- the nodes (sentences of comments) and edges (relations tions to personal stories, hate speech, or comments by between sentences) are processed with text mining and the article’s author. Our system allows integrating such graph analysis algorithms, such as semantic clustering, TextRank [15], and toxic comment classification (4).
The results are sent back to the cache. A web-based 1Interactive demo and code available at https://hpi.de/ naumann/s/comex user interface allows exploring the enriched graph at selection in the interface to filter the list of comments. diferent levels of detail (5). The architecture of the Afterwards, users can jump back to the original platform system is designed in such a way, that text and graph to comment, or react to a selection of comments directly processing modules are interchangeable and can easily in the visualization. be configured. More details on that are highlighted in the following section. The representation model and pipeline can be used programmatically for experiments 4. Graph Representation of or other applications. In the scope of our system, the Reader Comments data is accessed through a highly customizable API by our interactive frontend. The processing pipeline transforms the tabular comment
We enrich our graph representation through text data retrieved by the scrapers into a graph and further mining and graph analysis beyond the comment meta- enriches the information it contains. Each comment may data. These steps of analysis and aggregation come with contain more than one main semantic aspect, such as loss of details, which has to be balanced with the benefits diferent arguments or responses to other comments. of a better overview. Although we refer to the comments Thus, we heuristically assume sentences to be the smallon online news platforms as a discussion or discourse, est “atomic” semantic unit of a comment. Each sentence many statements and arguments are frequently repeated is added as a node in the graph representation of a by diferent users without referring to an already existing discussion. By adding edges between sentences of the comment. Our graph representation helps to identify and same comment, we are able to maintain data provenance visualize these inherent semantic clusters of comments. along with meta-data about the original content. In this
In the most simplified view, the graph representation section, we discuss possible data mining methods to is used to draw particles on a two-dimensional can- enrich the graph representation. vas.Hereby, the comment positions reflect semantic simi- The first step is the discovery of relations between larity and cluster afiliation. To convey more additional sentences and adding edges to represent these relations. information, particles can be drawn as glyphs or vary in Second, we assign class labels and scores to the nodes size or color. The canvas can be enriched by overlays of and edges. This allows us to rank and cluster them based cluster contours, heat maps, and explanatory keyphrases. on these assignments. Finally, the number of nodes and
We embed comment threads originating from multiple edges is reduced by filtering or merging them to provide news platforms in the same space, thus merging topically a comprehensible entry point. Note, that edges of the related discussions from various articles. In this way, graph are only the basis to internally represent reader we provide a global view and increase the diversity of comments and the layout. Edges won’t be directly visible represented opinions following our three key goals. The in the interface to reduce visual clutter. summarizing visualization aims for more engagement, reducing potential bias in discussions by having a broader Edge Discovery. Our approach builds on ideas by group of contributors and novel playful ways of inter- Barker and Gaizauskas [16], who represent arguments in action, such as reacting to clusters of comments. We comments (assertions or viewpoints) in a graph. Given anticipate a larger number of replies in general and this graph, they generate textual summaries of an entire deeper threads, which means more in-depth replies. If discussion. In contrast to their laborious process of a user receives a reply to his or her comment, this reply manually constructing the nodes and edges, we generate is an acknowledgment for the user and demonstrates them automatically and present them in an interactive that the comment is relevant to others. Readers should visualization. To this end, we construct a network be able to easily navigate to comments that are most of sentences as nodes adding edges if their pairwise interesting to them, as reading every single comment semantic similarity is above a certain threshold. This becomes infeasible for popular articles. Our interactive similarity is the cosine similarity of the sentence emvisualization condenses long discussions into groups of bedding vectors calculated with fastText [17]. Syntactic similar comments for more insights. In this way, we edges are added between all sentences that belong to are still able to show all contributed comments, while the same comment and also to its replies. Thereby, users can make an informed decision on which subset structural information of the comments and the discourse of comments to actually read. This overview could also is incorporated. The resulting network is drawn using a give information on diferent viewpoints, such as how force-based layout algorithm. Edges are hidden for the many commentators share a particular point of view. By user, so that the comment landscape only shows clusters retaining data provenance information, users are able to of points representing sentences. Since we include both switch back and forth between the generalized overview semantic and syntactic edges, the layout can provide and the underlying data for more details. Further, the an overview of the key topics of the discussion, while system includes a full-text search and time or lasso prevailing its overall structure.
Filtering and Merging. The class labels generated by the two neural networks are used to put more visual emphasis on the engaging comments than on the toxic comments. Nodes with a small number of edges represent sentences that are only loosely connected. In a simplified view, these nodes are either filtered completely or merged with a neighbored node. Nodes for sentences with almost similar embeddings are grouped together.
Clustering and Classification. On the node level, — instead of only appending a reply to an existing list.2 the TextRank algorithm [15] ranks sentences and assigns Additionally, they can rearrange or filter the nodes and weights to identify key statements, which we assume navigate the canvas by zooming and panning. When to be strongly connected and to form similarity commu- using a lasso to select nodes, comments on the right nities. The clustering progressively removes edges and panel are automatically filtered. By selecting an interval thereby conforms to our idea of reducing the discussion on the time histogram at the bottom, additional filters to its most essential statements for a comprehensible are applied. As stated before, nodes in the graph are overview. Further, a neural network model detects individual sentences of comments. By clicking a node, all toxic comments, such as insults or threats, which are other nodes belonging to the comment are highlighted comments that make other users leave a discussion [18]. and the comment is shown in the right panel. Once a lasso Another neural network model from related work [12] selection is active, users can vote up or down on multiple detects engaging comments, such as questions or factual comments to signal their agreement or disagreement. statements, which are likely to receive many reactions The fill color of the nodes is updated to convey areas by other users. of predominantly positive or negative sentiment. The size of nodes can be determined by multiple factors.
In Figure 2 the TextRank score is used, but the votes or number of replies on the originating platform has similar efects. Sliding a selection window over the time histogram shows how the discussion evolves over time.
For example, it reveals which topics came up early in the course of the discussion.
components: the news outlet selection, the interactive graph, and the detailed comment view (Figure 2). News Outlet Selection. The panel on the left-hand side of the interface allows selecting a set of reader discussions on articles from diferent news outlets. There are presets of news stories that were covered by many platforms but users are free to select (the comments of) any news article that is published on one of the seven platforms currently supported. Users can add an article by simply pasting its URL. Comments on this article are then retrieved by our server and merged with previously selected comments to construct a graph representation. Mckay et al. [19] suggested to build systems that support users in reflecting on their own view by comparing it with diverse views of others. By incorporating comments from many diferent news outlets, we implement this design idea in the context of online discussions.
graph layout of all the comments. Note that the edges of the underlying graph are used only by the force-layout and are not shown for simplicity. This visualization enables users to interact with single keyphrases of longer comments or with multiple comments at once Detailed Comment View. The panel on the righthand side lists the comment texts where the text of the currently selected comment is highlighted. With a search bar, users can quickly find comments that mention keywords they are interested in. At the top of the panel are also parameter controls to adjust the number of nodes and edges displayed. This view is also updated by filters applied to the interactive graph.
Additional Possibilities. In this section, we described features of the interface we thought to be essential for exploring the comment landscape. All these features are implemented in a prototype system. Further features could be added to enable users to analyze the data in more depth. The underlying graph representation of reader comments provides the basis for additional capabilities. As the graph implicitly maintains data provenance, tools for filtering comments based on metadata is possible at all times. Furthermore, the information could also be used to control the shape, color, or size of the visualized nodes. For example, a user might want to color all nodes based on the news outlet the respective comments were extracted from.
concepts and platform requires many active users and a sophisticated experimental setup. Such an evaluation
to one or multiple comments are not transmitted back to the news platforms.
A “These fires are not the result of climate change! They were set on purpose! [...]” Search...
B D C2 “I am not aware of any statistics that the periods of drouts are increasing, otherwise it would be mentioned in IPCC reports. [...]”
G
H
Die Welt Die Tageszeitung Zeit Online is beyond the scope of this paper and deferred to future work. Nevertheless, we conducted a small-scale case study to validate the presented ideas.
The system described in this paper was designed for the purpose of visualizing comments from diferent platforms on a single topic. We therefore use the notion of a news story, which is covered by several news articles on the same emerging news event.
Our initial findings are based on hand-selected news stories of seven diferent German news platforms: faz.net, tagesschau.de, spiegel.de, sz.de, taz.de, welt.de and zeit.de.
Each day between November 2019 and February 2020, we manually selected the most prevalent news stories. For to positive (green). each story, the annotators manually collected respective With this case study we have shown the novel way articles from the previously mentioned news platforms. our ComEx system enables users to interact with reader The resulting dataset comprises 150 news stories and comments. More evaluation is necessary to validate the 1,350 news articles. Only 570 of these articles have user-centered aspects of being more engaging and more publicly available reader comments, which we retrieved in-depth, and providing more insights. programmatically. In total, we retrieved 111,000 comments and the average comment length is 45 tokens. To give an example, one of the most discussed stories in 7. Conclusions and Future Work this dataset contains 4,696 comments and is covered by 4 news platforms. It is about the UN Climate Change To improve the way people exchange ideas online and Conference held in Madrid, 2019. to foster in-depth discussions, we studied the novel
The interaction features of the seven popular German- task of comment exploration for users of online news language news outlets we selected are limited to com- platforms. Previous work on conversation or discourse ment replies, upvotes, and downvotes as well as ranking exploration developed analytics tools for experts. In by time or number of votes or replies (summarized in contrast, we focused on letting comment readers and Table 1). ComEx, on the other hand, could drastically comment writers interact with the exploration tool. To change how users interact with online comments. It this end, we presented ComEx, a comment exploration provides a feature-rich exploration interface for a global system that implements diferent methodologies for the overview of comments from across multiple online interactive analysis and visualization of comments in news platforms. Going through the processing pipeline online discussions. by hand, the annotators printed all comments of two A promising path for future work is to study the exemplary stories and collaboratively assigned semantic impact of novel visualization and exploration methods groups similar to the argument graph described by Barker on online discussions. One exemplary research question and Gaizauskas [16]. Our manual results in general in this context would be how visualizations could help confirmed the graph layout automatically generated by to establish a higher conversion rate of comment readers into comment writers. Potential next steps are to conduct CoFmigEuxr.e 2 shows the interface for four articles on user studies to evaluate our prototype and identify Australian wildfires in 2020 with 413 reader comments. interaction patterns. The presented system is not limited If a user wants to add additional news articles, she to the news comments use case, but can be employed in can add a URL on the left pane (E). The ComEx sys- all kinds of scenarios where user-generated content can tem will then extract comments from the website in be linked to each other. the background, update the comment graph and the Although some examples we looked at in depth visualization in the center pane. In the displayed use showed initially promising results, we see room for imcase, we opted to visualize topical similarity leading to provement and potential for future work in the construcclusters of topically similar comments. The cluster at the tion and filtering of the underlying graph representation. top contains comments (C1) discussing climate change, Our modular architecture for node enrichment and edge while another cluster of comments (C2) on the bottom generation and filtering allows for a simple configuration primarily concerns droughts. The timeline at the bottom of the pipeline. One of the major challenges is to limit indicates the date the comments were published. By the number of generated edges, e.g., by introducing selecting a time-window (F), comments can be filtered. locality-sensitive thresholds for similarity-based edges, Comment outside the selected window are greyed out such as those based on sentence embedding distances. in the visualization and removed from the comment The visualization uses a force-based layout algorithm. pane (H). Users may also filter comments using full-text We experimented with several approaches to incorporate search (B). There are two modes (A) in which the user weighted aggregates of edge weights produced from can interact with the data displayed in the center pane. diferent sources, i.e., for combining cluster assignment, The first mode supports exploration, including zooming embedding similarity, temporal proximity, and reply and panning the visualization. Furthermore, clicking a structure. Finding a robust and ideally self-adjusting node in the visualization highlights the corresponding approach remains a task for future work. Furthermore, comment in the comment pane (H) and vice versa. The we found, that a keyword overlay to briefly describe the second mode supports engagement, by providing a lasso “meaning” of a visual neighborhood could be a useful tool to select several comments at once. Users can then addition to the interface. express their sentiment by voting up or down. We store this information and use color (G) to indicate the average sentiment of all votes from negative (red), neutral (blue), (NAACL), ACL, 2018, pp. 193–199.
[12] J. Risch, R. Krestel, Top comment or flop comment? [1] M. El-Assady, V. Gold, C. Acevedo, C. Collins, predicting and explaining user engagement in D. Keim, Contovi: Multi-party conversation explo- online news discussions, in: Proceedings of the ration using topic-space views, Computer Graphics International Conference on Web and Social Media Forum 35 (2016) 431–440. (ICWSM), AAAI, 2020, pp. 579–589. [2] P. Riehmann, D. Kiesel, M. Kohlhaas, B. Froehlich, [13] Y. Nedumov, A. Babichev, I. Mashonsky, N. Semina, Visualizing a thinker’s life, Transactions on Visual- Scinoon: Exploratory search system for scientific ization and Computer Graphics (TVCG) 25 (2018) groups, in: Proceedings of the Workshop on 1803–1816. Exploratory Search and Interactive Data Analytics [3] S. Sen, A. B. Swoap, Q. Li, B. Boatman, I. Dippenaar, (ESIDA@IUI), 2019.
R. Gold, M. Ngo, S. Pujol, B. Jackson, B. Hecht, Car- [14] E. Palagi, F. Gandon, A. Giboin, R. Troncy, A tograph: Unlocking spatial visualization through survey of definitions and models of exploratory semantic enhancement, in: Proceedings of the search, in: Proceedings of the Workshop on Conference on Intelligent User Interfaces (IUI), Exploratory Search and Interactive Data Analytics ACM, 2017, pp. 179–190. (ESIDA@IUI), 2017, pp. 3–8. [4] J. Peltonen, Z. Lin, K. Järvelin, J. Nummenmaa, Pi- [15] R. Mihalcea, P. Tarau, Textrank: Bringing order hvi: Online forum posting analysis with interactive into text, in: Proceedings of the Conference on hierarchical visualization, in: Proceedings of the Empirical Methods in Natural Language Processing Workshop on Exploratory Search and Interactive (EMNLP, 2004, pp. 404–411.
Data Analytics (ESIDA@IUI), CEUR-WS, 2018. [16] E. Barker, R. J. Gaizauskas, Summarizing multi[5] R. E. Prasojo, M. Kacimi, W. Nutt, Entity and party argumentative conversations in reader comaspect extraction for organizing news comments, ment on news, in: Proceedings of the Annual in: Proceedings of the Conference on Information Meeting of the Association for Computational and Knowledge Management (CIKM), ACM, 2015, Linguistics (ACL), ACL, 2016, pp. 12–20. pp. 233–242. [17] P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, [6] A. Funk, A. Aker, E. Barker, M. L. Paramita, Enriching word vectors with subword information, M. Hepple, R. Gaizauskas, The sensei overview of Transactions of the Association for Computational newspaper readers’ comments, in: Proceedings of Linguistics (TACL) 5 (2017) 135–146. the European Conference on Information Retrieval [18] J. Risch, R. Krestel, Bagging BERT models for (ECIR), Springer, 2017, pp. 758–761. robust aggression identification, in: Proceedings [7] A. Aker, E. Kurtic, A. Balamurali, M. Paramita, of the Workshop on Trolling, Aggression and E. Barker, M. Hepple, R. Gaizauskas, A graph-based Cyberbullying (TRAC@LREC), European Language approach to topic clustering for online comments to Resources Association (ELRA), 2020, pp. 55–61. news, in: Proceedings of the European Conference [19] D. Mckay, S. Makri, M. Gutierrez-Lopez, A. MacFaron Information Retrieval (ECIR), Springer, 2016, pp. lane, S. Missaoui, C. Porlezza, G. Cooper, We are 15–29. the change that we seek: Information interactions [8] Y. Zhang, J. Li, Y. Song, C. Zhang, Encoding conver- during a change of viewpoint, in: Proceedings of sation context for neural keyphrase extraction from the Conference on Human Information Interaction microblog posts, in: Proceedings of the Conference and Retrieval (CHIIR), ACM, 2020, p. 173–182. of the Association for Computational Linguistics (NAACL), ACL, 2018, pp. 1676–1686. [9] C. Harris, Searching for diverse perspectives in news articles: Using an lstm network to classify sentiment, in: Proceedings of the Workshop on Exploratory Search and Interactive Data Analytics (ESIDA@IUI), 2018. [10] K. Kucher, R. M. Martins, C. Paradis, A. Kerren,
Stancevis prime: visual analysis of sentiment and stance in social media texts, Journal of Visualization 23 (2020) 1015–1034. [11] C. Ambroselli, J. Risch, R. Krestel, A. Loos, Prediction for the newsroom: Which articles will get the most comments?, in: Proceedings of the Conference of the Association for Computational Linguistics