Fact-Checking via Structured Discussions in Virtual Communities Ricky J. Sethi Fitchburg State University rickys@sethi.org ABSTRACT social argumentation via structured discussions can build We have recently developed a prototype using a social argu- essential critical thinking, evidence-based reasoning skills. mentation framework to allow virtual communities to check We thus use structured discussions to guide users of our alternative facts. We use a graph-theoretic representation of virtual community in critically analyzing proposed alterna- an argument and also utilize semantic web and linked data tive facts. We utilize fundamental argumentation principles principles in creating an argumentation graph. The creation in a graph-theoretic framework that also incorporates se- of the argumentation graph is crowdsourced and mediated mantic web and linked open data principles [1, 4]. This by expert moderators in a virtual community. We also dis- approach requires us to not only critically examine the pro- cuss some research challenges and future applications of our posed alternative facts but also design a virtual community framework. that allows the social construction of these arguments by large groups so that many people handle small pieces that assemble into a whole. Keywords Fact Checking; Social Argumentation; Virtual Community; 2. CONSTRUCTING SOCIAL ARGUMENTS 1. BACKGROUND Argumentation has been shown to be a natural, substan- tiated approach for analyzing the veracity and reliability of So-called ”alternative facts” are often shared on social net- assertions and claims [6, 3]. In considering how to assess works and spread like wildfire across all sorts of social me- critical thinking, [3] asserts the need to identify conclusions, dia. Some people claim certain facts are true while others reasons, and assumptions as well as judging the quality of call them ”alternative facts” to indicate they are false. Who arguments and developing positions on an issue. is right when considering these potential ”alternative facts”? Argumentation is a formal way of working through ideas Fake news, imbibed with alternative facts, is an age old or hypotheses. An argument, in this formal sense, is a issue not only in politics but communication in general. In structure composed of Stances, Claims, and Evidence. In fact, many social scientists [12, 8] have long believed the our argumentation framework, a Stance is the final conclu- ”public was a malleable, unthinking entity that could be sion composed of Claims and Evidence, and their associ- manipulated by whomever possessed the necessary tools and ated Sources. Stances are fundamental stands on a topic expertise in the field of public relations” and the manipula- and can be mutually exclusive, should have cohesive sub- tion of public opinion was considered fundamental to the structures, and are composed of atomic argumentation com- profession of public relations. ponents (Claims, Evidence, and Sources). A Claim can be Just as with propaganda and misinformation, these so- directly supported by a Source or have multiple Evidence called ”alternative facts” have to be examined critically us- components, each supported by its own Source. Multiple ing evidence-based reasoning in the age of the World Wide Sources can support multiple Evidence nodes. Our method- Web. Collaborative interaction is seen as one of the keys for ology also incorporates Ratings for each Source and user in developing critical thinking and evidence-based reasoning in the system. Different trust, authority, and other attribute online forums [5, 13]. Some [2] also identify the need to dimensions are amalgamated and weighted in a Summary provide argumentation structures to create deeper personal- Rating. ized knowledge and go beyond a simpler social construction We create an Argumentation Graph, GA = (V, E, f ), com- of collective knowledge in collaborative online forums using posed of a set of vertices, V , edges, E, and a function, f , computer supported argumentation [7, 11]. As shown in [9], which maps each element of E to an unordered pair of ver- tices in V . Each fundamental Claim, Evidence, or Source in an argument thus constitutes an atomic argumentation component, va , and is embedded as a vertex in the graph such that va ∈ V . The vertices contain not just the com- ponent’s semantic content, but also the ratings, authority, trust, and other attribute dimensions of each atomic argu- mentation component. The edges e ∈ E contain weights ACM Hypertext Sideways ’17 Prague, Czech Republic along the various dimensions of trust and authority as well Copyright held by the author(s). as pro/con positions, while the function f maps how they’re connected. Depending on the context of the argument, this native facts? Is our approach an optimal way to do so? How graph can be undirected or directed, where the temporal can we best incentivise people to contribute, especially the component gives the direction to the directed graph. Responders who actually construct the argument and pro- In terms of a graph, we therefore see the set of vertices vide Evidence and Sources. Other questions surround algo- V as the set of Claims, Evidence, and Sources; the set of rithms to optimize expert-finding, Top-down vs bottom-up edges E as a set of links that may connect any two vertices. construction of these structured discussions, and community Each subgraph or path traversal that can be obtained from question-answering techniques which we can incorporate to a graph results in a Stance. There are two ways to repre- deal with newly submitted questions. Finally, we will need sent the stances: one way is by making the Stance another to finalize approaches for certifying Responders and Moder- node in GA that is added by the moderators in a top-down ators and ensure that he semantic web components which manner. The other is to designate each sub-graph as a dif- constitute the elements of these arguments are searchable ferent Stance. Once he GA is formed, we can form sub- on the Web. graphs which represent the different stances we can infer from the argumentation graph where each sub-graph would 4. REFERENCES be a separate Stance. Our approach supports both ways of [1] C. Bizer, T. Heath, and T. Berners-Lee. Linked determining the various stances (what we call top-down vs data-the story so far. International journal on bottom-up). Semantic Web and Information Systems, 5(3):1–22, 2.1 Virtual Community for Crowdsourcing 2009. This Argumentation Graph is one component of our three- [2] D. Chen and D. Hung. Personalised knowledge pronged system which also includes the User Interface and representations: the missing half of online discussions. the Social Community Network that actually creates and British Journal of Educational Technology, supports the overall argument. We organize the community 33(3):279–290, 2002. and system to work together in helping critically analyze al- [3] R. H. Ennis. Critical thinking assessment, 1993. ternative facts. Members of this virtual community can take [4] T. Heath and C. Bizer. Linked data: Evolving the Web three major roles: Users: the information seekers who ask into a global data space (1st edition), volume 1. 2011. questions and examine the resulting Stances; Responders: [5] H. Jeong and M. T. H. Chi. Knowledge convergence Users who have some degree of expertise or background to and collaborative learning. Instructional Science, add Claim, Evidence, and Source nodes; Moderators: con- 35(4):287–315, nov 2006. tributors that guide the question and answer flow, matching [6] R. Johnson. The Rise of Informal Logic. Windsor Responders to new questions, evaluating answers for quality Studies in Argumentation, 1996. assurance, etc. These roles are dynamic as they may evolve [7] C. Reed, D. Walton, and F. Macagno. Argument over time, and may be multi-faceted with different functions diagramming in logic, law and artificial intelligence. and capabilities. The Knowledge Engineering Review, 22(01):87–109, 2007. 3. FUTURE WORK AND CHALLENGES [8] M. B. Sethi. Information, education, and This general framework can be adopted for specific appli- indoctrination: The federation of american scientists cations or communities, whose organization can vary widely. and public communication strategies in the atomic For instance, different sites may certify Responders in differ- age. Historical Studies in the Natural Sciences, ent ways: some might require academic qualifications whereas 42(1):1–29, 2012. others would allow the Users to also be the expert Respon- [9] R. J. Sethi and Y. Gil. A Social Collaboration ders. In fact, different systems may have very different or- Argumentation System for Generating Multi-Faceted ganizations, with some sites not having any Moderators at Answers in Question & Answer Communities. In all while others employing the Users to also help moderate CMNA at AAAI Conference on Artificial Intelligence the site. (AAAI), 2011. These roles can thus be generalized for discussions amongst [10] R. J. Sethi, L. A. Rossi, and Y. Gil. Measures of domain experts as well as for formal classroom settings using Threaded Discussion Properties. In Intelligent Support threaded discussions between students and teachers. Our for Learning in Groups at International Conference on framework can be applied to everything from analyzing al- Intelligent Tutoring Systems (ITS), 2012. ternative facts to structuring discussions in online courses. [11] S. Shum and N. Hammond. Argumentation-based We have also developed an initial set of metrics to quantify design rationale: what use at what cost. International the structure of these threaded discussions by measuring the Journal of Human-Computer Studies, 40(4):603–652, redundancy of posts, the compactness of topics, and the de- 1994. gree of hierarchy in sub-threads [10] which we will incorpo- [12] M. J. Sproule. Propaganda and democracy : the rate here. American experience of media and mass persuasion. Existing social networking sites like Facebook could lever- Cambridge University Press, Cambridge, U.K. New age their current userbase and use our framework to create York, NY, 1997. a Social Community Network for checking alternative facts [13] Q. Wang and H. L. Woo. Investigating students’ within Facebook itself. Even new platforms like the recently critical thinking in weblogs: an exploratory study in a announced news platform WikiTribune could be represented Singapore secondary school. Asia Pacific Education in our framework. Review, 11(4):541–551, may 2010. More work needs to be done, though, as questions remain: is crowdsourcing fact-checking the best way to verify alter-