Mining Bipolar Argumentation Frameworks from natural language text Oana Cocarascu Francesca Toni Imperial College London Imperial College London oc511@imperial.ac.uk f.toni@imperial.ac.uk ABSTRACT but not vice versa. This choice is well suited in settings such We describe a methodology for mining topic-dependent Bipo- as online reviews, where information is provided incremen- lar Argumentation Frameworks (BAFs) from natural lan- tally, over time, and contributors have full vision of existing guage text. Our focus is on identifying attack and support data. argumentative relations between texts about the same topic, We illustrate our methodology with natural language text treating these texts as arguments when they are argumenta- drawn from a dataset of hotel reviews [17]. In this context, tively related to other texts. We illustrate our methodology the topics are represented by aspects that users mention in on a dataset of hotel reviews and outline some possible ap- the reviews, and BAFs represent how arguments from re- plications using the BAFs resulting from our methodology. views relate to arguments from other reviews as well as to arguments about the quality of the items being reviewed. For example, consider the following two reviews about a hotel (with the second being more recent than the first): 1 INTRODUCTION Argument Mining is a relatively new research area which r1 : Exceeds any expectations - rooms, food, atmosphere were involves, amongst others, the automatic detection in text heads above anywhere. Thanks for making my trip the best of arguments, argument components, and relations between of the best. arguments (see [15] for an overview). Argument Mining is a r2 : The room was not clean. Don’t waste your time or money complex task because of the lack of a clear argumentative here. structure in free natural language text. Argument Mining can be seen as a pipeline, composed of several stages, including: For the topic room, our methodology may give the BAF identifying argumentative sentences, detecting component graphically shown in Figure 1 (where nodes of the graph are boundaries and argument components, and determining rela- arguments in the BAF, edges labelled by + indicate support tions between arguments. and edges labelled by - indicate attack). Here, the (root) In this paper, we focus on identifying argumentative rela- G (stating that the hotel is good) is supported by Groom tions of attack and support between texts, assuming that if (stating that the rooms in this hotel are good), related to one text attacks/supports another, then both may be consid- the topic identified. Then, ↵1 and ↵2 , drawn from reviews r1 ered to be argumentative, irrespectively of their stand-alone and r2 respectively, are about the same topic room. Here, ↵1 argumentativeness. This task, referred to as Relation-based supports Groom and ↵2 attacks ↵1 . Because of the temporal Argument Mining (RbAM) [6], has attracted some atten- dimension underlying our methodology, ↵1 does not attack tion lately, seen as a stand-alone classification task to be ↵2 and ↵2 does not attack Groom . addressed by Machine Learning (ML) techniques [2, 5, 6]. As a classification problem, RbAM can be thought of as deter- G: the hotel is good mining a class amongst attack, support, and neither attack nor support for any given pair of texts, to determine which + type of argumentative relation the first element of the pair Groom : the rooms in this hotel are good is in with the second. Note that RbAM does not rely on or + assume any specific argument model or internal structure of arguments. ↵1 : rooms exceeded any expectations We propose a methodology for mining Bipolar Argumen- tation Frameworks (BAFs) [7] from natural language text, ↵2 : the room was not clean with RbAM at its core. BAFs are a well known kind of ar- gumentation framework in the literature on Argumentation & AI [19, 21]. RbAM is well-suited for mining BAFs since Figure 1: BAF extracted from reviews r1 and r2 . attack and support relations between (abstract) arguments are the main components of BAFs. Our methodology relies In [9], we have shown how mining automatically BAFs upon constructing a topic-dependent BAF from text, using from text, and reviews in particular, can be useful to support topics to identify pairs of chunks of text to be classified using other activities (deception detection in [9]). In the current RbAM, along a temporal dimension whereby more recent paper, we discuss some additional applications that can be chunks of text may either support or attack less recent ones, supported once the BAFs have been extracted from text. 18th Workshop on Computational Models of Natural Argument 65 Floris Bex, Floriana Grasso, Nancy Green (eds) 16th July 2017, London, UK The remainder of this paper is organised as follows. We No one who voted in favour of the tax increase is a desirable present background on BAFs and related work in Section 2. candidate. Therefore, Councilwoman Radcli↵e is not a desir- In Section 3 we outline our methodology for mining BAFs able candidate. from natural language text and illustrate its application to an excerpt from a dataset of reviews about a hotel. We discuss RbAM can be seen as a prerequisite for constructing BAFs. It some applications of BAFs mined from text in Section 4 and has been traditionally treated as a ML classification problem conclude in Section 5. We do not report any experimental with three classes: support, attack, neither support nor attack. results for our methodology in this paper; some experiments Various approaches have been used to determine (attack/support) are described in [9]. relations between arguments, varying from standard ML clas- sifiers [5] to textual entailment [3, 4]. Extracting attack and 2 BACKGROUND AND RELATED support relations was also done on a corpus consisting of WORK tweets [2]. Identifying attack and support relations between an evaluative expression and an argument was addressed in (Abstract) Argumentation Frameworks (AAFs) are pairs con- [12], on a French corpus covering domains such as hotels, sisting of a set of arguments and a binary relation between restaurants, and politics. arguments, representing attacks [11]. Formally, an AAF is any hAR, attacksi where attacks ✓ AR ⇥ AR. Bipolar Argumen- tation Frameworks (BAFs) extend AAFs by considering two 3 A METHODOLOGY FOR MINING independent binary relations between arguments: attack and BAFS support [7]. Formally, a BAF is any hAR, attacks, supportsi RbAM is a difficult task which amounts to identifying the where hAR, attacksi is an AAF and supports ✓ AR ⇥ AR. pieces of text between which there may be an argumentative For example, consider the following three texts: relation as well as the type of relation between these pieces of text. When dealing with large texts, e.g. drawn from online t1 : ‘We should grant politicians immunity from prosecution’ product reviews or online debates, analysing every possible t2 : ‘Giving politicians immunity from prosecution allows them pair of pieces of texts in order to determine relations is simply to focus on performing their duties’ not feasible. t3 : ‘The ability to prosecute politicians is the ultimate protec- We propose a methodology to extract BAFs from natural tion against the abuse of power’ language text where the arguments that form the BAFs are clustered based on the topics extracted from the text being Here t2 supports t1 and t3 attacks t1 . Thus, these three texts analysed. In order to construct BAFs, we use a temporal can be seen as the arguments in the BAF represented as the dimension to decide which texts to compare to determine the following graph (where nodes are arguments, edges labelled - relation as well as to determine the type of relation between indicate attacks and edges labelled + indicate supports): texts (i.e. support, attack, or neither). Thus, we construct + topic-dependent BAFs by determining the relations between t2 t1 t3 arguments that refer to the same topic, following the temporal Argument Mining is an emerging field whose aims include order, in such a way that more recent texts can relate to to identify argumentative sentences, argument components less recent ones, but not vice versa. The rationale behind and argument structures (such as claims and premises), as the topic-oriented approach is that arguments that mention well as to identify relations between arguments (such as sup- di↵erent topics are highly unlikely to be related (i.e. neither port and attack) (see [15] for a recent overview). Argument argument supports nor attacks the other argument). The Mining mostly relies on Natural Language Processing (NLP) rationale behind the reliance on a temporal ordering is that and Machine Learning (ML) techniques. In this paper we it allows to limit the number of relations in BAFs. Our focus on Relation-based Argument Mining (RbAM) [6], a illustration demonstrates that this approach is useful and subtask in Argument Mining which aims to automatically allows to generate BAFs that can be easily understood. identify argumentative relations between texts, of the kinds Our procedure for constructing a BAF from text is as (attack and support) occurring in BAFs. In RbAM, if one follows: text attacks/supports another, then both may be considered (1) split the text into temporally ordered sentences; we to be argumentative, irrespectively of their stand-alone argu- thus assume that each argument extracted from the mentativeness. For example, consider the following sentence: reviews is contained in a sentence, and that each sentence contains one or more potential argument; Councilwoman Radcli↵e voted in favour of the tax increase. (2) identify topics in texts and, for each topic, the sen- tences (potential arguments) related to the topic; Analysed in isolation, this sentence does not seem to be (3) for each topic, for each pair of sentences related argumentative but becomes an argument when read in con- to that topic, determine whether the most recent text: sentence supports, attacks, or neither supports nor attacks the less recent sentence; compare a sentence Councilwoman Radcli↵e voted in favour of the tax increase. with its (temporally) closest less recent sentence first, 2 66 18th Workshop on Computational Models of Natural Argument Floris Bex, Floriana Grasso, Nancy Green (eds) 16th July 2017, London, UK and compare the sentence with less recent sentences a pre-trained tokenizer for English. Sentences containing than its closest ones only if it neither supports nor specific keywords such as but, although, though, otherwise, attacks the closest ones; however, unless,whereas, are split since, in general, the phrases (4) construct the BAF. before and after these separators express di↵erent sentiments In the remainder of this section, we illustrate our method- (e.g. ‘The sta↵ was nice but the room was messy’ results in ology when applied to the reviews in Table 1, which represent two sentences with di↵erent sentiments). an excerpt from a dataset consisting of positive and negative At this step, in preparation for steps 3 and 4, we also de- hotel (deceptive or truthful) reviews [17]. In the illustration, termine the sentiment polarity of sentences. This polarity can we will sometimes refer to the implementation given in [9] of be identified using a lexicon of frequently used adjectives in our proposed methodology. product reviews annotated with scores for sentiment polarity [10]. R1 This hotel is absolutely beautiful. Our room was gorgeous. I As an illustration of this first step, from R1 , we identify do have 1 complaint. The bar is very boring and the restaurant is the following pieces of texts containing potential arguments, not that great, but other than that I loved it. I would definitely with polarity (+ for positive and - for negative sentiment) as stay there again. indicated: R2 Very pleasant front sta↵, large rooms, and free Internet for those that are members of the loyalty program. room was gorgeous (+) R3 Sta↵ were helpful & friendly, room was huge with fantastic the bar is very boring and the restaurant is not that great (-) view of the river. Adjoining Bar/Restaurant, great food. R4 I stayed here last August and I’m truly glad that I will whereas from R10 we identify the following potential ar- never have to stay here again. The website does a great job gument, with polarity as indicated: of creating an illusion. The rooms are so much smaller than it seems on the website. The wireless internet is free but it is located in a hard to find location, the restaurant is uncom- extremely slow. All and all, I would not recommend this hotel to anyone. fortably R5 It is one of the nicest hotels i have stayed at in my life, crowded, the sta↵ is hard to reach (-)1 clean, comfortable and pretty. The rooms were clean and the sta↵ is very caring. We will see next that some potential arguments can be split R6 I have stayed in hotels all over the world, and this is probably further. the worst that I’ve ever experienced. The sta↵ was unaccom- modating, the front desk sta↵ was condescending and not even 3.2 Illustration of Step 2 remotely helpful. The room was not clean. Don’t waste your In building a topic-dependent BAF from a set of reviews, we time or your money here. R7 The sta↵ are polite and well poised. The rooms, hallways first identify ‘topics’ mentioned in the reviews. and facilities were exceptionally clean and tidy. During my Various approaches for identifying topics in text exist, rang- stay, I stopped at their restaurant where I had one of the best ing from associating each noun encountered in texts to a topic, American style meal in a while. Overall, this hotel is a place I to more advanced techniques related to topic modeling such would surely stay at again if given the chance to visit Chicago as Latent Dirichlet Allocation (LDA) [1] and Non-negative for a second time. It is truly exceptional. Matrix Factorization (NMF) [14], able to uncover the under- R8 I loved the location and the amenities o↵ered by this hotel. lying semantic structure of text by identifying topics and the The room was charming with a window seat and a water view. words that belong to topics. Free wireless internet were a plus here. The sta↵ was helpful For our reviews, if we associate the nouns encountered in at and attentive. I would definitely stay here again. least a few reviews to a topic, then we identify the following R9 There was only one person from the sta↵ at the front desk topics: sta↵, room, internet, bar, restaurant, location. The when we arrived, preoccupied with something on their computer so our presence was not acknowledged for several minutes. Then intuition here is that, in the case of online reviews, if a topic is when we got to our room, I found it to be incredibly dusty. controversial/debatable, then it will be mentioned by at least Overall it was a good stay but those two inconveniences made a few users (either to support the argument given initially or us question the amount of money we paid for it. to attack it). R10 The hotel is located in a hard to find location in Chicago, We then identify the sentences/arguments related to these the restaurant is uncomfortably crowded, the sta↵ is hard to topics. In the case of topics being associated to nouns, we reach, overall it was not a pleasant hotel stay. extract the sentences that contain these specific nouns. For Table 1: Reviews for a hotel in Chicago. LDA/NMF, we extract the sentences containing any of the top words associated to the extracted topics. For example, from R1 , we identify the following arguments, with polarity and topics as indicated: 3.1 Illustration of Step 1 1 Note that we use components of argumentative sentences to stand for the full sentences. For example, the first part of the latter potential The first step in constructing BAFs is to split the texts argument stands for “The hotel is located in a hard to find location analysed into sentences. This can be done for example with in Chicago”. 3 18th Workshop on Computational Models of Natural Argument 67 Floris Bex, Floriana Grasso, Nancy Green (eds) 16th July 2017, London, UK Topic Argument Argument room: a1,1 : room was gorgeous (+) id bar : a1,2 : the bar is very boring (-) a2,1 very pleasant front sta↵ restaurant: a1,3 : the restaurant is not that great (-) a3,1 sta↵ were helpful & friendly a5,2 the sta↵ is very caring whereas from R10 we identify: a6,1 the sta↵ was unaccommodating, sta↵ the front desk sta↵ was conde- location: a10,1 : located in a hard to find location (-) scending and not even remotely restaurant: a10,2 : the restaurant is uncomfortably crowded (-) helpful sta↵ : a10, : the sta↵ is hard to reach (-) 2 a7,1 the sta↵ are polite and well poised 3.3 Illustration of Step 3 a8,4 the sta↵ was helpful and atten- tive Determining relations between sentences/arguments in any a9,1 there was only one person from pair can be viewed as a three-class problem, with classi- the sta↵ at the front desk when fication labels {attack, support, neither}. For this step, we we arrived, preoccupied with can use ML classifiers to determine relations between sen- something on their computer so tences/arguments associated to topics identified in the previ- our presence was not acknowl- ous step. In [9] we used Random Forests for classifying the edged for several minutes type of relation between arguments identified at step 2, using a10,3 the sta↵ is hard to reach the arguments’ sentiment polarity as a feature. a1,1 room was gorgeous a2,2 large rooms 3.4 Illustration of Step 4 a3,2 room was huge with fantastic The arguments in the BAF include a single special argument view G (for ‘good’) as well as a special argument Gt per topic t, room a4,1 the rooms are so much smaller as already seen in Section 1. Gt stands for ‘good as far as t than it seems on the website is concerned’, such that each Gt supports G. a5,1 the rooms were clean We use a temporal approach for determining the relations a6,2 the room was not clean between arguments related to topic t drawn from reviews a7,2 the rooms were exceptionally and the special argument Gt . In particular, we assume that clean and tidy a newer argument (with respect to time) can either support, a8,2 the room was charming with a attack, or neither support nor attack a previous argument or window seat and a water view Gt , but not vice versa. Note that the use of this temporal a9,2 our room, I found it to be incred- ‘filter’ is well suited in the context of online reviews, but it ibly dusty. might not be applicable in other settings. If an argument a2,3 free Internet for those that are at , related to topic t, does not support or attack another internet members of the loyalty program argument related to t from the same or some other review, as a4,2 the wireless internet is extremely determined by RbAM at step 3, then at will either support slow or attack Gt , according to its polarity. a8,3 free wireless internet were a plus Figure 2 shows the BAF extracted from the reviews in here Table 1. The extracted arguments occurring as nodes in a1,2 the bar is very boring the BAF are shown in Table 2. Note that not all text in bar a3,3 adjoining Bar, great food the reviews contributes to the arguments in the BAF. For a1,3 the restaurant is not that great example, the first sentence from R1 : “This hotel is absolutely a3,4 adjoining Restaurant, great food beautiful ” does not represent an argument as it mentions restaurant a7,3 at their restaurant where I had ‘hotel’ and not a specific topic related to hotel. one of the best American style As an illustration, consider the first three arguments about meal in a while the topic room: a1,1 , a2,2 , a3,2 . As a1,1 is the first argument a10,2 the restaurant is uncomfortably that mentions the topic, it is connected to Groom according crowded to its polarity (i.e. a1,1 supports Groom ). In this case step a8,1 I loved the location 3 deemed that a2,2 neither attacks nor supports a1,1 (we location a10,1 located in a hard to find location only consider relations from a2,2 to a1,1 as per our temporal Table 2: Arguments extracted from the reviews in approach). Then a2,2 supports Groom as it has a positive Table 1. polarity and there is no other less recent argument to be compared with. Here step 3 also identifies a support relation 2 In our notation, ax,y represents the yth argument from review x . 4 68 18th Workshop on Computational Models of Natural Argument Floris Bex, Floriana Grasso, Nancy Green (eds) 16th July 2017, London, UK G example in Section 3, the strengths for our Gt arguments are: strength(Gsta↵ ) = 0.955078 strength(Groom ) = 0.967773 + + strength(Ginternet ) = 0.6875 strength(Gbar ) = 0.375 + strength(Grestaurant ) = 0.40625 strength(Glocation ) = 0.625 + + + These provide a measure of how good the hotel is, along the various dimensions (topics) considered, according to the avail- Gsta↵ Groom Ginternet Gbar Grestaurant Glocation + + able reviews, and can also be used to compare the hotel with + + + + others. + a2,1 a3,1 a1,1 a2,2 a5,1 a2,3 a1,2 a1,3 a8,1 Further, BAFs could be employed in tasks such as sum- marisation as they provide a structured and concise view of + + + + the aspects (topics) mentioned in text. In our example hotel, a7,1 a5,2 a8,2 a3,2 a6,2 a4,2 a3,3 a3,4 a10,1 from the BAF in Figure 2, we could e.g. hypothesise that + internet used to be good, since the first review mentioning internet was positive (a2,3 ), but has since been unstable, a6,1 a4,1 a7,2 a8,3 a7,3 as the next review mentioning it is negative (a4,2 ) and is followed by a positive review (see a8,3 , which attacks a4,2 ). a8,4 a9,2 a10,2 BAFs can also help in identifying arguments that are widely accepted as well as identifying conflicting viewpoints that arise in debates. Consider the following simple example: a9,1 + room was not too good + + a10,3 the room was not pretty at allthe room was not very clean Figure 2: BAF obtained from the reviews in Table 1. Here the reviews lead to the conclusion that the rooms were not good in the particular hotel under consideration, and the root argument in the graph is widely accepted. between a3,2 and a2,2 . Thus this relation is included in the We leave the exploration of these and additional applica- BAF. tions of BAFs extracted by means of our methodology for Now consider argument a3,2 . While this can be deemed to future work. support both a2,2 and Groom , it only supports the latter in the BAF in Figure 2. Indeed, in our temporal approach we first 5 CONCLUSION check a3,2 against a2,2 . If a relation is found between these We proposed a methodology for mining Bipolar Argumenta- arguments, then we do not check for the relation between tion Frameworks (BAFs) from natural language text, relying a3,2 and Groom as we want a “minimal” BAF, in terms of on Relation-based Argument Mining (RbAM), a standard the number of relations it accommodates. Similarly, for a4,1 , classification problem in NLP, to identify argumentative rela- we check for a relation between this argument and the most tions between sentences, seen as arguments by virtue of being recent one, in this case a3,2 . If step 3 had not identified any in argumentative relations. In particular, our methodology relation between these two arguments, then a4,1 would have uses RbAM to construct BAFs by determining relations be- been checked against a2,2 , the next “related” argument. tween texts that refer to the same topic, along a temporal dimension whereby more recent texts may either support 4 SOME APPLICATIONS or attack less recent ones, but not vice versa. We have il- In [9] we have used the BAFs extracted from reviews to iden- lustrated our methodology on hotel reviews and discussed tify (argumentative) features to be fed to ML classifiers for the usefulness of our approach in application settings such detecting deception e↵ectively. The BAFs provide semantic as online user comments (reviews and debates) where ar- information on top of the syntactic features obtained through guments lack a clear structure or have incomplete/missing standard NLP techniques. justifications. These applications for BAFs mined from text The BAFs obtained from natural language text using our may help extract information and go well beyond the narrow methodology can be used for other purposes too. For exam- classification task underlying standard RbAM. ple, if applied to online settings such as debates and reviews, This paper gives a pilot investigation, by hand, of our various notions of dialectical strength or acceptability of ar- proposed methodology. We have referred, in our illustrations, guments in AAFs and BAFs may be deployed to evaluate the to an implementation of our methodology [9], that also gives outcomes of the debates or reviews, as suggested in [8]. For experimental results. Much future work is needed to explore illustration, using the DF-QuAD method [18], that quantifies other implementations and applicability in the settings we the strength of arguments by aggregating the strength of considered and beyond, supported by experimentation. We their attackers and supporters, in the case of the reviews also plan to test whether the temporal dimension is useful in 5 18th Workshop on Computational Models of Natural Argument 69 Floris Bex, Floriana Grasso, Nancy Green (eds) 16th July 2017, London, UK other settings, di↵erent from online reviews. We have focused [18] Antonio Rago, Francesca Toni, Marco Aurisicchio, and Pietro Ba- on extracting BAFs from text. Other works extract di↵erent roni. 2016. Discontinuity-Free Decision Support with Quantitative Argumentation Debates. In Principles of Knowledge Representa- types of argument graphs (e.g. [20]), for other application tion and Reasoning: Proceedings of the Fifteenth International areas (e.g parliamentary debates [20]). We plan to test and/or Conference, KR. AAAI Press, 63–73. [19] Iyad Rahwan and Guillermo R. Simari. 2009. Argumentation in adapt our approach for this and other settings. Finally, future Artificial Intelligence (1st ed.). Springer Publishing Company, work also includes experimenting whether first determining Incorporated. arguments based on their argumentative structure, e.g. as in [20] Zaher Salah, Frans Coenen, and Davide Grossi. 2013. Extracting Debate Graphs from Parliamentary Transcripts: A Study Directed [13, 16], may be useful to single out chunks of text to be fed at UK House of Commons Debates. In Proceedings of the Four- into RbAM. teenth International Conference on Artificial Intelligence and Law. 121–130. [21] Frans H. van Eemeren, Bart Garssen, Erik C. W. Krabbe, A. Fran- REFERENCES cisca Snoeck Henkemans, Bart Verheij, and Jean H. M. Wagemans. [1] David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent 2014. Argumentation and Artificial Intelligence. In Handbook of Dirichlet Allocation. Journal of Machine Learning Research 3 Argumentation Theory. Springer Netherlands, 615–675. (March 2003), 993–1022. [2] Tom Bosc, Elena Cabrio, and Serena Villata. 2016. Tweeties Squabbling: Positive and Negative Results in Applying Argument Mining on Social Media. In Computational Models of Argument COMMA. 21–32. [3] Elena Cabrio and Serena Villata. 2012. Natural Language Ar- guments: A Combined Approach. In ECAI, Vol. 242. IOS Press, 205–210. [4] Elena Cabrio and Serena Villata. 2013. Detecting Bipolar Seman- tic Relations among Natural Language Arguments with Textual Entailment: a Study. In Joint Symposium on Semantic Process- ing (JSSP). [5] Lucas Carstens. 2016. Using Argumentation to improve clas- sification in Natural Language problems. Ph.D. Dissertation. Imperial College London. [6] Lucas Carstens and Francesca Toni. 2015. Towards relation based Argumentation Mining. In Proceedings of the 2nd Workshop on Argumentation Mining. Association for Computational Linguis- tics, 29–34. [7] Claudette Cayrol and Marie-Christine Lagasquie-Schiex. 2005. On the Acceptability of Arguments in Bipolar Argumentation Frameworks. In Proceedings of the 8th European conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty. Springer Berlin Heidelberg, 378–389. [8] Federico Cerutti, Alexis Palmer, Ariel Rosenfeld, Jan Snajder, and Francesca Toni. 2016. A Pilot Study in Using Argumentation Frameworks for Online Debates. In Proceedings of the First International Workshop on Systems and Algorithms for Formal Argumentation (SAFA), Vol. 1672. CEUR-WS.org, 63–74. [9] Oana Cocarascu and Francesca Toni. 2016. Detecting de- ceptive reviews using Argumentation. In Proceedings of the 1st International Workshop on AI for Privacy and Security, PrAISe@ECAI. ACM, 9:1–9:8. [10] Tom De Smedt and Walter Daelemans. 2012. Pattern for Python. Journal of Machine Learning Research 13 (2012), 2031–2035. [11] Phan Minh Dung. 1995. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intelligence 77, 2 (1995), 321 – 357. [12] Maria De La Paz Garcia Villalba and Patrick Saint-Dizier. 2013. Opinion Analysis and Argumentation: Identifying and Character- izing the Glue between Evaluative Expressions and Arguments. In Computational Models of Natural Argumentation (CMNA). Proceedings of CMNA13. [13] Ivan Habernal and Iryna Gurevych. 2017. Argumentation Mining in User-Generated Web Discourse. Computational Linguistics 43, 1 (2017), 125–179. [14] Daniel D. Lee and H. Sebastian Seung. 1999. Learning the parts of objects by nonnegative matrix factorization. Nature 401 (1999), 788–791. [15] Marco Lippi and Paolo Torroni. 2016. Argumentation Mining: State of the Art and Emerging Trends. ACM Trans. Internet Techn. 16, 2 (2016), 10. [16] Marco Lippi and Paolo Torroni. 2016. MARGOT: A web server for argumentation mining. Expert Systems with Applications 65 (2016), 292 – 303. [17] Myle Ott, Claire Cardie, and Je↵rey T. Hancock. 2013. Negative Deceptive Opinion Spam. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. Association for Computational Linguistics. 6 70 18th Workshop on Computational Models of Natural Argument Floris Bex, Floriana Grasso, Nancy Green (eds) 16th July 2017, London, UK