<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Large scale agreements via microdebates?</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Simone Gabbriellini</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Paolo Torroni</string-name>
          <email>paolo.torronig@unibo.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>DEIS - University of Bologna V.le Risorgimento</institution>
          ,
          <addr-line>2, 40136, Bologna -</addr-line>
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Argumentative debates are a powerful tool for reaching agreements in open environments. However, in large scale settings, such as social networks and massive multi-agent systems, making sense of ongoing debates may be a compelling task, and debates risk to lose their e ectiveness. We thus propose \microdebates" to help organizing and confronting users' opinions in an automated way.</p>
      </abstract>
      <kwd-group>
        <kwd>abstract argumentation</kwd>
        <kwd>negotiation in online debate</kwd>
        <kwd>social networks</kwd>
        <kwd>agreement facilitation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1 Introduction</title>
      <p>In the last decade, Web 2.0 platforms have rapidly become a mass phenomenon
whereby billions of individuals consume and share resources. In such a setting,
people became accustomed to arguing online in long-lasting debates, mainly in
the form of comments in social network platform, such as FaceBook1 and
Twitter,2 but also in the form of structured debates in debate-friendly tools. Among
the latter we mention DebateGraph,3 a powerful visualization tool; DBee,4 a
global debating network which features scoring and ranking with both positive
or negative values; Debate.org,5 a social network platform where users can start
debate and comment with pro/cons rating against the main argument in the
debate; and Deliberatorium,6 a community-moderated system where comments
need a moderator approval to be certi ed and visible by a larger community of
commenters.</p>
      <p>
        Indeed, Mercier and Sperber's argumentative theory of reasoning [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] tells
us that people are good at reasoning when they communicate through an
argumentative context. Arguments are used by communicants to convince other
communicants, especially in absence of trust. When debating about an issues
in these online settings, we thus expect that users will not only publish their
opinion (like in a review setting), but also try to convince others by producing
arguments and rebut (attack) each others arguments.
      </p>
      <p>
        Thus, argumentative debate seems to be a very promising tool for reaching
agreement, with particularly interesting applications in a number of settings,
including e-participation an policy-making. Indeed, a number of di erent
platforms are being developed within EU-funded projects such as ePolicy,7 whose
aims include deriving social impacts through opinion mining on e-participation
data extracted from the web; IMPACT,8 which is developing an innovative
argumentation toolbox for supporting open, inclusive and transparent deliberations
about public policy; and WEGOV,9 aiming to provide a toolset for exploiting
existing social networking sites to engage citizens in two-way dialogs as part of
governance and policy-making processes. The idea is that Web 2.0 platforms may
overcome the limitations of traditional opinion gathering methods such as
questionnaires and polls, by allowing for online debates between informed citizens,
who can come up with new ideas and perspectives, as opposed to expressing
preferences upon some predetermined options, and all in a bottom-up fashion
[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. However, the \freedom of expression" provided by online debates comes at
a cost.
      </p>
      <p>In particular, when we think of settings involving multitudes of interacting
parties, such as social networks or large scale multi-agent systems, it becomes
very expensive for by-standers and external observers to make sense of opinions
emerging from online debates. An alternative approach could be to restrict
oneself to getting a feeling of the general sentiment of an ongoing discussion, without
necessarily having to really understand what is being said an why individuals
make such and such claim and express such and such opinion.</p>
      <p>
        State of the art opinion mining/sentiment analysis techniques and tools look
at sentiment orientation of opinions in terms of values in a positive/negative
scale, typically by looking at corpora that include a certain number of sentences
(e.g., online reviews about some product) [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ][
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Such an approach can be very
e ective especially if the domain is well de ned (e.g., a product, or a service).
In domains such as customer reviews [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] where the concepts involved can be
de ned in terms of specialized ontologies, and the jargon is pretty well de ned
and narrow, the classi cation accuracy of existing sentiment analysis algorithms
is quite high. However, this is not the case in other domains, such as political
debate [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Importantly, sentiment analysis does not explicitly tell why certain
opinions are in place and how they relate to other opinions.
      </p>
      <p>
        Our work goes in the perspective of encouraging free, unconstrained online
debate. In said policy-making context, this could be a tool in the hands of the
citizens, who can use it to voice their opinions, and convey them to the
policymakers. To achieve this vision, we need to provide the policy-makers with tools
to automatically make sense of possibly very lengthy online debates. Such tools
7 http://www.epolicy-project.eu
8 http://www.policy-impact.eu
9 http://www.wegov-project.eu
should not only show the general sentiment around a speci c topic, which is
the approach of current sentiment analysis tools. Instead, they should also be
able to identify speci c opinions, and the relations among them. Such relations
could be positive (support) or negative (counter). We identify computational
argumentation, and in particular abstract argumentation [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], as the conceptual
and computational framework to model opinions and reason from them
automatically.
      </p>
      <p>
        In computational abstract argumentation, as de ned by Dung [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], an
argumentation framework is de ned as a pair hX; Ai, where X is a set of atomic
arguments and A is a binary attacks relation over arguments, A X X,
with hx; yi 2 A interpreted as \argument x attacks argument y." Collections of
\justi ed" arguments can be described by various extension-based semantics [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>Current online debating tools, such as those we cited above, build on and
extend the traditional forum-like structure, where users can reply or quote other
users, by introducing debate-oriented concepts. They are not very di erent from
a standard discussion forum with reputation, moderators and recommendation
features. Moreover, they require the user to comply and adapt to the abstractions
they are built around, and not vice-versa.</p>
      <p>
        On the contrary, mainstream Web 2.0 social networking environments, such
as Twitter, are very successful in achieving user engagement, by blurring the
boundaries between ludic and serious [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Our proposal is thus to develop an
application based on a Twitter dialect that allows users to discuss about topics,
aided (in the back-end) by computational argumentation.
      </p>
      <p>
        People use Twitter to talk about their daily activities and to seek or share
information [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] by broadcasting brief textual messages (tweets) to people who
\follow" their activity [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], in a micro-blogging fashion. Micro-blogging is a new
form of communication whereby users can describe their current status in short
posts distributed by instant messages, mobile phones, email or the Web [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. We
therefore introduce the concept of microdebates.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Microdebates</title>
      <p>Microdebates are inspired by Twitter's microblogging character. A microdebate
is a stream of tweets where users annotate their messages by using some special
tags. Twitter posts contain terms called hashtags, i.e. a # symbol followed by
a text string, representing the stream of news the tweet belongs to. There may
be more than one hashtag per post (in case the same post is related to multiple
streams).</p>
      <p>
        Users on Twitter started the phenomenon of adding tags to their messages
sometime around February 2008 [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Twitter tagging behavior is distinct from
those in other Web 2.0 systems, because users are less likely to index messages
for later retrieval [14], and this is re ected by the fact that tagging patterns in
Twitter have a conversational, rather than organizational, nature [15].
      </p>
      <p>In line with Twitter users' tagging behavior, we propose a Twitter dialect
consisting of a custom set of tags to be used to annotate tweets in microdebates:
{ a hashtag that will identify the discussion (e.g., #debateName): as customary,
this ensures that the tweet will appear in the right stream (microdebate);
{ one or more annotation(s) using the $/!$ tags, where
$opinionName speci es the opinion this tweet supports, while
!$opinionName speci es the opinion this tweet counters.</p>
      <p>The syntax for a microdebate is thus as follows:</p>
      <p>h microdebate i ::= h content element i+
h content element i ::= h hashtag i h debate item i+
h debate item i ::= h free text comment i
j $h opinionName i
j !$h opinionName i</p>
      <p>Notation h . . . i+ indicates that multiple occurrences of the element in angle
brackets are allowed. A free text comment is any free text not containing the
special characters #/$. An opinionName is a tag given to a certain opinion; it should
be formatted according to Twitter's tag syntax (alphanumeric strings with no
spaces). The order of content elements in a debate, and of debate item inside
a content element, is immaterial. An example of a hypothetical microdebate is
shown in Figure 1.</p>
      <p>A microdebate is thus a set of elements of content (such as tweets), each
containing a contribution to a debate, such as an opinion, and may contain explicit
references to other elements of content. Each element of content in a
microdebate may use some combinations of characters (similar to hashtags) expressing
positive or negative relations with other content elements. In this way, all that is
asked of the user is to use certain combinations of characters in order to put their
opinion in the context of other opinions. In exchange, users will receive a help
in making sense of a (possibly lengthy) debate: microdebates can be processed
by automatic reasoners, such as argumentation-based reasoning tools [16] and
the output can be visualized graphically as clusters of coherent opinions, where
di erent cluster may attack each other. This could foster awareness of di erent
opinions on a topic and encourage arguers to reach an agreement.</p>
      <p>This is how microdebates work:
1. content elements are tweets with a suitable hashtag, used to identify the
microdebates users are contributing to. (Twitter then displays such tweet in
the public stream associated with such a hashtag);
2. users annotate their tweets using $/!$ tags. When a user A speci es $opinion1,
it means that his comment supports opinion1, which can be an opinion
expressed by the user himself in the comment, or by another users B. In that
case opinion1 will be seen as based on two comments, A's and B's
respectively. The opinion name is abstract, and does not need to be a summary of
the user's opinion;
3. users can attack (counter) opinions using the !$ tag, e.g., by adding the
!$opinion2 item in his tweet. This negation states that the tweet is a
comment, which supports a certain opinion, and at the same time attacks opinion2.</p>
      <p>This enables establishing relations amongst opinions;
4. if a user adds a tweet with a new $ tag, the user is in fact introducing a new
opinion in the microdebate;
5. reply and re-tweets are handled like new tweets, thus personal replies are
irrelevant to the debate (unless they contain $/!$ tags that are meaningful
for this debate).
3</p>
    </sec>
    <sec id="sec-3">
      <title>Microdebates at work</title>
      <p>We implemented a rst prototype of the system as an agent-based model in
NetLogo [17]. In this model, each agent represents an argument used in the
microdebate. Attacks between arguments are represented by directed links from
an agent to another one. We used the Twitter API to retrieve tweets from
Twitter, and the Netlogo API to bundle our system into an extension with a
basic parser (called microdebate), that enables NetLogo to visualize and analyze
the resulting argumentation framework.</p>
      <p>As a rst step, we extract and parse the stream of tweets in a selected
microdebate, so that we have:
{ for each new $opinionN ame tag, a new argument is created;
{ for each new !$opinionN ame tag, a new attack link is created against the
named argument</p>
      <p>To retrieve the microdebate, it su ces to enter a debate identi er, in the
form $debateN ame, in the GUI's debate text box (see Figure 2). In our
example, the debate identi er is $energyalt. Of course, there is a di erence between
an opinion and an argument, the former being a claim without evidence, the
latter being a claim with evidence (supported to convince others that the claim
is supported). At the same time, not all the comments expressed by users can
turn out to be \well-formed" arguments. Nevertheless, at this stage, we turn
every $opinionN ame into an argument belonging to a preliminary argumentative
framework that we de ne naive.</p>
      <p>In order to improve our framework in this respect, we store inside each
argument all the free text comments that refers to $opinionN ame in the
microdebate. We then propose argument classi cation as a way to verify if each claim is
a well-formed argument or not (see Figure 3):
{ if, based on the comments it contains, the claim proves to be indeed a
wellformed argument, we keep it in the argumentative framework;
{ otherwise, if based in its comments, the claim proves not to be a well-formed
argument, we exclude it from the argumentative framework.</p>
      <p>This method allows to obtain a polished up argumentation framework, where
all (and only) well-formed arguments are retained. Being this an initial
prototype, such processing is currently made by hand. In the concluding section of
this paper we elaborate on how we plan to improve this stage.</p>
      <p>Once we have only arguments and attacks among arguments, we can compute
semantic extensions on the argumentative framework.</p>
      <p>Our prototype can compute extensions based on a variety of semantics,
including admissible, complete, grounded, ideal, preferred, stable, semistable, and
stage semantics.</p>
      <p>
        In Figure 4 the complete semantic extension has been calculated, that states:
a set of arguments is a semantic extension i the set include all the
arguments that it defends. As we can see, the two arguments $sugarmills and
$recyclethewaste are the winners over $windmills, demonstrating that the stream
of tweets that compose the microdebate #energyalt can be summarized in a very
compact and e cient way.
The purpose of our proposal is to help reaching an agreement in a debate by
formalizing and rationalizing a debate. Recent ndings in cognitive science [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]
suggest that people are good at arguing, actually that the main function of
reasoning is argumentative. However, when big numbers are in play, it may
be di cult for by-standers and potential contributors to make sense of online
discussions. By microdebates, we aim to help people understand how a topic
is being discussed, what positions (arguments) are involved in the debate, and
what are the relations of attacks between such arguments. Ultimately, we aim to
help people argue in a better way, defend their reasons and learn how to rebut
each other's attacks.
      </p>
      <p>
        The \microdebates" we propose follow the bottom-up argumentation
philosophy introduced in [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Users contribute to a debate by sending out annotated
comments, and as a result, arguments arise bottom-up. In particular, it is not
necessary that the same user de nes a well-formed argument, because many
users can contribute, tweet by tweet, to support the same argument, by adding
elements that turn a claim into a well-formed argument, or by nding rebuttals
and counter-attacks to the arguments in place.
      </p>
      <p>All this e ort should help to produce a more relevant discussion, in a very
user-friendly way: users can annotate their messages in an everyday-life style,
and they do not have to conform to (rigid) rules of another debate-oriented
interface.</p>
      <p>The computational underpinning of our proposal is abstract argumentation,
and is orthogonal to the choice of an extension-based semantics, by design.
Different semantics may suit to di erent applications in various ways.</p>
      <p>Our tool is partially implemented. Two NetLogo extensions are already
implemented: microdebate, for the processing of tweets, and and arguments, for
computing extensions. We are still at an early stage in argument classi cation,
whose purpose is to lter arguments and keep well-formed ones only. For that,
we plan to use third-party semantic tagging tools, such as COGITO.10 Our idea
is to de ne what a \well-formed argument" is by way of COGITO rules, and
delegate to a COGITO module a fully automated argument ltering process.</p>
      <p>We also plann to extensively test our method with case-studies, in order to
understand the e ectiveness of this approach in a real-world setting. In
particular, we are designing tests with debates concerning renewable energy sources,
environment and sustainability, in the context of the above-mentioned ePolicy
EU project.</p>
      <p>The research presented here is high-risk, because many innovations are
required all together for this to succeed. For instance, using our syntax,Twitter
users may develop habits that could be di erent from what we expect, leading
to unforeseen system behavior. The parser will probably need to get adjusted
once some data from a case-study has been retrieved.</p>
      <p>Moreover, since our method (and bottom-up argumentation in general) needs
active engagement from users, we could end up with poor data to analyze if
our users will not get truly involved in the process. However, we hope that
this factor may be mitigated by a unique feature of microdebates: they do not
require a dedicated platform. Users do not need to learn and get accostumed to
new interfaces, because microdebates are only based on tweets (as opposed to
graphic items, such as bubbles and links).</p>
      <p>Having said that, we believe that the strengths and potential of our approach
overcome its limitations. First, microdebates allows deep analysis of arguers
position in a debate, an important step toward the reaching of an agreement
between arguers. Furthermore, by-standers may be encouraged to participate,
since they have a clear visualization of what is happening in the debate - and
what position (arguments set) is going to dominate or get defeated.</p>
      <p>Second, there is no need to manually analyze documents, because posts are
annotated by users. This form of crowdsourcing reduces the amount of quali ed
labor needed. An important bottle-neck is argument classi cation, but we hope
to be able to set up automated procedures for it.</p>
      <p>Third, the microdebate approach develops a technology that may be useful
in many other domains, because it is based on a multi-disciplinary approach
that well suits the needs of diverse domains where debates are allowed, such
as policy-making, Moreover, such technology, initially developed for
human-tohuman interaction, may as well be exported to software agents. We can think
of agents communicating with one another in a tweet-like fashion, and new
algorithms could be developed to automatically reach agreements between agents
on a variety of domain. This may open a promising strand of research.</p>
      <p>Fourth, our approach exploits the so-called wisdom of the crowds (as in
bottom-up argumentation): arguments arise bottom-up from the debate and it is
not necessary for a single user to express the argument entirely, because other
users can contribute to the same argument. Finally, it has an open approach
that allows all users to visualize dynamically the outcome of the analysis.
10 http://www.expertsystem.net/products-technology/cogito-semantic-tagger</p>
    </sec>
    <sec id="sec-4">
      <title>Acknowledgments</title>
      <p>We thank the anonymous reviewers for their useful and encouraging feedback.</p>
      <p>This work was partially supported by the ePolicy EU project
FP7-ICT-20117, grant agreement 288147. Possible inaccuracies of information are under the
responsibility of the project team. The text re ects solely the views of its authors.
The European Commission is not liable for any use that may be made of the
information contained in this paper.
14. Romero, D.M., Meeder, B., Kleinberg, J.: Di erences in the mechanics of
information di usion across topics: idioms, political hashtags, and complex contagion on
twitter. In: Proceedings of the 20th international conference on World wide web.</p>
      <p>WWW '11, New York, NY, USA, ACM (2011) 695{704
15. Huang, J., Thornton, K.M., Efthimiadis, E.N.: Conversational tagging in twitter.</p>
      <p>In: Proceedings of the 21st ACM conference on Hypertext and hypermedia. HT
'10, New York, NY, USA, ACM (2010) 173{178
16. Egly, U., Gaggl, S., Woltran, S.: ASPARTIX: Implementing argumentation
frameworks using answer-set programming. In Garcia de la Banda, M., Pontelli, E., eds.:
ICLP: Proceedings of the 24th International Conference on Logic Programming.</p>
      <p>Volume 5366 of Lecture Notes in Computer Science. Springer (2008) 734{738
17. Wilensky, U.: Netlogo. Center for Connected Learning and Computer-Based
Modeling, Northwestern University. Evanston, IL. (1999)</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Mercier</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sperber</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Why do humans reason? arguments for an argumentative theory</article-title>
          .
          <source>Behavioral and Brain Sciences</source>
          <volume>34</volume>
          (
          <issue>02</issue>
          ) (
          <year>2011</year>
          )
          <volume>57</volume>
          {
          <fpage>74</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Toni</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Torroni</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Bottom-up argumentation</article-title>
          . In Modgil, S.,
          <string-name>
            <surname>Oren</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Toni</surname>
          </string-name>
          , F., eds.
          <source>: TAFA 2011: Revised Selected Papers from the First International Workshop on Theory and Applications of Formal Argumentation</source>
          . Volume
          <volume>7132</volume>
          of Lecture Notes in Computer Science., Springer (
          <year>2012</year>
          )
          <volume>249</volume>
          {
          <fpage>262</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Pang</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Opinion mining and sentiment analysis</article-title>
          .
          <source>Found. Trends Inf. Retr</source>
          .
          <volume>2</volume>
          (
          <issue>1</issue>
          -2) (
          <year>January 2008</year>
          )
          <volume>1</volume>
          {
          <fpage>135</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data (DataCentric Systems and</article-title>
          Applications). Springer-Verlag New York, Inc., Secaucus, NJ, USA (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>S.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hovy</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          :
          <article-title>Automatic identi cation of pro and con reasons in online reviews</article-title>
          .
          <source>In: Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions. (July</source>
          <year>2006</year>
          )
          <volume>483</volume>
          {
          <fpage>490</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Yu</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kaufmann</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Diermeier</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Exploring the characteristics of opinion expressions for political opinion classi cation</article-title>
          .
          <source>In: Proceedings of the 2008 international conference on Digital government research. dg.o '08</source>
          , Digital Government Society of North America (
          <year>2008</year>
          )
          <volume>82</volume>
          {
          <fpage>91</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Dung</surname>
            ,
            <given-names>P.M.</given-names>
          </string-name>
          :
          <article-title>On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games</article-title>
          .
          <source>Arti cial Intelligence</source>
          <volume>77</volume>
          (
          <issue>2</issue>
          ) (
          <year>1995</year>
          )
          <volume>321</volume>
          {
          <fpage>357</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Baroni</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Giacomin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Semantics of abstract argument systems</article-title>
          . In Simari, G.,
          <string-name>
            <surname>Rahwan</surname>
          </string-name>
          , I., eds.
          <source>: Argumentation in Arti cial Intelligence</source>
          , Springer-Verlag (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Kwak</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Park</surname>
            , H., Moon,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>What is Twitter, a social network or a news media?</article-title>
          <source>In: WWW '10: Proceedings of the 19th international conference on World wide web</source>
          , New York, NY, USA, ACM (
          <year>2010</year>
          )
          <volume>591</volume>
          {
          <fpage>600</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Java</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Song</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Finin</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tseng</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>Why we twitter: understanding microblogging usage and communities</article-title>
          .
          <source>In: Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis</source>
          , New York, NY, USA, ACM (
          <year>2007</year>
          )
          <volume>56</volume>
          {
          <fpage>65</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Huberman</surname>
            ,
            <given-names>B.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Romero</surname>
            ,
            <given-names>D.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <article-title>Social networks that matter: Twitter under the microscope</article-title>
          .
          <source>First Monday</source>
          <volume>14</volume>
          (
          <issue>1</issue>
          ) (
          <year>January 2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>DeVoe</surname>
          </string-name>
          , K.M.: Bursts of information: Microblogging.
          <source>The Reference Librarian</source>
          <volume>50</volume>
          (
          <issue>2</issue>
          ) (
          <year>2009</year>
          )
          <volume>212</volume>
          {
          <fpage>214</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Bruns</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burgess</surname>
            ,
            <given-names>J.E.</given-names>
          </string-name>
          :
          <article-title>The use of twitter hashtags in the formation of ad hoc publics</article-title>
          .
          <source>In: 6th European Consortium for Political Research General Conference</source>
          , University of Iceland, Reykjavik (
          <year>August 2011</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>