<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Developing Argumentation Dialogues for Open Multi-Agent Systems</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Bas Testerink</string-name>
          <email>B.J.G.Testerink@uu.nl</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Floris Bex?</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Utrecht University</institution>
        </aff>
      </contrib-group>
      <abstract>
        <p>Dialogue systems attempt to capture aspects of multi-agent communication with the aim of understanding, improving, and automatically recreating such communication. Though there is ample research into the formal aspects of dialogue games for argumentation, actual software and development tools that allow for the deployment of open, multiagent dialogue environments are lacking. This demo presents the rst steps towards a development framework for an open multi-agent system in which agents can engage in peer-to-peer argumentation dialogues.</p>
      </abstract>
      <kwd-group>
        <kwd>Open Multi-Agent Systems</kwd>
        <kwd>Argumentation Dialogues</kwd>
        <kwd>AgentOriented Programming</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        In [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], we present a generic framework for expressing argumentation dialogues
in open multi-agent systems. One of our motivations is that we aim to make
the development of multi-agent argument dialogue systems more accessible and
thus increase the deployment of such systems in realistic, large-scale settings. As
a case-in-point, we are currently developing a prototype multi-agent system for
the Dutch National Police [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], where various agents work together to build a case
regarding internet trade fraud (e.g. scammers on eBay or fake online stores). We
have agents that interact with human users (victims, police detectives), agents
that exchange information with external services (banks, trade sites) and agents
that automatically combine and reason with information from these di erent
sources. Furthermore, the agents communicate peer-to-peer, that is, there is no
governing middleware or entity that checks the communication. We have opted
for this distributed, open multi-agent system because (i) the police uses strict
privacy policies that make it undesirable to centrally gather and reason with
data; and (ii) not all participating agents in the system are known beforehand
{ for instance, the human users that le online complaints are unknown.
      </p>
      <p>
        As our main technique for reasoning we use argumentation [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] this ts the
legal reasoning in the trade fraud cases very well and has as added bene t that
? https://git.science.uu.nl/B.J.G.Testerink/OO2APL-P2PArgumentationDialogDemo
contains a video demo, a working demo and the source code, please see readme.md
we can make discussions among agents more transparent to human users.
Communication among the agents is therefore governed by argumentation dialogue
protocols, which allow us to determine whether a speci c message from one agent
to another is legal in the context of the dialogue so far.
      </p>
      <p>
        In [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] we present our formal foundation for peer-to-peer argumentation
protocol speci cations based on reusable elements called dialogue templates [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
In this paper we demonstrate a practical realization of this principle which is
a development framework that supports programming protocols and
protocolinterpreting agents. Our framework only requires programming knowledge of
Java. Agents that use our modules always have access to the state of dialogue
according to the protocol, based on the messages that they themselves sent and
received. This view can be exported as a JSON object, so that the decision logic
on what to do with that view (i.e., determining whether to utter a locution, and
if so, what the next locution to utter is) can be implemented in another language
if so desired.
      </p>
      <p>Our framework is open-source and we hope that it stimulates the
development of argumentation dialogue systems or may serve as an example to other
developers. In this demonstration we present four main contributions: 1) the
ability to develop protocols, 2) the agent modules that interpret a protocol to
automatically determine which messages are legal and how they ought to be
interpreted, 3) an example multi-agent system that can re-enact dialogues and
4) a visualization of the view of an agent on a dialogue, given the protocol.</p>
      <p>In Section 2 we discuss the relevant terminology to understand the demo.
Section 3.1 discusses the approach towards implementing protocols. Section 3.2
describes the example multi-agent system for dialogue re-enactment. Finally, 3.3
goes into detail on our example scenario and the visualization. Section 4 discusses
related and future work and concludes the paper.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Background</title>
      <p>
        Argumentation dialogues [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] consist of a coherent series of locutions, or
utterances, made by the participants in the dialogue. Such multi-agent dialogues are
governed by dialogue protocols, which describe the permitted locutions, how and
when the dialogue starts and ends, how locutions commit the players to certain
claims and how locutions may be combined into exchanges. Take, for example,
one of the possible protocols for persuasion dialogues given in [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. The protocol
mandates that a dialogue starts when one agent, the proponent, makes a claim
P, committing it to P . The other agent, the opponent, can either challenge this
claim by uttering why P, which can in turn be countered by the proponent either
retracting its commitment to P or by providing a supporting argument P since
Q. A claim P can also be attacked by providing a counterargument :P since Q.
Furthermore, any argument of the form R since S made by either party can be
countered by the other party by giving an argument :R since T or :S since T .
The dialogue terminates when either the proponent retracts its original claim P
or when the opponent concedes proponent's original claim P .
      </p>
      <p>Dialogue
(Argument 2)</p>
      <p>Alice:
claim Carl is a</p>
      <p>fraud</p>
      <p>Alice is
committed to</p>
      <p>
        For argumentation dialogues it is important to distinguish between argument
1 { arguments as a static structure of premises that are reasons for or against
conclusions (as in, he prepared an argument) { and argument 2 { arguments
as debates or dialogues (as in, they had an argument). The idea is that during
a dialogue (argument 2) the participants construct and navigate an underlying
argument 1 structure of claims and reasons [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. For an example, consider Fig. 1.
At the top (grey boxes), a simple dialogue in which Alice claims that Carl is a
fraud is rendered, where the arrows indicate the ow of the dialogue. At the
bottom (white boxes), the corresponding argument 1 is rendered: from the premise
`Alice paid Carl &amp; Carl did not send the product' it is inferred that `Carl is a
fraud'. The link between argument 1 and 2 stems from speech act theory, where
a speech act is a locution (`Alice says that Carl is a fraud'), but also an
illocutionary act which consists of the illocutionary force, that is, the intention of
uttering a locution (Intake Agent says `why Carl is a fraud?' with the intention
of challenging that `Carl is a fraud') and the propositional content (`Carl is a
fraud'). Note that the same locution can encompass multiple illocutionary acts:
by saying `Carl is a fraud' Alice not only asserts that `Carl is a fraud' but also
commits herself to this proposition (cf. the small rounded box in Fig. 1).
      </p>
      <p>
        Two core formal concepts in [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], on which our implementation is based, are
dialogue graphs and dialogue templates. A dialogue graph captures the state of
the dialogue from the point of view of one of the agents. In our implementation,
this graph represents argument 1, argument 2 and commitments (cf. Fig. 1).
The argument graphs in our implementation are slightly adapted versions of
AIF argument graphs [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], which are based on the underlying AIF ontology
speci cation [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. This ontology places a distinction between information nodes or
I-nodes, to represent locutions or propositions, and scheme nodes or S-nodes, to
denote general patterns of reasoning or the relations between I-nodes. Subclasses
of S-nodes are RA (Rule Application) nodes and CA (con ict application) nodes.
We further add commitment nodes indicating the commitment of an agent to
an I-node.
      </p>
      <p>Dialogue templates de ne how to update the graph when receiving and
sending locutions. They are used to represent dialogue protocol rules, as they contain
for each type of locution that is sent or received the possible replies (transitions
to other locutions). For example, one template says that if a why P message is
received, the argument 2 graph is changed to include this locution, the
argument 1 graph remains unchanged, and a P since Q message may be sent back.
Another template says that sending a P since Q message not only updates
argument 2 but also updates the argument 1 graph to include Q and the inference
(RA-node) between Q and P , and also commits the sender to P and Q.
3
3.1</p>
    </sec>
    <sec id="sec-3">
      <title>Software Overview and Demonstration</title>
      <sec id="sec-3-1">
        <title>Programming Protocols</title>
        <p>Our framework provides preprogrammed classes that help a developer to specify
dialogue graphs and templates. We preprogrammed the following types of nodes
in dialogue graphs: Argument2Node's represent locutions that were uttered and
edges between these nodes represent transitions from one locution to the next.
AIF I- and S-nodes are implemented as INode's and SNodes's. CommitmentNode's
track which agent is committed to what propositions. EffectNode's track the
illocutionary force of locutions, that is, what the e ect of a locution is on the
argument 1 graph or the commitments. The default dialogue graph
implementation can be extended for added functionalities. This is demonstrated with our
demo protocol where the DemoDialogGraph maintains a mandate for each agent
what they may utter to what other agent.</p>
        <p>Templates can be made by supplying supplying three functions called the
guard, response and argument 1 update function. The guard of the template
tells given a dialogue graph and a new argument 2 node whether the template
is applicable. The response function returns, given a dialogue graph and a new
locution, the argument 2 nodes from the dialogue graph to which the new
locution is a direct response. The argument 1 update is a function that accepts
a dialogue graph and a new argument 2 node. The template class has a
preprogrammed apply-method that for a new message checks the guard, and if the
guard passes, applies the response function and argument 1 update. Any changes
that were made during the execution of the argument 1 update are automatically
recorded. After the function is applied, the template will create an e ect node
that connects the new message to the changes that were made while executing
the argument 1 update function. A protocol itself is simply a set of templates as
demonstrated in the demoProtocol package.</p>
        <p>
          Finally, we preprogrammed classes for the development of Java-based agents
that interact using argumentation dialogue protocols. For this we built on top of
OO2APL [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]1. When an agent receives a message it will check if the sender was
allowed to sent the message by checking whether any of the templates applies
given the current dialogue graph. If not, then the agent immediately replies with
a bounce message to the sender. If there was an applicable template, then that
template updates the dialogue graph. The agent can send a message with a
preprogrammed send method which automatically does the entire process again,
but for the message to be sent. If the agent tries to illegally send a message it
automatically receives an internal error.
1 https://git.science.uu.nl/OO2APL/
3.2
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>Re-enacting Dialogues</title>
        <p>Our demonstration comes with a dialogue reader which is a multi-agent system
that can re-enact a dialogue. There are di erent ways in which the reader can be
used. When developing a protocol, one can write manually dialogues (similar to
user stories) and see whether the protocol builds the dialogue graph as intended.
For instance in our example scenario we do not want that the intake agent
commits itself in the dialogues with complainants to the fact that someone is a
fraud2. Another usage is to re-enact the dialogue given a log to trace back when
certain agreements where reached or where some agent violated the protocol.</p>
        <p>There are two mandatory input les for the dialogue reader. The rst one is
the invitations le, that tells the system who recruited who into the
argumentation session. The second input le is the script/log of the locutions that were
uttered. On the Java level the reader further requires a parser from strings (the
locutions in the locutions le) to the locution objects that represent them, and
the constructor of the protocol that is to be used. The reader then reads all the
les, re-enacts the dialogue, and outputs the dialogue graphs of the agents as
JSON. The re-enactment is done by instantiating DialogReaderAgent agents.
Such an agent's decision logic when it receives a message is to simply prompt
the next agent to send the next locution in the script.</p>
        <p>Note that this dialogue reader multi-agent system can be used as a starting
point: the agents in have a method called handleReceivedArgumentationMessage
of which the body can be replaced by a di erent decision logic on how to
respond. Thus, multi-agent systems that include agents with their own knowledge
bases and decision logics can be easily developed.
3.3</p>
      </sec>
      <sec id="sec-3-3">
        <title>Example Dialogue and Visualization</title>
        <p>The dialogue reader creates JSON objects that represent the dialogue graphs of
the dialogue participants, which are visualized on a web page3. The graph is split
into ve components; the dialogue itself, the argument 2 graph, the argument 1
graph, the illocutionary force inspector and the AIF graph.</p>
        <p>Our demonstration uses an example that is based on our project at the Dutch
National Police, and also strongly resembles the current modus operandi for
handling for instance online trade fraud cases. The scenario is that complainants
can le their complaints by interacting with an intake agent and a detective can
interact with an analyst agent. When the detective asks the analyst agent a
question, it discusses the question with the intake agent and another agent that may
requisition information from third parties. Our demo thus contains agents that
represent humans, third parties and automated agents. The human agents in
our demo are the complainants Alice and Bob, who le complaints that accuse
2 With online trading fraud, your complaint can have all the signs of a fraud case
without being a fraud case. Roughly speaking, if someone just forgot to send an
order once, then the police will not immediately arrest that person for fraud.
3 see /visualizer/visualizer.html in the demo repository
some person named Carl of fraud, and a detective, who asks if Carl is a
suspected fraudster. The third parties are the Dutch trading website Marktplaats,
a bank, and the general administration system from municipalities called GBA.
The automated agents stand in between the other agents. A requisition agent
can demand information from the third parties, an intake agent interacts with
complainants and an analyst agent interacts with detectives. The requisition,
intake and analyst agent can also interact with each other to discuss topics.</p>
        <p>For a visualization of the agents and who is talking to who, see the top
part of the demo page or `connectivityGraph.png' in the screen shot directory.
The agents in our demonstration discuss the topic of whether Carl is a fraud.
The full transcript of the discussion that the agents have can be found in the
`dialogues/intake/locutions.csv' le. We can click on an agent's icon in order to
inspect its view on the dialogue. As an example we will go through the view
of Alice who les a complaint with the intake agent. In the dialogue tab (or
`dialogue.png') we can see a timeline with events as observed by Alice. Here, we
only see the messages that Alice sent and received to and from the intake agent.
Initially, Alice claims that Carl is a fraud. The agent replies with a why-challenge
to trigger Alice into defending her claim. She then replies with the argument that
since she paid and Carl did not sent her package, it must be the case that Carl
has scammed her. To the intake agent it is not obvious why Alice thinks that the
package has not been sent. It therefore challenges this claim with another why
locution. Alice defends her claim by stating that she waited but that the package
was not delivered. The intake agent then concedes the point that the package
was probably not sent by Carl. Alice has nothing else to say so she skips. The
intake agent then asks whether Carl sent any pictures of an ID card and whether
Carl gave his address. Both questions are answered a rmatively by Alice and
the intake dialogue ends.</p>
        <p>In the argument 2 tab, we can click on the cluster `Alice, Intake agent' to
show the locutions that were uttered and the response relation among them
(alternatively, see `argument2.png'). From this view we can see that the protocol
interpreted the dialogue between Alice and the intake agent as three coherent
sub-dialogues. There is a connected graph concerning the topic of whether Alice
thinks Carl is a fraud, a connected graph about whether Carl sent picture of an
ID card, and a connected graph about whether Carl gave his address.</p>
        <p>The argument 1 tab (alternatively, see `argument1.png') shows the e ect of
the dialogue in terms of argument 1, that is, the propositions that were asserted
or committed to and the inferences that were made. We can see, for instance,
that an inference was made from `Alice paid' and `The package was not sent' to
`Carl is a fraud'. We can also see that the agent does not commit to this inference,
nor to the conclusion. This is a desired outcome, since the intake agent ought
not commit itself to propositions regarding criminal facts.</p>
        <p>
          In the illocutionary force tab we can see for each locution how it modi ed
the argument 1 graph. For instance we can see that the concede locution (click
on locution 6) from the intake agent that referred to the proposition that Alice's
package was not sent, commits the agent to that proposition (alternatively, see
`illocutionaryForce.png' in the screenshot directory). Finally, the AIF tab shows
for inference in the argument 1 graph the applied argument scheme and how
the elements are connected to the AIF ontology (cf. [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ]). We use in this
demonstration only the schemes `apply defeasible rule' for inference and `con ict by
negation' proposition con icts.
        </p>
        <p>When we look at the dialogue from the detective's view, then we only see
it ask the analyst whether Carl is a fraud, and immediately see the analyst's
response that this is not the case. The detective does not see the entire
discussion that went on between the analyst, intake and requisition agents about this
question, nor is he bothered with the led complaints. Even though the intake
agent is initially convinced that Carl is a fraud, based on the complaints, the
analyst persuades the intake agent that actually Carl just did not deliver as an
exception to generally bona de behaviour. Notice that Alice and Bob argue
that Carl is a fraud di erently from the intake agent. Even though the intake
agent uses di erent argumentation rules than Alice and Bob, it can still have
meaningful and useful dialogues with them.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Conclusion</title>
      <p>In this paper we have discussed our development framework for programming
argumentation dialogue protocols that are suitable for peer-to-peer settings. We
demonstrated this framework with an example that was taken from an ongoing
research project with the Dutch National Police. The framework allows a
programmer to make a protocol by specifying the dialogue templates. The software
also provides a multi-agent system that can read transcripts of dialogues and
produces the accompanying dialogue graphs for each participant. The dialogue
graphs are exported as JSON objects, which makes the software also usable to
connect to other technologies through a web-interface. Finally, we demonstrated
a web-based visualization of the dialogue graph JSON objects so that a developer
can inspect the graphs in a more human-readable manner.</p>
      <p>
        Closely related to our framework is the Dialogue Game Execution Platform
(DGEP) [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], which is also a tool that helps in developing argumentation dialogue
protocols. The main di erence with our framework is that DGEP is not an open
multi-agent framework but rather a middleware platform that, given a locution,
outputs the legal moves. Our framework can be used to program agents that
interact with the DGEP platform if desired, although this is not strictly necessary
since our agents can interpret protocol speci cations themselves.
      </p>
      <p>
        Future work concerns the research that serves as the basis for this demo. We
have formalized peer-to-peer protocols [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] but certain aspects of the demo, such
as the connection between argument 1 and 2, have not yet been fully worked out.
A further aim is to be able to automatically compile the DGDL speci cations [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]
of the many argumentation dialogue protocols that are available to a protocol in
the presented development framework. Another major point is the formalization
and implementation of di erent agent argumentation strategies.
      </p>
      <p>
        Finally, with respect to intelligent interfaces, such as the intake agent, it is
desirable that humans can interact with such agents in a natural way - while
some form-based interaction with pre-set choices is ne, there are also situations
in which we would want the human to be able to simply type in natural
language text which is then parsed and used by the automated intake agent. In our
project, we have taken rst steps to evaluate the possibilities of such
information extraction [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] and aim to further develop this in the context of the types of
dialogue explained in this demonstration.
      </p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <p>This research is part of the project Intelligence Ampli cation for Cybercrime,
which has been funded by the Dutch National Police Innovation Programme.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>T. J.</given-names>
            <surname>Bench-Capon</surname>
          </string-name>
          and
          <string-name>
            <surname>P. E. Dunne.</surname>
          </string-name>
          <article-title>Argumentation in arti cial intelligence</article-title>
          .
          <source>Arti cial intelligence</source>
          ,
          <volume>171</volume>
          (
          <fpage>10</fpage>
          -15):
          <volume>619</volume>
          {
          <fpage>641</fpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>F.</given-names>
            <surname>Bex</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Lawrence</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Reed</surname>
          </string-name>
          .
          <article-title>Generalising argument dialogue with the dialogue game execution platform</article-title>
          .
          <source>In Proceedings of COMMA</source>
          <year>2014</year>
          , volume
          <volume>266</volume>
          of Frontiers in
          <source>Arti cial Intelligence and Applications</source>
          , pages
          <volume>141</volume>
          {
          <fpage>152</fpage>
          . IOS Press,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>F.</given-names>
            <surname>Bex</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Modgil</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Prakken</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Reed</surname>
          </string-name>
          .
          <article-title>On logical speci cations of the argument interchange format</article-title>
          .
          <source>Journal of Logic and Computation</source>
          ,
          <volume>23</volume>
          (
          <issue>5</issue>
          ):
          <volume>951</volume>
          {
          <fpage>989</fpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>F.</given-names>
            <surname>Bex</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Peters</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Testerink</surname>
          </string-name>
          .
          <article-title>A.I. for online criminal complaints: From natural dialogues to structured scenarios</article-title>
          .
          <source>In Workshop A.I. for Justice - Proceedings of ECAI</source>
          <year>2016</year>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>F.</given-names>
            <surname>Bex</surname>
          </string-name>
          and
          <string-name>
            <given-names>C.</given-names>
            <surname>Reed</surname>
          </string-name>
          .
          <article-title>Dialogue templates for automatic argument processing</article-title>
          .
          <source>In Proceedings of COMMA</source>
          <year>2012</year>
          , volume
          <volume>245</volume>
          of Frontiers in
          <source>Arti cial Intelligence and Applications</source>
          , pages
          <volume>366</volume>
          {
          <fpage>377</fpage>
          . IOS Press,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>M.</given-names>
            <surname>Dastani</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Testerink</surname>
          </string-name>
          .
          <article-title>From multi-agent programming to object oriented design patterns</article-title>
          .
          <source>In Engineering Multi-Agent Systems</source>
          , pages
          <fpage>204</fpage>
          {
          <fpage>226</fpage>
          . Springer International Publishing,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>H.</given-names>
            <surname>Prakken</surname>
          </string-name>
          .
          <article-title>Formal systems for persuasion dialogue</article-title>
          .
          <source>The Knowledge Engineering Review</source>
          , (
          <volume>21</volume>
          ):
          <volume>163</volume>
          {
          <fpage>188</fpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>C.</given-names>
            <surname>Reed</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Wells</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Budzynska</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Devereux</surname>
          </string-name>
          .
          <article-title>Building arguments with argumentation: the role of illocutionary force in computational models of argument</article-title>
          .
          <source>In Proceedings of COMMA</source>
          <year>2010</year>
          ., volume
          <volume>216</volume>
          of Frontiers in
          <source>Arti cial Intelligence and Applications</source>
          , pages
          <volume>415</volume>
          {
          <fpage>426</fpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>M.</given-names>
            <surname>Schraagen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Brinkhuis</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Bex</surname>
          </string-name>
          .
          <article-title>Evaluation of Named Entity Recognition in Dutch online criminal complaints</article-title>
          .
          <source>Computational Linguistics in the Netherlands Journal</source>
          ,
          <volume>7</volume>
          ,
          <year>2017</year>
          . To appear.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <given-names>B.</given-names>
            <surname>Testerink</surname>
          </string-name>
          and
          <string-name>
            <given-names>F.</given-names>
            <surname>Bex</surname>
          </string-name>
          .
          <article-title>Speci cations for peer-to-peer argumentation dialogues</article-title>
          .
          <source>In Proceedings of PRIMA 2017, Lecture Notes in Arti cial Intelligence</source>
          . Springer,
          <year>2017</year>
          . To appear.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>S.</given-names>
            <surname>Wells</surname>
          </string-name>
          and
          <string-name>
            <given-names>C.</given-names>
            <surname>Reed</surname>
          </string-name>
          .
          <article-title>A domain speci c language for describing diverse systems of dialogue</article-title>
          .
          <source>Journal of Applied Logic</source>
          ,
          <volume>10</volume>
          (
          <issue>4</issue>
          ):
          <volume>309</volume>
          {
          <fpage>329</fpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>