<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <issn pub-type="ppub">1613-0073</issn>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>what we argue for</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Giulia D'Agostino</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Università della Svizzera Italiana</institution>
          ,
          <addr-line>via Bufi 13, 6900 Lugano</addr-line>
          ,
          <country country="CH">Switzerland</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Interactions between financial analysts and company managers are particularly meaningful in the context of quarterly Earnings Conference Calls (ECCs), notably in the Q&amp;A sessions. The current contribution delves into the alternation between argumentation and explanation within such sessions, particularly those in which the stakes are particularly high - i.e., in the context of a company crisis. The motivation for such a study is the acknowledgement that explanations found in such a situation globally contribute to the overarching argumentative goal of defending company reputation.</p>
      </abstract>
      <kwd-group>
        <kwd>argumentation</kwd>
        <kwd>explanation</kwd>
        <kwd>financial communication</kwd>
        <kwd>argumentative reconstruction</kwd>
        <kwd>argument mining</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>CEUR
ceur-ws.org</p>
    </sec>
    <sec id="sec-2">
      <title>1. Introduction</title>
      <p>
        Explanations represent a feature of a more general argumentative discussion [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. The current
context of inquiry is the financial domain and, particularly, Q&amp;A sessions of Earnings Conference
Calls (ECCs) – dialectical exchanges between financial analysts and corporate representatives.
Particularly, corporate representatives are called to account on results and choices related to
the previous quarter and are asked for insights and predictions about the following one. In such
exchanges:
1. Explanations are implemented in the context of an overall activity type the goal of which is
to give enough information to let analysts and investors valuate the company in the most
favourable way for the latter [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Thus, the guidance the company tries to convey is to
hold or buy its stocks, and explanations graft on the global reliability and accountability
discourse – taking shape of proof of good-willingness, sincerity and integrity of the
company.
      </p>
      <p>interests.
2. Distinctly, causal explanations describe the relationship between a cause and an efect.</p>
      <p>Here it is particularly relevant to notice the framing of the cause: if the efect is positive,
the attributed cause will be internal (deliberate, mostly permanent); if the efect is negative,
the imputed cause will be external (uncontrollable and hopefully temporary). In both
cases, the overall argumentative strategy aims at showing that the management worked
properly and – even in the case of failure – it did its best in safeguarding investors’
∗The present research is supported by the Swiss National Science Foundation, grant n° 200857
CEUR
Workshop
Proceedings</p>
      <p>
        Explanations - and not just causal ones [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] – therefore possess an indirect argumentative
relevance, closely associated with the structural and mechanical properties of the development
of a critical discussion in context [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Explanations are expected to be integrated into the
argumentative stage where the explanans-explanandum complex is implemented as an
argumentative premise. Besides, the unearthing of the general argumentative use of explanations
for argumentative purposes is of concern for the argument mining community notwithstanding
with the application to a specific textual genre; with such broad goal in mind, § 6 will showcase
a preliminary pipeline for the automation of extraction and classification of arguments and
explanations.
      </p>
    </sec>
    <sec id="sec-3">
      <title>2. Theoretical framework</title>
      <sec id="sec-3-1">
        <title>2.1. Explanations</title>
        <p>
          Regarding the concept of argumentation, we will adopt the pragma-dialectical definition [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]:
Argumentation is a verbal, social, and rational activity aimed at convincing a
reasonable critic of the acceptability of a standpoint by putting forward a constellation
of propositions justifying or refuting the proposition expressed in the standpoint.
        </p>
        <p>
          Conversely, not to conflict with the ongoing theoretical complex discussion about what
explanations are or how they should be described (see [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ] for a brief overview on the matter),
the present contribution will board onto a stipulative approach towards what an explanation is:
An explanation is an antecedent-subsequent pair of statements logically connected
through a process of reasoning. It is partly similar to argumentation, where a
conclusion is supported by shared premises. However, the ‘standpoint’ (i.e., the
explanandum) is already known and accepted in the context, whereas the ‘argument’
side (i.e., the explanans) – either already known or promptly introduced to the
conversation – needs to be approved as inevitably holding a meaningful relationship
with the explanandum.
        </p>
        <p>
          The core of the analysis revolves around the instruments of Inference Anchoring Theory
(IAT) [
          <xref ref-type="bibr" rid="ref6 ref7">6, 7</xref>
          ], which gives annotators the possibility to both specify the logical relations between
locutions (inference, conflict and rephrase) and the illocutionary forces characterising either a
single locution or the relationship between two or more locutions. Thus, exploiting the design
ofered by such a theory, the analytical framework draws the divide between argumentation
and explanation as follows: both instances are depicted by two (or more) locutions connected
by an inference; however, argumentative pairs are anchored with an ‘arguing’ force (see Figure
1), whereas explanatory ones are anchored with an ‘explaining’ force (see Figure 2). Undeniably,
the boundaries between argumentation and explanation are not always clear-cut; not only
because it is not necessarily clear which element of the pair should be taken for granted or
truthful - maybe one is just interactionally presented as such - but also because their relationship
(i.e., its illocutionary force) is drafted contextually.
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>2.2. Conceptual remarks</title>
        <p>
          The analysis will employ the high-level concepts of text segmentation Maximal Interrogative
Unit (MIU) and Maximal Answering Unit (MAU) [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. A MIU is a collection of sentences that
could maximally identify with a question turn, which is characterized by the following attributes:
• It is a macro-unit which groups discursive moves within the same question turn and
comprising no less than one question, each discursive move being the length of (at least)
one sentence;
• All the discursive moves in a MIU prepare, rephrase or modulate the same objective;
• All discursive moves within a MIU can be satisfied by a single corresponding Maximal
        </p>
        <p>Answering Unit.</p>
        <p>A MAU is determinable in relation to the MIU from which it is triggered: it is a collection
of sentences, within an answer turn, that globally react to a MIU. A MAU can maximally
correspond to the entire turn, or minimally correspond to a single sentence.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>3. Data and method</title>
      <p>The dataset for the current study is constituted by the 65 question-and-answer pairs drawn
from the four ECCs delivered in 2021 by Swiss bank Credit Suisse. It comprises 111 MIUs and
118 MAUs, for a total of 1,710 sentences and 32,102 words.</p>
      <p>The reasons behind the choice of Credit Suisse as a case study include the evaluation of
the company’s performance: in a precarious environment, financial analysts are expected to
perform adversarial questions. The most remarkable negative features that were taken into
consideration are:
• CS steadily reported losses along the whole financial year
• although a certain variability in CS stock prices can somewhat be traced, their value
inevitably drops in occasion of each ECC
• CS incurred in at least two major scandals during the period considered (financial and
reputational crises)</p>
      <p>
        The Q&amp;A pairs of interest were selected as follows. Argumentation is, by design, a pervasive
strategy of dialectical exchanges in this context, and therefore associated with any type of
request; however, studies in the genre showed that the type of request impacts the density of
argumentation in the related answers [
        <xref ref-type="bibr" rid="ref10 ref9">9, 10</xref>
        ]. Since the distribution of explanatory stances
across request types is not known a priori, the current working hypothesis is that explanations
will certainly appear in reply to requests overtly demanding them. Therefore, the subset is
constituted by MIU-MAU pairs where the MIU contains at least one request for explanation (or
for a clarification regarding an explanation).
      </p>
      <p>
        The pairs were first annotated by trained annotators – supported by the oficial annotation
manual of the project [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] – with respect to Dialogue moves, Request types and MIU-MAU
linking. This step was performed on the INCEpTION annotation platform [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]; inter-annotator
agreement Kappa [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] was consistently substantial or better (&gt;0.60) for all annotation layers.
Subsequently, requests for explanation and for clarification queried from the previous step were
annotated by team researchers according to a genre-specific adaptation of IAT on the OVA3
platform 1 [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] and stored on AIFdb 2 [15].
      </p>
    </sec>
    <sec id="sec-5">
      <title>4. Hypotheses</title>
      <p>The hypotheses for the current inquiry are that, in co-occurrence with requests more or less
explicitly targeting an explanation as a reaction:</p>
      <p>H1: The reply will exhibit at least one explanation, aside from a variable number of
argumentative instances, especially if the MIU does comprise more than one question (whether
more than one request for elaboration or not)
H2: The ratio between (occurrences of) argumentation and explanation will be consistent
across answers</p>
    </sec>
    <sec id="sec-6">
      <title>5. Analysis and results</title>
      <p>
        The combination of requests for explanation and for ‘clarification on an explanation’
(henceforth: both under the label ‘for explanation’) appear to be the third most numerous type of
requests in the dataset, following opinion and elaboration. This is consistent with previous
studies [
        <xref ref-type="bibr" rid="ref10 ref9">16, 9, 10</xref>
        ]; in Table 1 the number and distribution of requests. Of the 35 requests for
explanation listed in the dataset, 16 co-occurred with other requests within the same MIU; of
those, 7 comprised at least one other request for an explanation. This led to the analysis of 24
distinct MIUs.
      </p>
      <p>H1: verified
The correlation between the density of requests for explanation within a MIU and the amount
of explanations in the answer is significant (r(22)=.13, p=.552), although weak; however, as
expected by H1, we can see a general stronger eagerness towards providing explanations if the
request(s) for an explanation are accompanied by other request types within the same MIU
(r(22)=.17, p=.426).</p>
      <p>H2: verified
The correlation between argumentation and explanation is overall slightly significant (r(22)=.25,
p=.232); however, the results were still quite below expectations (data aggregated by quarter in
Table 2. To investigate the causes, an additional conjecture was added: constraining questions,
i.e., proposing a possible answer (namely, polar [17] or alternative [18] formulation) would
receive fewer explanations as a reaction: since preferable answers are provided, explanations
behind them are taken for granted. The correlation between explaining instances in the reply
and constraining question formulation was indeed negative and significant (r pb= -.36, p=.08708),
see Table 3 for the specifics. Conversely, the correlation between non-constraining MIUs and
both arguing and explaining instances in answers was finally significant and in line with the
hypothesis (r(12)=.32, p=.257).</p>
      <sec id="sec-6-1">
        <title>1http://ova3.arg.tech/ 2the OVA3 corpus is available at: http://corpora.aifdb.org/ArgExplCMNA23</title>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>6. Further developments</title>
      <p>On the qualitative side, further steps will explore whether and how explanations have a
distinct role in the overall argumentative strategy of the answerer; this would include extending
the analysis to other request types. This study, however, configures a preliminary move in the
perspective of developing a pipeline of automatic identification and extraction of explanations
(as opposed and complementary to arguments). This appears to be a challenging although
meaningful objective both for its general purpose application, as well as its specific activity
type-related outcomes, along the lines of the work by Chen et al. [19] on opinion mining in
the financial domain. Therefore, we presently display a tentative sequence of tasks - some of
which are already under development within the general project this paper is an instance of
– operationalising such an all-inclusive efort. The course of action needs to focus on three
main areas of automation - each of which constitute a progressive step in the pipeline but also
account for a meaningful accomplishment in itself for argument mining, which is the area of
research defined as “the automatic identification and extraction of the structure of inference
and reasoning expressed as arguments presented in natural language” [20]. These are:</p>
      <sec id="sec-7-1">
        <title>1. Connection of MAUs to antecedent MIUs;</title>
        <p>2. Identification of inferences (RA nodes, in AIF terms) within units [ 21] - both for MIUs
and MAUs;
3. Classification of illocutionary connections anchored in RA nodes, i.e., diferentiating
between argumentative and explanatory instances.</p>
        <p>The advancement of the first two tasks is currently ongoing, although at diferent stages. The
full success of the second step would be reached with the correct detection of unit boundaries
which is currently under development, starting from MIUs. The third step, however, advances a
series of technical issues bound to the contextual characteristics that, by definition, delineate
the very existence of an explanation.</p>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>7. Conclusions</title>
      <p>In the present contribution an exploratory analysis on a dialogical case study in the financial
domain was performed, examining the relationship between argumentation and explanation;
explanations were claimed to have a relevant argumentative role in the overall discussion
which unfolds in the interaction. A stable parallelism between MIUs containing requests for
explanation, especially featuring multiple requests, and the existence of explanatory stances in
their answers was observed; further investigation will also include diferent types of request
in the sample. A valid correlation between the volume of argumentation and explanation
across answers to requests for explanation was drawn. Finally, a pipeline of analysis in the
argument mining domain was launched, aiming at the automatic recognition of argumentation
and explanation. Such a task is believed to be fundamental towards the uncovering of both
the nature of the argumentative strategy of answerers and the overall strategic alternation of
argumentative patterns [22, 23] in the dialectical exchange.
2014/comma2014-ova.pdf. doi:1 0 . 3 2 3 3 / 9 7 8 - 1 - 6 1 4 9 9 - 4 3 6 - 7 - 4 6 3 .
[15] J. Lawrence, C. Reed, AIFdb Corpora, in: Computational Models of Argument: Proceedings
of COMMA 2014, 2014, pp. 465–466. doi:1 0 . 3 2 3 3 / 9 7 8 - 1 - 6 1 4 9 9 - 4 3 6 - 7 - 4 6 5 .
[16] R. Palmieri, A. Rocci, N. Kudrautsava, Argumentation in earnings conference calls.
Corporate standpoints and analysts’ challenges, Studies in communication sciences 15, 2015
(2015) 120–132. doi:1 0 . 1 0 1 6 / j . s c o m s . 2 0 1 5 . 0 3 . 0 1 4 .
[17] G. D’Agostino, C. Lucchini, Do you think this? Constructing and suggesting preferable
standpoints in questions, Leiden, 2023.
[18] A. Rocci, O. Yaskorska-Shah, G. D’Agostino, C. Lucchini, Argumentative patterns initiated
by closed-list questions in accountability dialogues. A corpus study of financial conference
calls, Rome, 2022.
[19] C.-C. Chen, H.-H. Huang, H.-H. Chen, From Opinion Mining to Financial Argument
Mining, SpringerBriefs in Computer Science, Springer Singapore, Singapore, 2021. URL:
https://link.springer.com/10.1007/978-981-16-2881-8. doi:1 0 . 1 0 0 7 / 9 7 8 - 9 8 1 - 1 6 - 2 8 8 1 - 8 .
[20] J. Lawrence, C. Reed, Argument Mining: A Survey, Computational Linguistics 45 (2019)
765–818. URL: https://doi.org/10.1162/coli_a_00364. doi:1 0 . 1 1 6 2 / c o l i _ a _ 0 0 3 6 4 .
[21] J. Lawrence, C. Reed, Mining Argumentative Structure from Natural Language text using
Automatically Generated Premise-Conclusion Topic Models, in: Proceedings of the 4th
Workshop on Argument Mining, Association for Computational Linguistics, Copenhagen,
Denmark, 2017, pp. 39–48. URL: http://aclweb.org/anthology/W17-5105. doi:1 0 . 1 8 6 5 3 / v 1 /
W 1 7 - 5 1 0 5 .
[22] F. H. van Eemeren, Identifying Argumentative Patterns: A Vital Step in the
Development of Pragma-Dialectics, Argumentation 30 (2016) 1–23. URL: https://doi.org/10.1007/
s10503-015-9377-z. doi:1 0 . 1 0 0 7 / s 1 0 5 0 3 - 0 1 5 - 9 3 7 7 - z .
[23] F. H. van Eemeren, Context-dependency of argumentative patterns in discourse, Journal
of Argumentation in Context 6 (2017) 3–26. doi:1 0 . 1 0 7 5 / j a i c . 6 . 1 . 0 1 e e m .</p>
    </sec>
    <sec id="sec-9">
      <title>B. Tables</title>
      <p>MIU
of which explanation
relation
arguing explaining
formulation
constraining question</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>F. van Eemeren</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Grootendorst</surname>
          </string-name>
          ,
          <article-title>A Systematic Theory of Argumentation: The Pragmadialectical Approach</article-title>
          , Cambridge University Press,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>B.</given-names>
            <surname>Crawford Camiciottoli</surname>
          </string-name>
          ,
          <article-title>Earnings calls: Exploring an emerging financial reporting genre</article-title>
          ,
          <source>Discourse &amp; Communication</source>
          <volume>4</volume>
          (
          <year>2010</year>
          )
          <fpage>343</fpage>
          -
          <lpage>359</lpage>
          .
          <source>doi:1 0 . 1 1</source>
          <volume>7 7 / 1 7 5 0 4 8 1 3 1 0 3 8 1 6 8 1 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>R.</given-names>
            <surname>Den</surname>
          </string-name>
          <string-name>
            <surname>Boef</surname>
          </string-name>
          , R. Van Woudenberg,
          <article-title>Non-causal Explanations in the Humanities: Some Examples</article-title>
          ,
          <source>Foundations of Science</source>
          (
          <year>2023</year>
          ). URL: https://link.springer.
          <source>com/10.1007/ s10699-023-09910-3. doi:1 0 . 1 0 0 7 / s 1 0</source>
          <volume>6 9 9 - 0 2 3 - 0 9 9 1 0 - 3</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>F. H. van Eemeren</surname>
          </string-name>
          ,
          <source>Argumentation Theory and Argumentative Practices: A Vital but Complex Relationship, Informal Logic</source>
          <volume>37</volume>
          (
          <year>2017</year>
          )
          <fpage>322</fpage>
          -
          <lpage>350</lpage>
          . URL: https://informallogic.ca/ index.php/informal_logic/article/view/5002.
          <source>doi:1 0 . 2 2</source>
          <volume>3 2 9</volume>
          / i l .
          <source>v 3 7 i 4 . 5 0</source>
          <volume>0 2 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>W. H.</given-names>
            <surname>Newton-Smith</surname>
          </string-name>
          , Explanation, in: W. H.
          <string-name>
            <surname>Newton-Smith</surname>
          </string-name>
          (Ed.),
          <article-title>A companion to the philosophy of science, number 18 in Blackwell companions to philosophy</article-title>
          , Blackwell Publishers, Malden, Mass,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>K.</given-names>
            <surname>Budzynska</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Reed</surname>
          </string-name>
          ,
          <article-title>Speech acts of argumentation: inference anchors and peripheral cues in dialogue, in: Computational models of natural argument: papers from the 2011</article-title>
          <source>AAAI Workshop</source>
          , volume WS-
          <volume>11</volume>
          -10, AAAI Press,
          <year>2011</year>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>10</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>K.</given-names>
            <surname>Budzynska</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Janier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Reed</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Saint-Dizier</surname>
          </string-name>
          ,
          <article-title>Theoretical foundations for illocutionary structure parsing</article-title>
          ,
          <source>Argument and Computation</source>
          <volume>7</volume>
          (
          <year>2016</year>
          )
          <fpage>91</fpage>
          -
          <lpage>108</lpage>
          . URL: https://content.iospress.com/download/argument-and
          <article-title>-computation/aac005?id= argument-and-computation%2Faac005. doi:1 0 . 3 2 3 3 / A A C - 1 6 0 0 0 5</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>G. D'Agostino</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Reed</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Puccinelli</surname>
          </string-name>
          ,
          <article-title>Segmentation of Complex Question Turns for Argument Mining: A Corpus-based Study in the Financial Domain</article-title>
          , in: Submitted to COLING-LREC
          <year>2024</year>
          ,
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>G. D'Agostino</surname>
          </string-name>
          , ”
          <article-title>(so long, and) thanks for all the color”</article-title>
          .
          <source>Requests of Elaboration and Answers They Trigger in Earnings Conference Calls, in: Proceedings of the 22nd Edition of the Workshop on Computational Models of Natural Argument (CMNA)</source>
          ,
          <year>Cardif</year>
          ,
          <year>2022</year>
          . URL: http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>3205</volume>
          /paper9.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>C.</given-names>
            <surname>Lucchini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Rocci</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>D'Agostino, Annotating argumentation within questions. Prefaced questions as genre specific argumentative pattern in earnings conference calls</article-title>
          ,
          <source>in: Proceedings of the 22nd Edition of the Workshop on Computational Models of Natural Argument (CMNA)</source>
          ,
          <year>2022</year>
          . URL: http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>3205</volume>
          /paper7.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>C.</given-names>
            <surname>Lucchini</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>D'Agostino, Good answers, better questions. Building an annotation scheme for financial dialogues</article-title>
          ,
          <source>Technical Report</source>
          ,
          <year>2023</year>
          . URL: https://n2t.net/ark:/12658/srd1326777.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>J.-C. Klie</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Bugert</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Boullosa</surname>
            , R. Eckart de Castilho,
            <given-names>I. Gurevych</given-names>
          </string-name>
          , The INCEpTION Platform:
          <article-title>Machine-Assisted and Knowledge-Oriented Interactive Annotation</article-title>
          ,
          <source>in: Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations</source>
          ,
          <article-title>Association for Computational Linguistics</article-title>
          , Santa Fe, New Mexico,
          <year>2018</year>
          , pp.
          <fpage>5</fpage>
          -
          <lpage>9</lpage>
          . URL: https://aclanthology.org/C18-2002.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>J.</given-names>
            <surname>Cohen</surname>
          </string-name>
          ,
          <article-title>A Coeficient of Agreement for Nominal Scales</article-title>
          ,
          <source>Educational and Psychological Measurement</source>
          <volume>20</volume>
          (
          <year>1960</year>
          )
          <fpage>37</fpage>
          -
          <lpage>46</lpage>
          . doi:h t t p s : / / d o i .
          <source>o r g / 1 0 . 1 1</source>
          <volume>7 7 / 0 0 1 3 1 6 4 4 6 0 0 2 0 0 0 1 0 4 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M.</given-names>
            <surname>Janier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Lawrence</surname>
          </string-name>
          , C. Reed,
          <source>OVA+: an Argument Analysis Interface, Computational Models of Argument</source>
          (
          <year>2014</year>
          )
          <fpage>463</fpage>
          -
          <lpage>464</lpage>
          . URL: http://arg-tech.org/people/chris/publications/
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>