<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Study of Conceptual Bases of Software Functioning for the Representation of Deliberative Argumentation</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>St Petersburg State University</institution>
          ,
          <addr-line>7/9 Universitetskaya emb., 199034 St Petersburg</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2020</year>
      </pub-date>
      <fpage>0000</fpage>
      <lpage>0003</lpage>
      <abstract>
        <p>Deliberative reasoning is widely used in various fields of human activity. In the modern information society, the use of methods of deliberative argumentation is associated with the development and use of appropriate application software, which is intended for visualization and modeling of intellectual activity to solve various types of practical problems, as well as argumentation. At the same time, various software designed for modeling and representation of argumentation explicitly or implicitly contains its conceptual grounds for argumentation. In this study, based on the identification of software intended for the simulation of deliberative reasoning, analysis of its purpose and main functions, the conceptual foundations of their functioning are determined, which is the initial stage for the formulation of a body of criteria for evaluating this software and its subsequent classification. The authors propose two preliminary independent classifications based on conceptual grounds, which are significant characteristics for the classification of the corresponding software.</p>
      </abstract>
      <kwd-group>
        <kwd>Deliberative Reasoning</kwd>
        <kwd>Conceptual Bases</kwd>
        <kwd>Software</kwd>
        <kwd>Modeling</kwd>
        <kwd>Representation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>Deliberative argumentation is a kind of practical argumentation about actions. In
contrast to the theoretical argumentation with its objective to prove the truthfulness of a
proposition, the deliberative argumentation focuses on the rationale for what has to be
done or what we should do in a given situation. Along with the actions which the
deliberative argumentation aims at justifying and the propositions by means of which it
intends to justify those actions, it also involves norms, goals and values as the atomic
elements of the deliberative arguments. For that reason, it finds wide application in
various fields of human social practices, including political, legal, moral, religious and
everyday life.</p>
      <p>In the context of creating of the software for the representation of deliberative
argumentation, its main difference from the theoretical argumentation consists in two
aspects, technical and conceptual. The technical difference amounts to the fact that for
the modelling of the deliberative argumentation, there is a need for a wider expressive
power of the formal languages and ontologies in order to take into account the
intentions, goals, norms, values, etc., in addition to the descriptive propositions conveying</p>
      <p>Copyright ©2020 for this paper by its authors.</p>
      <p>
        Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
facts, which exhaust the atomic elements of the theoretical arguments [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. The
conceptual difference lies in the distinction between the argumentation as a competition
among justifications, which belongs to the field of modeling reasoning in general,
manipulating knowledge bases and information that make up the cognitive direction in the
computer science where information reliability and truthfulness if the key criteria for
wining that competition, and the decision- making as a strategy and tactics of behavior
belonging to the field of psychology and management where the key criteria for wining
is effectiveness in terms of means-ends or resources-ends relations.
      </p>
      <p>The selection and classification of the software based on these aspects is the final
goal of our study. At its initial pilot stage, we conducted a preliminary selection of the
software for the representation of the argumentation and reasoning in general by
separating it from the software used for manipulating knowledge bases and information. At
the second stage, we set the task of selecting and classifying the software for modeling
the deliberative argumentation. The outcomes of those two stages of our study will be
incorporated into the final classification of the software. Here we talk about general
approaches in the second stage.</p>
      <p>
        In the contemporary information society, modeling of the deliberative argumentation
is connected to the development and use of the special software applications, which is
designed to visualize the intellectual activity in solving the practical questions of
various kinds by means of reasoning and argumentation. This means that most software
applications abstract from the above conceptual distinctions and embrace the reasoning
and the decision-making altogether. Decision-making support methods for the
intellectual activity with the elements of deliberation using the information and communication
technologies find their application in various fields of the human activity: medicine
[
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], public policy and e-democracy [
        <xref ref-type="bibr" rid="ref13 ref5">5, 13</xref>
        ], scientific or academic argumentation
(including technical, medical and humanities) [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ], education, business and other fields
[
        <xref ref-type="bibr" rid="ref24 ref27">24, 27</xref>
        ].
      </p>
      <p>
        The contemporary research varies in considering different aspects of the software
application. Analysts and practitioners have developed the techniques for the
application of such software, which is reflected in numerous publications, for example [
        <xref ref-type="bibr" rid="ref10 ref12 ref13 ref24 ref27 ref9">9, 10,
12, 13, 24, 27</xref>
        ].
      </p>
      <p>
        As a rule, when analysts focus on the software products for determining the
possibilities of their application for modelling argumentation, they seldom correlate them
with the concepts of argumentation. At the same time, the rationale for analysts to look
for the software of that kind refers mostly to training the critical thinking and other soft
skills, and to a lesser extent to its use in business analytics, law, management of
complex social systems – to the key areas of practical application of the software in
question, where conducting evidence-based reasoning based on knowledge and
argumentation is at stake. Although some analysts examine the aspects of using such software in
the applied fields [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], the theoretical foundations implemented in the software for
modeling argumentation, deliberative reasoning and mind mapping are considered
superficially. In addition, the available results of those studies cover restricted amount of the
existing software, and no reasons are given for the selection they make. Most of the
existing classifications are based on the experience of applying the software rather than
on the examining the conceptual foundations of the software, and they mainly consider
the methodology of their application in various educational practices. Therefore, in the
contemporary studies, there is a challenge for the theoretically substantiated body of
the software evaluation criteria with respect to modeling arguments, deliberative
reasoning in a broad sense and for the mind mapping.
      </p>
      <p>
        Many Russian authors focus on creating the software for modeling the plausible
reasoning rather than on modeling reasoning in general, which seems to be a more
promising approach to solving the problem of the adequacy of modeling of the software.
This problem amounts to selecting the software appropriate to the objectives of its
application which implies taking into account the conceptual foundations of the software
at issue, contrary to selecting it on the basis of the software availability and IT-brand
fashion. An influential contribution to the development of the concept of argumentation
as plausible reasoning was made by the school of Viktor Finn [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], whose research group
created several algorithms for the automated solution of problems through
non-deductive, probable and abductive, reasoning [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Russian researchers are developing
concepts of plausible reasoning as applied to the intellectual and expert systems, including
the deliberative systems (decision making), as well as databases, within the framework
of which the argumentative reasoning functions as a special competence of highly
intelligent agents [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ]. They develop the bodies of the methods for discourse analysis of
the Internet discussions on socially significant topics, and propose to study them in one
of the aspects of the deliberative reasoning - from the perspective of agreement or
disagreement with values, norms, etc. [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ].
      </p>
      <p>The software designed to model and represent the argumentation is based on diverse
conceptual foundations of the argumentation which are visible in the software explicitly
or implicitly. Our study aims at determining the conceptual foundations of how such
software functions, which is the initial stage for formulating of a body of the evaluation
criteria for this software and its subsequent comprehensive classification. We intend to
determine those foundations by means of identifying the software designed to model
deliberative argumentation, analyzing its purpose and basic functions.</p>
      <p>In the development of such a classification we confine ourselves to the following
objectives:
─ the development of proposals for solving the problem of the theoretical gap between
the concepts of argumentation formulated as a result of its research studies and the
concepts explicitly or implicitly implied in a number of the software systems and the
software applications designed for the modeling and representation of
argumentation. Most of those systems and applications have descriptive character: they are
limited to the visualization of argumentative dialogs (disputes) and offer no
mechanisms for their solutions which highlights the relevance of establishing those
particular aspects of deliberation the definite software visualizes;
─ to draw the conceptual borderlines between the three approaches: the modeling of
argumentation as kind of intellectual cognitive activity aimed at identifying the
consistency or soundness of the views of the parties on the issue at stake; visualization
of the critical and deliberative reasoning by applied methods of the mind mapping,
and the mind mapping as the sets of graphical tools of representing information,
including reasoning, using associative diagrams;
─ to provide the academic, research and educational community with a tool for the
effective selection of the software systems and applications for use in research and
educational activities where the deliberative argumentation plays significant role;
─ to suggest the methodological support by formulating recommendations for the
creation of the domestic software systems and applications for modeling arguments
which would respond to the local needs in an adequate linguistic and dialectical
setting.
1</p>
      <p>
        Identification of the conceptual foundations of the software
for deliberative reasoning and argumentation
In the earlier pilot study, we selected the software designed for modeling, analysis and
teaching of argumentation and critical thinking skills [
        <xref ref-type="bibr" rid="ref17 ref18">17, 18</xref>
        ]. We considered the
purpose and the aspects of the application of the software for formulating meaningful
criteria for distributing the software into the following categories: the type of reasoning or
interactions modelled by the software; the kind of interactions – monological, or
agentless, dialogical with two or more agents; whether the software takes into account
the logical correctness of reasoning; whether it discriminates between the descriptive
and non-descriptive information; whether it has the functions of heuristic search of
solutions. With respect to the software application for solving practical tasks, taking those
aspects into account allows to separate the software into the following three groups:
modeling arguments in a broad sense; analysis and visualization of the processes of
generating and evaluating the argumentative discourse - this software can be
effectively used for teaching academic writing skills; solving practical tasks based on the
use of argumentation, for example, in the field of legal or moral argumentation.
Accordingly, the selected software was divided into several main categories in accordance
with its following purpose:
─ modeling of argumentation;
─ visualization of critical and deliberative reasoning;
─ mind mapping.
      </p>
      <p>
        After that we grouped the selected software systems and applications into several
categories according to the following methods:
─ we have used some of the software for a long time in the educational process and
determined some of its conceptual foundations;
─ some of the software systems and applications are widely used and their
characteristics are described in the educational and research publications;
─ developers of some software give their own descriptions of its functional properties.
For example, Carneades and Rationale software applications that we widely use in the
educational process have clear descriptive objective of representing argumentation in a
definite way, although many researchers view those applications as based on the
argumentation model of S. Toulmin, which has normative character with respect to
constructing arguments and their performance in the dialogs. Moreover, the concepts of
argumentation implemented in Carneades and Rationale are different from each other
albeit both are descriptive. Carneades software supplements Toulmin’s argumentation
model with some types of defeasible reasoning which the original model lacks.
Rationale adds to the model two kinds of evaluations of the arguments, the quantitative
estimates of the argument strength (poor, good, undefined) and the qualitative estimates
of the arguments based on the establishment of their meaningful sources (data,
statistics, expert opinions, etc.), as well as the templates for generating arguments in the text.
Recently, Rationale developers have proposed a new application for modeling
decision-making b’cisive, which is based on the concept of deliberative protocol [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
b’cisive developers position it for visualization of arguments in the deliberations and
decision making altogether, by which with they explicitly avoid drawing distinctions
between them.
      </p>
      <p>Some software programs are no longer supported or developed. These include, for
example, Araucaria software, which we classified for use for the purpose of modeling
arguments and deliberative reasoning, but in 2006 the developers stopped supporting it
and switched to developing their new web-based software OVA
(http://ova.argtech.org), designed for argumentation mapping for the analysis and modeling of
argumentation in the text. Contrary to Araucaria, OVA discriminates the sorts of arguments
in terms of their kinds, strength and functions in the dialog. The discrimination is
realized by the sets of argumentation schemes available at user’s choice (for example,
Walton presumptive inference, Rutgers SALTS, Cornell, Dundee illocutionary, Second
order illocutionary, Basic conflict, Extended Conflict, Deductive inference). Those sets
have been proposed by the research groups and can be used for mapping the
argumentation in texts originating from various subject areas. In terms of logic, the sets of the
argumentative schemes include deductive and a variety of non-deductive arguments.
Users may choose to identify the arguments they are mapping by other criteria like
speech acts, dialectical role, rhetorical shape, etc.</p>
      <p>From what was said above we conclude that the software designed for modelling
argumentation sensitively varies in the degrees and the ways of implementation of the
concepts of argumentation on which it is based, including the cases when the software
products differently implement the same concepts. For example, Rationale software
users need no prerequisite knowledge about the analysis or structure of the argument is
for starting to work with it. The visualization of argumentation by Rationale is close to
its intuitive mapping and reflect the basics of the theories which are normally given in
standard textbooks on argumentation. Rationale supports generating argumentative
texts in the vein of design thinking, as well as the multivariate assessment of the
effectiveness of argumentation. OVA does not support the latter two options. In contrast, in
OVA software, the users have to choose the argumentation schemes themselves, and
they have to be aware of the dialectical concept of argumentation implemented in OVA
for being able to construct their maps with OVA.</p>
      <p>
        As a part of our study, we single out into a separate group the software systems and
platforms used to support the deliberative democracy such DemocracyOS, Democracy
2.1, Loomio, OpaVote, Delib, Decidim [
        <xref ref-type="bibr" rid="ref11 ref2">2, 11</xref>
        ], which promote deliberations as a
necessary tool for generating and shaping public opinion as well as for formulating of the
political agenda and observing the controversial issues in the political decision-making.
Those systems are the social online platforms or forums, they provide the digital and
technical tools for polls, opinion exchange, debates and discussions, statistical analysis
and visual representation of its results.
      </p>
      <p>
        They are designed for supporting decision-making for state and municipal
management and are often employed for similar governmental purposes. In those systems the
decisive analytical function is assigned to humans who, however, may choose rely on
the systems’ AI analytical potential, too. Enhancing that potential of AI in the vein of
substituting humans at the analytical chair of those software platforms is a relevant
subject of concern for many apologists of wider implementation of AI and the natural
language processing (NLP) technologies in the deliberative democracy for analyzing
public discourse and decision making [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. The platforms for the deliberative
democracy pay no special attention to the kinds and sorts of argumentation performed there,
and it is impossible to identify the conceptual grounds of argumentation implemented
in them.
      </p>
      <p>We propose two preliminary classifications of the software and give the theoretical
descriptions of the aspects of argumentation for creating such software. At the first step,
we divide the software into two groups, depending on whether they are descriptive
formalisms, platforms, protocols, or ontologies, that visualize the argumentative reasoning
of their users, or normative systems that simulate what conclusions should be drawn,
what assumptions are recommended to accept or what decisions have to be taken given
the rationale visualized by the software (Table 1).</p>
      <p>Note that we call the platforms, protocols, or ontologies altogether formalisms and
place them into one group with respect to their functional capability to represent
discussions where argumentation is used as one of the tools, which means that for this
classification we abstract from their diverse functional capabilities in how those
representations are realized in each of them. At the second step, we identify two groups of
theoretical concepts laid down by their developers in the corresponding software,
depending on whether it supports the visualization of the modifiable (defeasible)
reasoning or not (Table 2).</p>
      <p>Note that the criteria we have chosen for the two groupings give the independent
groups despite the fact that they overlap, and we see it impractical to create a
generalized classification, since we propose to use those criteria and groupings for different
practical purpose. The descriptive vs normative groups are meant for guiding the users
in finding an appropriate software for their tasks regarding reasoning in general; the
modifiable vs non-modifiable groups are meant for doing so with respect to
argumentation of the definite kinds. Perhaps, the only purpose those two groups of the criteria
are both equally suitable is their further development, and generally the development
of the software and its classification, since those criteria and groupings make explicit
some of the software essential properties.
The relevance of the software classification which we are developing with respect to
its conceptual foundations is that it will enable the users to choose the software
regarding their practical goals in a more rational way with respect to solving the tasks of
argument analysis and its digital mapping. Another aspect of the relevance is that the
classification will respond to the new educational challenges in the teaching of skills of
practical argumentation and deliberative reasoning in various fields of human activity.
For example, at St. Petersburg State University, the Digital Transformations of
Argumentation in Science module is included in the general compulsory course ‘Digital
Culture’. As part of the module, master students study argumentation digital mapping using
Rationale and OVA software and apply it to solving the tasks with the help of practical
argumentation. In this vein, they develop applied competencies in digital argumentation
mapping, deliberative reasoning, defense and criticism of arguments.</p>
      <p>Normative software applications
modelling of the defeasible reasoning (DeLP,
DefLog)
modelling argumentation with AI systems
(ArgTools, Dung-O-Matic)
Modelling of probable inferences with
modifiable assumptions PITA
Modelling of cognitive reasoning for rational
agents OSCAR
Modelling argumentation in dialog by a
twoagent game Convince Me
Modifiable (non-monotonous)
modelling of defeasible reasoning
(defeasible reasoning) (DeLP, DefLog)
Modelling of argumentation with the
AIsystems (ArgTools, Dung-O-Matic)
Modelling of probable inferences with
modifiable assumptions PITA
Modelling of cognitive reasoning for rational
agents OSCAR
Protocol for argumentation about actions
ATR
System of plausible argumentation</p>
      <p>Note that along with the software solutions that we have studied and described here,
there are other software systems and platforms that implement mechanisms of
deliberation, argumentation, and support for intellectual activity. However, they bear
theoretical character and are described only in the research papers (for example, ProGraph,
ConArg2), or there are no links either to the websites of their developers or to the
software itself. Therefore, those software products are most likely ongoing development or
test versions, which lack information or functional performance for determining their
capabilities and describing their key properties, including the conceptual basis for their
construction.</p>
    </sec>
    <sec id="sec-2">
      <title>Conclusion</title>
      <p>At the current stage of the study, we have identified two grounds for classifying the
software designed for analyzing reasoning - on the basis of the descriptive or normative
approach to modelling deliberations and with respect the modifiable or non-modifiable
character of argumentation at issue, which form the foundation of the classification of
the software we are intended to develop. Those grounds are independent of each other
and are both essential properties of the software we focus on.</p>
      <p>In the future, we are going to develop a comprehensive classification of the software
designed for modeling and representing argumentation, in which we will identify its
key conceptual foundations that will enable us to form large groups of the software. We
will consider the differences in the implementation of these concepts in the software as
its functional properties and separate them into a special characteristic, which will make
users’ selection of the software more justified with respect to the tasks they wish to
solve by means of it.</p>
      <p>For the practical approbation of the results obtained we will conduct an applied
study. As part of that study, we plan to select an example of a text with a description of
a problem solved by means of practical argumentation. We will model that
argumentation first with the help of OVA or OVA+ software, which will be a sample for the
further modelling by means of two other software applications for analyzing
argumentation. Those applications will be selected on the basis of the alternative conceptual
grounds compared to OVA. In the selection of those software applications we will take
into account the demand in the research community based on the results of the
quantitative analysis of the research publications and accessibility both in terms of licensing
and availability for the users. This comparative modelling will enable us to clarify the
results obtained by our classification and to verify the recommendations for further
development of the software we are intended to formulate as another result of our study.
Acknowledgement. The reported study is funded by RFBR according to the research project
#20-011-00485-a.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Anshakov</surname>
            ,
            <given-names>O.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Skvortsov</surname>
            ,
            <given-names>D.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Finn</surname>
            ,
            <given-names>V.K.</given-names>
          </string-name>
          :
          <article-title>On the axiomatizability of multivalued logics associated with the formalization of plausible reasoning</article-title>
          .
          <source>In: Logicheskie issledovaniya</source>
          , pp.
          <fpage>222</fpage>
          -
          <lpage>247</lpage>
          . Nauka, Moscow (
          <year>1993</year>
          ). [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Aragón</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kaltenbrunner</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Calleja-López</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pereira</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Monterde</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Barandiaran</surname>
            ,
            <given-names>X.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gómez</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>Deliberative Platform Design: The Case Study of the Online Discussions in Decidim Barcelona (</article-title>
          <year>2017</year>
          ). ArXiv:
          <volume>1707</volume>
          .
          <fpage>06526</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Arshinskiy</surname>
            ,
            <given-names>L.V.</given-names>
          </string-name>
          :
          <article-title>Features of the inference engine of the plausible reasoning modeling system "Heraclitus"</article-title>
          .
          <source>Information and mathematical technologies in science and management. 2</source>
          ,
          <fpage>18</fpage>
          -
          <lpage>29</lpage>
          (
          <year>2016</year>
          ). [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Atkinson</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bench-Capon</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Practical reasoning as presumptive argumentation using action based alternating transition systems</article-title>
          .
          <source>Artificial Intelligence</source>
          .
          <volume>171</volume>
          ,
          <fpage>855</fpage>
          -
          <lpage>874</lpage>
          (
          <year>2007</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Davies</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chandler</surname>
          </string-name>
          , R.:
          <article-title>Online deliberation design: Choices, criteria, and evidence</article-title>
          . In: Nabatchi ,
          <string-name>
            <given-names>T.</given-names>
            ,
            <surname>Weiksner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Gastil</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            ,
            <surname>Leighninger</surname>
          </string-name>
          , M. (eds.).
          <article-title>Democracy in motion: Evaluating the practice and impact of deliberative civic engagement</article-title>
          ,
          <fpage>103</fpage>
          -
          <lpage>131</lpage>
          . Oxford univ. press, Oxford (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Dung</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming, and n-person games</article-title>
          .
          <source>Artificial Intelligence</source>
          .
          <volume>77</volume>
          ,
          <fpage>321</fpage>
          -
          <lpage>357</lpage>
          (
          <year>1995</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Easterday</surname>
            ,
            <given-names>M. W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kanarek</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harrell</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Design requirements of argument mapping software for teaching deliberation</article-title>
          . In: Davies,
          <string-name>
            <given-names>T.</given-names>
            ,
            <surname>Gangadharan</surname>
          </string-name>
          ,
          <string-name>
            <surname>S. P</surname>
          </string-name>
          . (Eds.).
          <source>Online deliberation: Design</source>
          , research, and practice, pp.
          <fpage>317</fpage>
          -
          <lpage>323</lpage>
          (
          <year>2009</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Finn</surname>
            ,
            <given-names>V.K.</given-names>
          </string-name>
          :
          <article-title>Plausible reasoning in intelligent systems such as DSM</article-title>
          . In:
          <article-title>Avtomaticheskoe porozhdenie gipotez v intellektual'nykh sistemakh</article-title>
          , pp.
          <fpage>10</fpage>
          -
          <lpage>48</lpage>
          . VINITI RAN, Moscow (
          <year>2009</year>
          ). [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Gelder</surname>
          </string-name>
          , T. van:
          <article-title>Using argument mapping to improve critical thinking skills</article-title>
          . In: Davies,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Barnett</surname>
          </string-name>
          ,
          <string-name>
            <surname>R</surname>
          </string-name>
          . (eds).
          <source>The Palgrave Handbook Critical Thinking in Higher Education</source>
          , pp.
          <fpage>183</fpage>
          -
          <lpage>192</lpage>
          . Palgrave Macmillan,
          <string-name>
            <surname>US</surname>
          </string-name>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Gordon</surname>
            ,
            <given-names>T.F.</given-names>
          </string-name>
          :
          <article-title>An Overview of the Carneades Argumentation Support System</article-title>
          . In: Tindale,
          <string-name>
            <given-names>C.W.</given-names>
            ,
            <surname>Reed</surname>
          </string-name>
          , C. (eds.).
          <source>Dialectics, Dialogue and Argumentation. An Examination of Douglas Waltons Theories of Reasoning</source>
          , pp.
          <fpage>145</fpage>
          -
          <lpage>156</lpage>
          (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Grassle</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>Digital tools for participatory democracy</article-title>
          .
          <source>GovLab Blog March</source>
          ,
          <volume>5</volume>
          (
          <year>2015</year>
          ). http://thegovlab.org
          <article-title>/digital-tools-for-participatory-democracy.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Janier</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lawrence</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reed</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>"OVA+: an Argument Analysis Interface"</article-title>
          . In: Parsons,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Oren</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            ,
            <surname>Reed</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Cerutti</surname>
          </string-name>
          ,
          <string-name>
            <surname>F</surname>
          </string-name>
          . (eds).
          <source>Proceedings of the Fifth International Conference on Computational Models of Argument (COMMA</source>
          <year>2014</year>
          ), pp.
          <fpage>463</fpage>
          -
          <lpage>464</lpage>
          . IOS Press, Pitlochry (
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Iandoli</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Supporting Collaborative Deliberation Using a Large-Scale Argumentation System: The MIT Collaboratorium</article-title>
          .
          <source>In: Proceedings of the Eleventh Directions and Implications of Advanced Computing Symposium and the Third International Conference on Online Deliberation (DIAC</source>
          <year>2008</year>
          /OD 2008), pp.
          <fpage>5</fpage>
          -
          <lpage>12</lpage>
          . Berkeley, California (
          <year>2008</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Krauthoff</surname>
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Meter</surname>
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Baurmann</surname>
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Betz</surname>
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mauve</surname>
            <given-names>M.:</given-names>
          </string-name>
          <article-title>D-BAS-A Dialog-Based Online Argumentation System</article-title>
          .
          <source>In: Computational Models of Argument: Proceedings of COMMA</source>
          <year>2018</year>
          , pp.
          <fpage>325</fpage>
          -
          <lpage>336</lpage>
          . (
          <year>2018</year>
          ). DOI:
          <volume>10</volume>
          .3233/978-1-
          <fpage>61499</fpage>
          -906-5-325.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Kunz</surname>
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rittel</surname>
            <given-names>H.</given-names>
          </string-name>
          :
          <article-title>Issues as elements of information systems</article-title>
          . Working Paper No.
          <volume>131</volume>
          , Institute of Urban and Regional Development, Univ. of California, Berkeley, Calif. (
          <year>1970</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Lisanyuk</surname>
            ,
            <given-names>E.N.</given-names>
          </string-name>
          <article-title>Argumentation and persuasion</article-title>
          . Nauka, St.
          <source>Peterburg</source>
          (
          <year>2015</year>
          ). [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Lisanyuk</surname>
            ,
            <given-names>E.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prokudin</surname>
            ,
            <given-names>D.E.</given-names>
          </string-name>
          <article-title>Models of teaching argumentation using software</article-title>
          .
          <source>Logikofilosofskie shtudii</source>
          .
          <volume>13</volume>
          (
          <issue>2</issue>
          ),
          <fpage>217</fpage>
          -
          <lpage>218</lpage>
          (
          <year>2016</year>
          ). [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Lisanyuk</surname>
            ,
            <given-names>E.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prokudin</surname>
            ,
            <given-names>D.E.</given-names>
          </string-name>
          :
          <article-title>Classifying software for argumentation modelling</article-title>
          .
          <source>In: Proceedings of International Conference «Internet and Modern Society» (IMS-2017)</source>
          , St. Petersburg, June,
          <fpage>21</fpage>
          -
          <lpage>23</lpage>
          ,
          <year>2017</year>
          . ITMO University, pp.
          <fpage>11</fpage>
          -
          <lpage>13</lpage>
          (
          <year>2017</year>
          ). http://ojs.ifmo.ru/index.php/IMS/article/view/516 [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Mahadevan</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Dubey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Balasubramanian</surname>
          </string-name>
          , Karsai G.:
          <article-title>Deliberative, search-based mitigation strategies for model-based software health management</article-title>
          .
          <source>In: Innovations in Systems and Software Engineering</source>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>26</lpage>
          . Springer, London (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Mejia</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>How can digital tools support deliberation? Join the conversation</article-title>
          ! In: Medium.
          <year>2020</year>
          . April,
          <volume>10</volume>
          . (
          <year>2020</year>
          ). https://medium.com/participo/how
          <article-title>-can-digital-tools-support-deliberation-join-the-conversation-2f0622bda62a.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <given-names>Misnikov</given-names>
            <surname>Yu</surname>
          </string-name>
          .G.,
          <string-name>
            <surname>Filatova</surname>
            <given-names>O.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chugunov</surname>
            <given-names>A.V.</given-names>
          </string-name>
          :
          <article-title>Electronic interaction between the authorities and society: areas and methods of researches</article-title>
          .
          <source>St. Petersburg State Polytechnical University Journal. Humanities and Social Sciences</source>
          .
          <volume>1</volume>
          (
          <issue>239</issue>
          ),
          <fpage>52</fpage>
          -
          <lpage>60</lpage>
          (
          <year>2016</year>
          ). DOI:
          <volume>10</volume>
          .5862/JHSS.239.6 [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22. Online Deliberation: Design, Research, and
          <string-name>
            <surname>Practice</surname>
          </string-name>
          . Davies T.,
          <string-name>
            <surname>Gangadharan</surname>
            <given-names>S.P</given-names>
          </string-name>
          . (eds.).
          <source>CSLI Publications</source>
          , Stanford, (
          <year>2009</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Pollock</surname>
            ,
            <given-names>J.: Cognitive</given-names>
          </string-name>
          <string-name>
            <surname>Carpentry</surname>
          </string-name>
          .
          <article-title>A Blueprint for How to Build a Person</article-title>
          . MIT Press, Cambridge (
          <year>1995</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Schneider</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Voigt</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Betz</surname>
          </string-name>
          , G.:
          <article-title>Argunet: A software tool for collaborative argumentation analysis and research</article-title>
          .
          <source>In: Paper presented at the 7th Workshop on Computational Models of Natural Argument (CMNA</source>
          <year>2007</year>
          ). Hyderabad,
          <string-name>
            <surname>India</surname>
          </string-name>
          (
          <year>2007</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Vagin</surname>
            ,
            <given-names>V.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Golovina</surname>
            ,
            <given-names>E.Yu.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zagoryanskaya</surname>
            ,
            <given-names>A.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fomina</surname>
            ,
            <given-names>M.V.</given-names>
          </string-name>
          :
          <article-title>Reliable and plausible conclusion in intelligent systems</article-title>
          . Vagin,
          <string-name>
            <given-names>V.N.</given-names>
            ,
            <surname>Pospelov</surname>
          </string-name>
          ,
          <string-name>
            <surname>D.A.</surname>
          </string-name>
          . (eds). Fizmatlit, Moscow (
          <year>2004</year>
          ). [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Vagin</surname>
            ,
            <given-names>V.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Morosin</surname>
            ,
            <given-names>O.N.</given-names>
          </string-name>
          :
          <article-title>Software implementation of an argumentation system with justification degrees</article-title>
          .
          <source>Software &amp; Systems</source>
          <volume>1</volume>
          ,
          <fpage>21</fpage>
          -
          <lpage>27</lpage>
          (
          <year>2015</year>
          ). DOI:
          <volume>10</volume>
          .15827/
          <fpage>0236</fpage>
          -
          <lpage>235X</lpage>
          .
          <fpage>109</fpage>
          .
          <fpage>021</fpage>
          -
          <lpage>027</lpage>
          [In Russian].
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <article-title>Visualizing argumentation: software tools for collaborative and educational sense-making.</article-title>
          <string-name>
            <surname>Kirschner</surname>
            ,
            <given-names>P.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Buckingham</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Simon</surname>
            ,
            <given-names>J</given-names>
          </string-name>
          . (eds).
          <source>Computer supported cooperative work</source>
          . Springer-Verlag, London; New York (
          <year>2003</year>
          ). DOI:
          <volume>10</volume>
          .1007/978-1-
          <fpage>4471</fpage>
          -0037-9.
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Zagorulko</surname>
            ,
            <given-names>Yu.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Garanina</surname>
            ,
            <given-names>N.O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Borovikova</surname>
            ,
            <given-names>O.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Domanov</surname>
            ,
            <given-names>O.A.</given-names>
          </string-name>
          :
          <article-title>Argumentation modeling in popular science discourse using ontologies</article-title>
          .
          <source>Ontology of Designing. 9</source>
          .
          <issue>4</issue>
          (
          <issue>34</issue>
          ),
          <fpage>496</fpage>
          -
          <lpage>509</lpage>
          (
          <year>2019</year>
          ). DOI:
          <volume>10</volume>
          .18287/
          <fpage>2223</fpage>
          -9537-2019-9-4-
          <fpage>496</fpage>
          -509 [In Russian].
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>