<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>On Structural Analysis of Non-Ground Answer-Set Programs</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Institut für Informationssysteme</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Arbeitsbereich Wissensbasierte Systeme</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Engineering, Faculty of Engineering, Marmara University</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Technische Universität Wien</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2015</year>
      </pub-date>
      <history>
        <date date-type="accepted">
          <day>5</day>
          <month>6</month>
          <year>2015</year>
        </date>
      </history>
      <abstract>
        <p>The development of answer-set programs often involves domain experts without a background in logic programming. In such situations, it would be beneficial to translate programs into a form which is easier to understand and closer to natural language. Since the structure of a program determines to a great extent how a program should be explained in a clear and comprehensible way, as a first step towards a natural-language representation of answer-set programs, in this paper, we introduce methods for analysing the structure of disjunctive non-ground answer-set programs. In particular, as most programs follow the generate-define-test paradigm, we introduce formal definitions to characterise the respective generate, define, and test parts of a program. Thereby, we define the non-deterministic core of a program, effectively determining the program's active solution-space generators, following ideas of the weakly perfect model semantics as introduced by Przymusinska and Przymusinski, and we prove that our definitions fulfil desirable properties. Moreover, we also provide an implementation of a tool, using a metaprogramming approach, which classifies the rules of a given program according to our definitions. Finally, we propose an algorithm that, based on our generate-define-test classification, computes the order in which the rules of a program should be explained when translated into natural language.</p>
      </abstract>
      <kwd-group>
        <kwd>answer-set programming</kwd>
        <kwd>generate-define-test paradigm</kwd>
        <kwd>program analysis</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>BENJAMIN KIESL,1 PETER SCHÜLLER,2 and HANS TOMPITS1</p>
    </sec>
    <sec id="sec-2">
      <title>1 Introduction</title>
      <p>Answer-set programming (ASP) has proven to be a viable tool for knowledge
representation, reasoning, and solving complex search problems. In many cases, the
process of developing answer-set programs involves domain experts, i.e., persons
with a detailed knowledge of some certain field of application but often without
This work has been supported by the Austrian Science Fund (FWF) under project W1255-N23
and by the Scientific and Technological Research Council of Turkey (TUBITAK) under grant
114E777.
a background in ASP. Such experts provide their know-how of a specific topic
to software developers who are then concerned with encoding this expertise into
answer-set programs. In such a context, arguably the software-engineering process
for answer-set programs involving non-ASP experts could be eased if
representations of the programs and their output in a form which is closer to natural language
would be available.</p>
      <p>
        Since the structure of a program determines to a great extent how a program
should be explained in a clear and understandable way, as a first step towards
realising natural-language representations of answer-set programs, we introduce in
this paper methods for analysing the structure of disjunctive non-ground
answerset programs. We note that work addressing representations in the other direction,
i.e., from natural language to ASP, has been discussed in the literature, e.g., by
        <xref ref-type="bibr" rid="ref13">Erdem and Yeniterzi (2009)</xref>
        , who formulated queries over biomedical ontologies
using a controlled natural language (CNL), which are then translated into ASP for
evaluation, and by
        <xref ref-type="bibr" rid="ref23">Schwitter (2012</xref>
        ; 2013) who specified various logical puzzles in
CNL and provided their solutions via translations into ASP.
      </p>
      <p>
        A key structural feature of most answer-set programs is that they adhere to
the so-called generate-define-test paradigm which was informally introduced by
        <xref ref-type="bibr" rid="ref17">Lifschitz (2002)</xref>
        as a refinement of the well-known guess-and-check paradigm
        <xref ref-type="bibr" rid="ref11 ref12 ref16">(Eiter
et al. 2000; Leone et al. 2006; Eiter et al. 2009)</xref>
        . According to this paradigm, a
program can be considered as a combination of three parts, respectively referred
to as the generate, define, and test part. The generate part non-deterministically
generates a set of candidate solutions from which the test part eliminates those
candidates which are not the desired solutions of the specified problem. In order
to do so, both the generate and the test part may use concepts which are specified
in the define part. Although generate-define-test is, as indicated by
        <xref ref-type="bibr" rid="ref9">Denecker et
al. (2012)</xref>
        , the dominating methodology of ASP, formal criteria to characterise the
generate, define, and test parts of an answer-set program have not been introduced
so far to the best of our knowledge. We introduce such formal conditions and argue
for their adequacy by showing that they possess intuitively desirable properties.
      </p>
      <p>
        For a natural definition of the non-deterministic generate part of a program, it
is necessary to identify cases in which default negation leads to non-deterministic
behaviour. From a result of
        <xref ref-type="bibr" rid="ref27">You and Yuan (1994)</xref>
        , it immediately follows that a
non-disjunctive program has multiple answer sets only if its (ground) dependency
graph contains cycles with an even number of negative edges. Furthermore, odd
negative cycles can eliminate certain answer-set candidates. But, as demonstrated
by
        <xref ref-type="bibr" rid="ref22">Przymusinska and Przymusinski (1990)</xref>
        , negative cycles do not always become
active which is one reason why the existence of even negative cycles is not
sufficient for the creation of multiple answer sets and why odd negative cycles do not
necessarily eliminate answer-set candidates.
      </p>
      <p>
        Based on ideas underlying the procedure for the computation of the
weaklyperfect model of a logic program
        <xref ref-type="bibr" rid="ref22">(Przymusinska and Przymusinski 1990)</xref>
        , we define
the non-deterministic core of a program which is obtained by removing certain
rules and literals, thereby eliminating negative cycles that cannot become active.
Given this, we then define the generate part of a program as those rules which have
instances that are involved in even negative cycles within the dependency graph
of the non-deterministic core, together with disjunctive rules. From the remaining
rules, those which are constraints or which have instances that are involved within
odd negative cycles of the non-deterministic core form the test part of a program,
while all other rules comprise the define part.
      </p>
      <p>We also present an implementation of a tool, based on a metaprogramming
approach, which classifies the rules of a given program according to our definitions.
Finally, we propose an algorithm that makes use of our generate-define-test
classification to compute a specific order for explaining the rules of a program in natural
language.</p>
    </sec>
    <sec id="sec-3">
      <title>2 Preliminaries</title>
      <p>We deal with logic programs which are finite sets of rules of form</p>
      <p>L1 _ : : : _ Lk</p>
      <p>Lk+1; : : : ; Lm ; not Lm+1; : : : ; not Ln ;
(1)
where L1; : : : ; Ln are literals, i.e., atoms from some (implicitly given)
functionfree first-order language L, possibly preceded by the strong-negation symbol “ :”.
Literals which are possibly preceded by the default-negation symbol “ not” are
called default literals. Given a rule r as above, we define head(r ) = fL1; : : : ; Lk g,
body+(r ) = fLk+1; : : : ; Lm g, and body (r ) = fLm+1; : : : ; Ln g. The elements of
head(r ) are referred to as the head literals of r , while the elements of body+(r ) and
body (r ) are respectively referred to as the positive and negative body literals of r .
Moreover, we also define body(r ) = body+(r ) [ body (r ). We call a rule of form (1)
disjunctive if k &gt; 1, a fact if body(r ) = ;, and a constraint if head(r ) = ;. A
program is extended if it contains no disjunctive rules, normal if it contains neither
disjunctions nor strong negations, and positive if for every r 2 , body (r ) = ;.
Given a rule or program e, we denote by at(e) (resp., lit(e)) the set of atoms (resp.,
literals) in e.</p>
      <p>
        The grounding of a logic program relative to its Herbrand universe HU ( ) is
defined as usual and denoted by grd( ). Let M be a set of ground (variable-free)
literals. M satisfies a rule r if body+(r ) M and body (r )\M = ; only if head(r )\
M 6= ;. Furthermore, M is closed under a program if M satisfies all rules in
, and M is logically closed if it is consistent (i.e., for no literal L 2 M we have
fL; :Lg M) or contains all literals. The reduct, M, of relative to M
        <xref ref-type="bibr" rid="ref15">(Gelfond
and Lifschitz 1988)</xref>
        is given by fhead(r ) body+(r ) j r 2 grd( ) and body (r ) \
M = ;g. M is an answer set of if it is a minimal set that is both closed under
M and logically closed.
      </p>
      <p>
        The dependency graph, DG( ), of a program
        <xref ref-type="bibr" rid="ref26">(Ullman 1988)</xref>
        is the graph
which contains as vertices lit(grd( )) and a positive (resp., negative) edge (B ; H )
iff there is a rule r 2 grd( ) such that H 2 head(r ) and B 2 body+(r ) (resp.,
B 2 body (r )). A directed path with an even (resp., odd) number of negative edges
is called an even (resp., odd) negative path. Accordingly, a path without negative
edges is called a positive path. A cycle is a directed path whose start and end
vertex coincide and in which no other vertex is visited more than once. Let be
the function which maps every edge e in DG( ) to the set of all rules in that
cause e to be contained in DG( ). Then, we say that a rule r is involved in a path
P of DG( ) if P contains an edge e with r 2 (e).
      </p>
      <p>Given a program and literals A and B , A depends positively (resp., depends
negatively ) on B if DG( ) contains a positive (resp., negative) path from B to A.
Moreover, A depends on B if A depends positively or negatively on B .</p>
      <p>
        The rule graph, RG( ), of a program
        <xref ref-type="bibr" rid="ref10 ref5">(Dimopoulos and Torres 1996)</xref>
        is the
graph which contains as vertices the rules of and an edge (r1; r2) iff there is a
predicate p which occurs in the body of r1 and in the head of r2.
      </p>
      <p>
        A normal logic program is stratified if there exists a mapping f assigning
each predicate symbol a natural number such that, for every rule r 2 and every
predicate P1 and P2, (i) if P1 occurs in head(r ) and P2 occurs in body+(r ) then
f (P1) f (P2), and (ii) if P1 occurs in head(r ) and P2 occurs in body (r ) then
f (P1) &gt; f (P2). Let 0 be the program obtained from grd( ) by replacing every atom
with a corresponding propositional predicate symbol. Then, is locally stratified
if 0 is stratified
        <xref ref-type="bibr" rid="ref1">(Apt et al. 1988)</xref>
        .
      </p>
    </sec>
    <sec id="sec-4">
      <title>3 A Formal Generate-Define-Test Classification</title>
      <p>In this section, we introduce formal definitions of the generate, define, and test
parts of an answer-set program. Furthermore, we also sketch a metaprogram which
classifies the rules of a given answer-set program according to our definitions.</p>
      <p>As already noted in Section 1, the generate part should contain those rules which
cause non-determinism in the sense that they generate multiple answer-set
candidates. Disjunctive rules and negative cycles are obvious sources of such
nondeterminism, but, although even negative cycles are in the absence of disjunction
necessary for the creation of multiple answer sets, they are not sufficient. Consider,
for illustration, the program 1, consisting of the facts n(1), n(2), and succ(1; 2),
together with the following rule which, intuitively, chooses every second element of
the set n:
choose(Y )
n(X ); n(Y ); succ(X ; Y ); not choose(X ).
(2)
(3)
(4)
(5)
(6)
The grounding of
stances of (2):
choose(1)
choose(1)
Rules (4) and (5) are involved in an even negative cycle with literals choose(1) and
choose(2). Still, 1 has the unique answer set fn(1); n(2); succ(1; 2); choose(2)g.
Intuitively, the negative cycle does not become active in the sense that the body
atom succ(2; 1) in Rule (4) cannot be contained in an answer set of 1. Modern
grounders automatically eliminate a rule like (4) from the grounding of 1 but there
are more intricate cases where grounders are not able to remove such negative cycles.
1 contains, additionally to the facts, the following ground
inn(1); n(1); succ(1; 1); not choose(1);
n(2); n(1); succ(2; 1); not choose(2);
n(1); n(2); succ(1; 2); not choose(1);
n(2); n(2); succ(2; 2); not choose(2).</p>
      <p>
        The observation that in many cases negative cycles do not become active has
already been made by
        <xref ref-type="bibr" rid="ref22">Przymusinska and Przymusinski (1990)</xref>
        when they introduced
the weakly-perfect-model semantics and the class of weakly stratified programs. The
distinguishing feature of weakly stratified programs is that they have unique answer
sets even though they are a strict superclass of locally stratified programs.
      </p>
      <p>We next use ideas behind the weakly-perfect-model semantics to define the
nondeterministic core (short, nd-core) of a program. The nd-core is obtained by
removing certain rules and literals such that negative cycles which cannot become active
are eliminated. We begin with the definition of weakly minimal literals.</p>
      <sec id="sec-4-1">
        <title>Definition 1</title>
        <p>A literal L 2 lit(grd( )) is weakly minimal if it does not depend negatively on
other literals and if it does not depend on a literal in the head of a disjunctive rule.</p>
      </sec>
      <sec id="sec-4-2">
        <title>Definition 2</title>
        <p>The deterministic bottom-stratum, dbs( ), of is the set of all weakly-minimal
literals which do not occur in the heads of disjunctive rules. Furthermore, the
deterministic bottom-layer, dbl( ), of is the set fr 2 grd( ) j head(r ) dbs( )g.
The deterministic bottom-layer of a program can be seen as the positive
deterministic portion of a program since it is a non-disjunctive positive program which
clearly has a unique answer set.</p>
      </sec>
      <sec id="sec-4-3">
        <title>Example 1</title>
        <p>Let 2 consist of the set of facts fn(1); n(2); succ(1; 2)g and the following rules:
chooseEven; n(X ); n(Y ); succ(X ; Y ); not choose(X );
chooseAll; n(X );
not chooseAll;
not chooseEven.
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
2 has two answer sets: one in which choose(n) is true for every number n and one
in which choose(n) is only true for even n. The grounding of 2 contains the facts
together with Rules (9) and (10), and the following instances of (7) and (8):
chooseEven; n(1); n(1); succ(1; 1); not choose(1);
chooseEven; n(2); n(1); succ(2; 1); not choose(2);
chooseEven; n(1); n(2); succ(1; 2); not choose(1);
chooseEven; n(2); n(2); succ(2; 2); not choose(2);
chooseAll; n(1);
chooseAll; n(2).</p>
        <p>Then, the deterministic bottom-stratum of grd( 2) is dbs(grd( 2)) = fn(1); n(2);
succ(1; 1); succ(1; 2); succ(2; 1); succ(2; 2)g, and thus the deterministic bottom-layer
is dbl(grd( 2)) = fsucc(1; 2) ; n(2) ; n(1) g.
For defining the nd-core of a program we repeatedly apply the transformation
given in Definition 3 until a fixed-point is reached. The intuitive idea behind this
transformation is to first evaluate the deterministic positive portion of the program,
identified by its deterministic bottom-layer. The deterministic bottom-layer, as well
as further rules whose bodies can never become satisfied or which are redundant, are
then removed. Additionally, default literals that are satisfied by the unique answer
set of the deterministic bottom-layer are removed from the remaining rules. This
eliminates certain negative cycles that have a deterministic effect.</p>
      </sec>
      <sec id="sec-4-4">
        <title>Definition 3</title>
        <p>Let be a ground logic program and M the unique answer set of dbl( ). Then,
( ) is the program obtained from by
(i) removing all rules which contain a negative body literal N such that N 2 M,
a positive body literal P such that P 2 dbs( ) n M, or a head literal H with
H 2 M,
(ii) removing those body default literals such that P 2 M for positive body literals</p>
        <p>P or N 2 dbs( ) n M for negative body literals N , and
(iii) removing all non-facts whose heads appear as facts in the program.
The removal of non-facts whose heads appear as facts is important for programs
like 3 = fa not b; b not a; a g. Here, it allows to remove the first rule
although dbs( 3) = ; and so ( 3) has, in contrast to 3, no negative cycles.</p>
        <p>Based on the operator , we can now define the non-deterministic core of a
(possibly non-ground) program, intuitively obtained by partially evaluating its
deterministic portion (which may involve negative default literals) and transforming
the program accordingly.</p>
        <p>As a preparatory step, let 0( ) = and i+1( ) = ( i ( )), for i 2 N+.
Clearly, the application of to a ground logic program removes from all the
literals of dbs( ) and does not add any literals. Hence, there is an i such that either
(a) i ( ) = ; and thus i+1( ) = ;, or (b) i+1( ) is non-empty and does not
differ from i ( ), as dbs( i ( )) = ;, and i ( ) does not contain any non-facts
whose heads occur as facts. Thus, the following holds:</p>
      </sec>
      <sec id="sec-4-5">
        <title>Theorem 1</title>
        <p>For every ground logic program
k ( ), for each k i0.
, there is a smallest number i0 such that i0 ( ) =</p>
        <sec id="sec-4-5-1">
          <title>Let us write</title>
          <p>1( ) for i0 ( ) with i0 as in Theorem 1. We then define:</p>
        </sec>
      </sec>
      <sec id="sec-4-6">
        <title>Definition 4</title>
        <p>The non-deterministic core (or nd-core) of a program
is given by
1(grd( )).</p>
      </sec>
      <sec id="sec-4-7">
        <title>Example 2</title>
        <p>Let = grd( 3), where 3 is the program from the above. In Example 1, we
already identified that dbl( ) = fn(1) ; n(2) ; succ(1; 2) g. Its unique answer
set is M = fn(1); n(2); succ(1; 2)g. Now, when computing ( ), Rules (11), (12),
and (14) are removed since their body literals succ(1; 1), succ(2; 1), and succ(2; 2)
are contained in dbs( ) n M. As well, the facts succ(1; 2) , n(1) , and n(2)
are removed since their heads are contained in M. Finally, the body literals n(1),
n(2), and succ(1; 2) are removed from the remaining rules since they are contained
in M. We thus obtain 1( ) = ( ) as follows:
choose(2)
Since 1( ) does not contain any weakly-minimal literals, we have that dbs( 1( ))=
;. Therefore, 2( ) = 1( ), and so 1( ) is already the nd-core of 3.
In this example, only one application of was necessary for computing the nd-core
of 3. But this is not always the case. In fact, the number of computation steps
cannot be bounded by a constant, as stated by the following theorem:</p>
      </sec>
      <sec id="sec-4-8">
        <title>Theorem 2</title>
        <p>For every n 2 N, there is a ground program
n such that
1( n ) 6=
n 1( n ).</p>
        <p>Having the nd-core of a program at our disposal, we can now define the generate,
define, and test parts. In the following, an instance of a rule r refers to a rule which
was obtained from r by grounding it and which has possibly been modified by .</p>
      </sec>
      <sec id="sec-4-9">
        <title>Definition 5</title>
        <p>Let be a logic program. Then,
(i) the generate part of consists of all rules which are disjunctive or which have
instances that are involved in even negative cycles within the dependency
graph of the nd-core of ,
(ii) the test part of consists of all rules which are not in the generate part and
which are either constraints or which have instances that are involved in (odd)
negative cycles within the dependency graph of the nd-core of , and
(iii) the define part of consists of all rules which are neither in the generate part
nor in the test part.</p>
        <p>Rules of the generate, define, and test part are referred to as generating, defining,
and testing rules, respectively.</p>
        <p>Note that we are explicitly speaking of instances of (non-ground) rules which are
involved in negative cycles and not of rules whose head literals or predicates are
contained in a negative cycle. Note also that facts are contained in the define part
but it would be straightforward to define a separate part for them.</p>
        <p>Also, often in non-ground ASP, one deals with uniform problem encodings, in
which a program is detached from the set of facts providing the input of the
program. From a database point of view, the program is the intensional database
(IDB) whilst the facts constitute the extensional database (EDB). When trying to
analyse the structure of such a uniform encoding , the analysis of can be done
by computing the nd-core of [ f , where f consists of input facts comprising
instances of atoms of all extensional predicates of .</p>
      </sec>
      <sec id="sec-4-10">
        <title>Example 3</title>
        <p>For the program</p>
        <p>2, the generate part consists of the rules
chooseEven
not chooseAll and chooseAll
not chooseEven,
while the define part contains the facts together with the following rules:
chooseEven; n(X ); n(Y ); succ(X ; Y ); not choose(X );
chooseAll; n(X ).
(22)
(23)</p>
        <sec id="sec-4-10-1">
          <title>The test part of 2 is empty.</title>
          <p>In the dependency graph of 2, instances of (22) are involved in negative cycles
with an even number of negative edges as well as in negative cycles with an odd
number of negative edges. But, contrary to common intuition, these cycles neither
cause the creation of multiple (candidate) answer sets nor do they eliminate them.
Hence, (22) is a defining rule and not a generating or a testing rule. In fact, (22)
would even be a defining rule if predicate succ would not be defined by facts but
by an equivalent set of rules including default negation.</p>
          <p>Next, we discuss some properties of the above definitions.</p>
        </sec>
      </sec>
      <sec id="sec-4-11">
        <title>Lemma 1</title>
        <p>The nd-core of every weakly stratified program, and thus of every (locally) stratified
program, is empty.</p>
        <sec id="sec-4-11-1">
          <title>Consequently, the following holds:</title>
        </sec>
      </sec>
      <sec id="sec-4-12">
        <title>Theorem 3</title>
        <p>The generate part of every weakly stratified program, and thus of every (locally)
stratified program, is empty.</p>
        <p>The following theorem expresses that a non-deterministic behaviour of a program
always has its origin in the generate part.</p>
      </sec>
      <sec id="sec-4-13">
        <title>Theorem 4</title>
        <p>A program with an empty generate part has at most one answer set.
Note that the converse of this statement does not hold since the generate part can
create multiple answer-set candidates which are all eliminated by the test part.</p>
        <p>In concluding this section, we remark that we implemented a tool which classifies
logic programs into its generate, define, and test parts according to our definitions.
The tool is implemented in Java and performs the computation of the nd-core and
the identification of rules which are involved within its negative cycles by means
of an ASP metaprogramming approach, using an answer-set program consisting of
the following modules:
(i)
reify , representing the input program as a set of facts,
(ii)
(iii)
(iv)
(v)
dependency , defining the dependency notions,
bottomlayer , identifying the deterministic bottom-stratum and the
corresponding deterministic bottom-layer,
nd-core , encoding the operator and computing the nd-core, and
cycle , identifying rules which are involved in even and odd negative cycles
within the dependency graph.</p>
        <p>Since the whole metaprogram consists of more than 100 rules we do not list it here.
It is available at www.kr.tuwien.ac.at/staff/kiesl/gdt_classification.lp.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>4 An Explanation Order on Rules</title>
      <p>For obtaining an understandable explanation of an answer-set program, the order
in which the rules are explained is crucial. One way to obtain arguably useful
explanations is to start with generating rules and then continue by explaining how
the predicates of the generate part are constrained by rules from the test part,
thereby also taking the involved defining rules into account. In what follows, we
discuss a method for assigning an explanation order on rules which also copes in
an intuitive way with testing rules that constrain several generating rules.</p>
      <sec id="sec-5-1">
        <title>Example 4</title>
        <p>Assume that we want to assign colours to the vertices of a graph such that every
vertex is either red, blue, or has no colour at all. Furthermore, some of the vertices are
not allowed to be either red or blue, represented by predicates must_not_be_red
and must_not_be_blue. A possible encoding is the following program 4:
There are two generating rules, (24) and (25). Among the three constraints, (26)
is only related to literals generated by (24), (27) is only related to those generated
by (25), and (28) is related to literals that are generated by both generating rules.</p>
        <p>We believe that a useful explanation of the whole program is as follows: we
start by explaining the generating rule (24) together with its corresponding
constraint (26); after that we explain (25) together with (27), and at the end we
mention (28) which constrains both generating rules. A natural-language explanation
realising this order could be as follows:
1. For any vertex, choose whether it is red or not [Rule (24)] such that no vertex
which must not be red is red [Rule (26)].</p>
      </sec>
      <sec id="sec-5-2">
        <title>2. For any vertex, choose whether it is blue or not [Rule (25)] such that no vertex</title>
        <p>which must not be blue is blue [Rule (27)].</p>
      </sec>
      <sec id="sec-5-3">
        <title>3. Additionally, it must not be the case that there is a vertex which is both red and</title>
        <p>blue [Rule (28)].
vertex(X ); red(X ); must_not_be_red(X )</p>
        <p>vertex(X ); blue(X ); must_not_be_blue(X )
vertex(X ); red(X ); blue(X )</p>
        <p>To formalise the intuition behind this order, we make use of the strongly-connected
components (SCCs) of the rule graph RG( ) of a program . Given a rule r 2
we denote by SCC(r ) the SCC of RG( ) containing r . We call the graph obtained
by collapsing SCCs of RG( ) the reduction graph of RG( ). An SCC of RG( ) is
maximal if it does not have an outgoing edge within the reduction graph of RG( ).</p>
      </sec>
      <sec id="sec-5-4">
        <title>Definition 6</title>
        <p>A rule of a program</p>
        <p>is maximal if it is contained in a maximal SCC of RG( ).</p>
      </sec>
      <sec id="sec-5-5">
        <title>Definition 7</title>
        <p>A generating rule rG of a program is maximal if in RG( ) (i) no generating rule
in SCC(rG ) has more outgoing edges than rG and (ii) there is no directed path from
rG to a generating rule in a different SCC of RG( ).</p>
        <p>The intuition behind this definition is to give priority to explaining generating rules
that are most influential in the program (according to outgoing paths) and that are
nearest to testing rules and definitions.</p>
        <p>We next formalise that testing rules can be associated with certain rules.</p>
      </sec>
      <sec id="sec-5-6">
        <title>Definition 8</title>
        <p>Given a logic program , let rT 2 be a testing rule and r 2 a rule. Then, r is
constrained by rT if there is a directed path from r to rT in RG( ), otherwise r is
unconstrained by rT .</p>
        <p>Note that testing rules are trivially constrained by themselves.</p>
      </sec>
      <sec id="sec-5-7">
        <title>Example 5 (continued )</title>
        <p>Figure 1 depicts the rule graph of 4. According to our definitions, (26) constrains
only (24), (27) constrains only (25), and (28) constrains both (24) and (25). All
three constraints induce maximal components and thus they are maximal rules.
Both generating rules (24) and (25) are maximal ones.</p>
        <p>Algorithm 1 computes an order for explaining rules of a given program. Intuitively,
rules are explained in the order they are visited during a depth-first traversal of the
rule graph, starting from maximal rules and maximal generating rules. During the
traversal, rules which are in the same SCC are preferred. Additionally, as soon as all
rules from which a generating rule rG is reachable have been visited, testing rules
which constrain rG are explained. Step (a) first explains unconstrained maximal
rules, then maximal generating rules, and finally arbitrary maximal rules.</p>
      </sec>
      <sec id="sec-5-8">
        <title>Theorem 5</title>
        <p>Algorithm 1 always terminates and assigns a unique number to each rule.
input : rule graph G
output: order in which the rules are explained, defined by order
let sameSCC and otherSCC be stacks
let i := 1
while G contains unlabelled rules do
if otherSCC is empty then
let r be an unlabelled rule according to the following priority:
- first choose unconstrained maximal rules,
- then choose maximal generating rules,
- then choose maximal rules.</p>
        <p>push(sameSCC; r )
while sameSCC or otherSCC is not empty do
if sameSCC is not empty then</p>
        <p>let r := pop(sameSCC)
else</p>
        <p>let r := pop(otherSCC)
if r is not labelled then
let order(r ) := i // now we consider r to be labelled
let i := i + 1
// iterate in the same order as the r 0 occur in the body of r
for all edges (r 0; r ) in G do
if r 0 is in the same SCC as r then</p>
        <p>push(sameSCC; r 0)
else</p>
        <p>push(otherSCC; r 0)
(b)
let l be the list of unlabelled testing rules
order l by the number of generating rules they constrain, descending
for c in l do
if all generating rules constrained by c are labelled then</p>
        <p>push(otherSCC; c)</p>
        <p>Algorithm 1: Computes the order in which rules are explained.</p>
        <p>Example 6 (continued )</p>
        <p>4 does not contain unconstrained rules, so in Step (a) of Algorithm 1 we choose
one of the generating rules, say (24). It has no incoming edges but after labelling it
with 1, all rules constrained by (26) are labelled, hence (26) is pushed in Step (b)
and labelled with 2. Subsequently, the next unlabelled maximal generating rule is
chosen in (a), viz. Rule (25), and labelled with 3. Then, (27) is pushed in (b) and
labelled with 4 since all rules which are constrained by (27) were labelled. Finally,
(28) is pushed in (b) since now both rules which it constrains have been labelled.
Note that (28) is visited after (27) because (28) constrains more generating rules
than (27). We end up with the following order:
1:
2:
blue(X ) _ :blue(X )
vertex(X ),
vertex(X ),
vertex(X ); red(X ); must_not_be_red(X ),
4:
5:
vertex(X ); blue(X ); must_not_be_blue(X ),
vertex(X ); red(X ); blue(X ),
which is exactly the order we wanted at the beginning of this section.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>5 Conclusion</title>
      <p>We introduced methods for structural analysis of non-ground disjunctive answer-set
programs. In particular, we introduced formal definitions for the generate, define,
and test parts of a program. In order to arrive at natural definitions, we first defined
the non-deterministic core of a program, which is obtained by eliminating
deterministically determined rules and literals. This has the potential to eliminate those
negative cycles that cannot become active and thus should not be considered in the
generate or test part. We then defined generating rules to be disjunctive rules as
well as rules having instances that are involved in the non-deterministic core’s even
negative cycles. From the remaining rules, we defined the test part to contain
constraints and rules which have instances that are involved in the core’s odd negative
cycles, and all remaining rules are then considered to represent definitions.</p>
      <p>We implemented a tool based on metaprogramming which classifies rules in an
answer-set program accordingly. The aim of this tool is to apply it for translating
programs into (controlled) natural language, however by itself this tool can be useful
for developers to quicker gain an understanding of a given program. Based on our
definitions of generate-define-test, we introduced an algorithm which computes an
order for translating rules of a program into CNL. We believe we found criteria that
yield a clear and understandable explanation of the overall program.</p>
      <p>
        Our approach differs from justification-based methods (
        <xref ref-type="bibr" rid="ref21">Pontelli et al. 2009</xref>
        ;
        <xref ref-type="bibr" rid="ref8">Damásio et al. 2013</xref>
        ) which aim at explaining the truth of certain atoms rather than
explaining the whole program itself and which in turn are closely related to the
debugging of logic programs
        <xref ref-type="bibr" rid="ref14 ref18 ref19 ref20 ref25 ref3 ref4 ref7">(Pereira et al. 1993; Brain and De Vos 2005; Brain et al.
2007; Gebser et al. 2008; Oetsch et al. 2010; Oetsch et al. 2011; Shchekotykhin
2015)</xref>
        .
      </p>
      <p>As future work, we plan to automatically realise a translation of answer-set
programs into controlled natural language. Moreover, we want to extend our methods
to programs of broader syntactic classes, e.g., classes with language features like
choice constructs or aggregate relations.</p>
      <p>
        We also plan to investigate how our definition of the non-deterministic core
relates to program normal forms as introduced by Brass et al. (1996; 2001) for the
computation of the well-founded semantics. Note that the normal forms introduced
by
        <xref ref-type="bibr" rid="ref7">Costantini and Provetti (2005)</xref>
        might differ quite drastically from the original
program, so they do not seem promising for our purposes of rule classification.
      </p>
      <p>
        We think that optimisation of non-ground logic programs can profit from our
work, e.g.,
        <xref ref-type="bibr" rid="ref2">Baral (2003)</xref>
        demonstrated that for many programs the efficiency can be
improved by modifying the generate part such that conditions from the test part are
already taken into account when generating candidate solutions, thereby shrinking
the solver search space. Arguably, such optimisations can be performed easier and
possibly automated if corresponding program parts can be identified automatically.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Apt</surname>
            ,
            <given-names>K. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blair</surname>
            ,
            <given-names>H. A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Walker</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <year>1988</year>
          .
          <article-title>Towards a theory of declarative knowledge</article-title>
          .
          <source>In Foundations of Deductive Databases and Logic Programming</source>
          . Morgan Kaufmann, Chapter
          <volume>2</volume>
          ,
          <fpage>89</fpage>
          -
          <lpage>148</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Baral</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <year>2003</year>
          .
          <article-title>Knowledge Representation, Reasoning, and Declarative Problem Solving</article-title>
          . Cambridge University Press, New York, NY, USA.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Brain</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>De Vos</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <year>2005</year>
          .
          <article-title>Debugging logic programs under the answer set semantics</article-title>
          .
          <source>In Proceedings of the 3rd International ASP Workshop (ASP</source>
          <year>2005</year>
          ).
          <source>CEUR Workshop Proceedings</source>
          . 142-
          <fpage>152</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Brain</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gebser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pührer</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schaub</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tompits</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Woltran</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <year>2007</year>
          .
          <article-title>Debugging ASP programs by means of ASP</article-title>
          .
          <source>In Proceedings of the 9th International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR</source>
          <year>2007</year>
          ), C. Baral, G. Brewka, and J. Schlipf,
          <source>Eds. Lecture Notes in Computer Science</source>
          , vol.
          <volume>4483</volume>
          . Springer,
          <fpage>31</fpage>
          -
          <lpage>43</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Brass</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Dix</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <year>1996</year>
          .
          <article-title>Characterizing D-WFS: Confluence and iterated GCWA</article-title>
          .
          <source>In Proceedings of the 4th European Workshop on Logics in Artificial Intelligence (JELIA</source>
          <year>1996</year>
          ),
          <string-name>
            <given-names>J. J.</given-names>
            <surname>Alferes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Pereira</surname>
          </string-name>
          , and E. Orlowska,
          <source>Eds. Lecture Notes in Computer Science</source>
          , vol.
          <volume>1126</volume>
          . Springer,
          <fpage>268</fpage>
          -
          <lpage>283</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Brass</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dix</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Freitag</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Zukowski</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          <year>2001</year>
          .
          <article-title>Transformation-based bottom-up computation of the well-founded model</article-title>
          .
          <source>Theory and Practice of Logic Programming</source>
          <volume>1</volume>
          ,
          <fpage>497</fpage>
          -
          <lpage>538</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Costantini</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Provetti</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <year>2005</year>
          .
          <article-title>Normal forms for answer sets programming</article-title>
          .
          <source>Theory and Practice of Logic Programming</source>
          <volume>5</volume>
          ,
          <fpage>747</fpage>
          -
          <lpage>760</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Damásio</surname>
            ,
            <given-names>C. V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Analyti</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Antoniou</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          <year>2013</year>
          .
          <article-title>Justifications for logic programming</article-title>
          .
          <source>In Proceedings of the 12th International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR</source>
          <year>2013</year>
          ), P. Cabalar and T. C. Son,
          <source>Eds. Lecture Notes in Computer Science</source>
          , vol.
          <volume>8148</volume>
          . Springer,
          <fpage>530</fpage>
          -
          <lpage>542</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Denecker</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lierler</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Truszczynski</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Vennekens</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <year>2012</year>
          .
          <article-title>A Tarskian informal semantics for answer set programming</article-title>
          .
          <source>In Technical Communications of the 28th International Conference on Logic Programming (ICLP</source>
          <year>2012</year>
          ),
          <string-name>
            <given-names>A.</given-names>
            <surname>Dovier</surname>
          </string-name>
          and V. S. Costa, Eds.
          <source>LIPIcs</source>
          , vol.
          <volume>17</volume>
          .
          <string-name>
            <surname>Schloss</surname>
          </string-name>
          Dagstuhl - Leibniz-Zentrum für Informatik,
          <volume>277</volume>
          -
          <fpage>289</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Dimopoulos</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Torres</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <year>1996</year>
          .
          <article-title>Graph theoretical structures in logic programs and default theories</article-title>
          .
          <source>Theoretical Computer Science</source>
          <volume>170</volume>
          ,
          <issue>1</issue>
          ,
          <fpage>209</fpage>
          -
          <lpage>244</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Eiter</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Faber</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Leone</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Pfeifer</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          <year>2000</year>
          .
          <article-title>Declarative problem-solving using the DLV system</article-title>
          .
          <source>In Logic-Based Artificial Intelligence, J</source>
          . Minker, Ed. Kluwer Academic Publishers,
          <fpage>79</fpage>
          -
          <lpage>103</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Eiter</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ianni</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Krennwallner</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <year>2009</year>
          .
          <article-title>Answer set programming: A primer</article-title>
          .
          <source>In Reasoning Web. Semantic Technologies for Information Systems, 5th International Summer School</source>
          <year>2009</year>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Tessaris</surname>
          </string-name>
          , E. Franconi,
          <string-name>
            <given-names>T.</given-names>
            <surname>Eiter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Gutierrez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Handschuh</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.-C. Rousset</surname>
            , and
            <given-names>R. A</given-names>
          </string-name>
          . Schmidt,
          <source>Eds. Lecture Notes in Computer Science</source>
          , vol.
          <volume>5689</volume>
          . Springer,
          <fpage>40</fpage>
          -
          <lpage>110</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Erdem</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Yeniterzi</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <year>2009</year>
          .
          <article-title>Transforming controlled natural language biomedical queries into answer set programs</article-title>
          .
          <source>In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing (BioNLP</source>
          <year>2009</year>
          ).
          <source>Association for Computational Linguistics</source>
          ,
          <fpage>117</fpage>
          -
          <lpage>124</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Gebser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pührer</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schaub</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tompits</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <year>2008</year>
          .
          <article-title>A meta-programming technique for debugging answer-set programs</article-title>
          .
          <source>In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI</source>
          <year>2008</year>
          ). Vol.
          <volume>8</volume>
          .
          <fpage>448</fpage>
          -
          <lpage>453</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Gelfond</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Lifschitz</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <year>1988</year>
          .
          <article-title>The stable model semantics for logic programming</article-title>
          .
          <source>In Proceedings of the 5th International Conference and Symposium on Logic Programming (ICLP/SLP</source>
          <year>1988</year>
          ),
          <string-name>
            <given-names>R. A.</given-names>
            <surname>Kowalski</surname>
          </string-name>
          and
          <string-name>
            <given-names>K. A.</given-names>
            <surname>Bowen</surname>
          </string-name>
          , Eds. MIT Press,
          <fpage>1070</fpage>
          -
          <lpage>1080</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Leone</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pfeifer</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Faber</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eiter</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gottlob</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Perri</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Scarcello</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          <year>2006</year>
          .
          <article-title>The DLV system for knowledge representation and reasoning</article-title>
          .
          <source>ACM Transactions on Computional Logic 7</source>
          ,
          <issue>3</issue>
          ,
          <fpage>499</fpage>
          -
          <lpage>562</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <surname>Lifschitz</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <year>2002</year>
          .
          <article-title>Answer set programming and plan generation</article-title>
          .
          <source>Artificial Intelligence</source>
          <volume>138</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>2</lpage>
          ,
          <fpage>39</fpage>
          -
          <lpage>54</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Oetsch</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pührer</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tompits</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <year>2010</year>
          .
          <article-title>Catching the Ouroboros: On debugging non-ground answer-set programs</article-title>
          .
          <source>Theory and Practice of Logic Programming</source>
          <volume>10</volume>
          ,
          <fpage>513</fpage>
          -
          <lpage>529</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Oetsch</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pührer</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tompits</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <year>2011</year>
          .
          <article-title>Stepping through an answer-set program</article-title>
          .
          <source>In Proceedings of the 11th International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR</source>
          <year>2011</year>
          ),
          <string-name>
            <given-names>J. P.</given-names>
            <surname>Delgrande</surname>
          </string-name>
          and W. Faber,
          <source>Eds. Lecture Notes in Computer Science</source>
          , vol.
          <volume>6645</volume>
          . Springer,
          <fpage>134</fpage>
          -
          <lpage>147</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <surname>Pereira</surname>
            ,
            <given-names>L. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Damásio</surname>
            ,
            <given-names>C. V.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Alferes</surname>
            ,
            <given-names>J. J.</given-names>
          </string-name>
          <year>1993</year>
          .
          <article-title>Diagnosis and debugging as contradiction removal</article-title>
          .
          <source>In Proceedings of the 2nd International Workshop on Logic Programming and Non-monotonic Reasoning (LPNMR</source>
          <year>1993</year>
          ),
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Pereira</surname>
          </string-name>
          and
          <string-name>
            <surname>A</surname>
          </string-name>
          . Nerode, Eds. The MIT Press,
          <fpage>316</fpage>
          -
          <lpage>330</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <surname>Pontelli</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Son</surname>
            ,
            <given-names>T. C.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Elkhatib</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          <year>2009</year>
          .
          <article-title>Justifications for logic programs under answer set semantics</article-title>
          .
          <source>Theory and Practice of Logic Programming</source>
          <volume>9</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>56</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Przymusinska</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <article-title>and</article-title>
          <string-name>
            <surname>Przymusinski</surname>
            ,
            <given-names>T. C.</given-names>
          </string-name>
          <year>1990</year>
          .
          <article-title>Weakly stratified logic programs</article-title>
          .
          <source>Fundamenta Informaticae</source>
          <volume>13</volume>
          ,
          <issue>1</issue>
          ,
          <fpage>51</fpage>
          -
          <lpage>65</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>Schwitter</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <year>2012</year>
          .
          <article-title>Answer set programming via controlled natural language processing</article-title>
          .
          <source>In Proceedings of the 3rd International Workshop on Controlled Natural Language (CNL</source>
          <year>2012</year>
          ).
          <source>Lecture Notes in Computer Science</source>
          , vol.
          <volume>7427</volume>
          . Springer,
          <fpage>26</fpage>
          -
          <lpage>43</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <string-name>
            <surname>Schwitter</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <year>2013</year>
          .
          <article-title>The jobs puzzle: Taking on the challenge via controlled natural language processing</article-title>
          .
          <source>Theory and Practice of Logic Programming</source>
          <volume>13</volume>
          ,
          <fpage>4</fpage>
          -
          <lpage>5</lpage>
          ,
          <fpage>487</fpage>
          -
          <lpage>501</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <string-name>
            <surname>Shchekotykhin</surname>
            ,
            <given-names>K. M.</given-names>
          </string-name>
          <year>2015</year>
          .
          <article-title>Interactive query-based debugging of ASP programs</article-title>
          .
          <source>In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI</source>
          <year>2015</year>
          ),
          <string-name>
            <given-names>B.</given-names>
            <surname>Bonet</surname>
          </string-name>
          and S. Koenig, Eds. AAAI Press,
          <fpage>1597</fpage>
          -
          <lpage>1603</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <string-name>
            <surname>Ullman</surname>
            ,
            <given-names>J. D.</given-names>
          </string-name>
          <year>1988</year>
          .
          <article-title>Principles of Database and Knowledge-Base Systems</article-title>
          . Vol.
          <volume>1</volume>
          . Computer Science Press.
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          <string-name>
            <surname>You</surname>
            ,
            <given-names>J.-H.</given-names>
          </string-name>
          <article-title>and</article-title>
          <string-name>
            <surname>Yuan</surname>
            ,
            <given-names>L. Y.</given-names>
          </string-name>
          <year>1994</year>
          .
          <article-title>A three-valued semantics for deductive databases and logic programs</article-title>
          .
          <source>Journal of Computer and System Sciences</source>
          <volume>49</volume>
          ,
          <issue>2</issue>
          ,
          <fpage>334</fpage>
          -
          <lpage>361</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>