<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A Divide-And-Conquer-Method for Computing Multiple Conflicts for Diagnosis</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Kostyantyn Shchekotykhin</string-name>
          <email>kostyantyn.shchekotykhin@aau.at</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Dietmar Jannach</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Thomas Schmitz</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Alpen-Adria University Klagenfurt</institution>
          ,
          <country country="AT">Austria</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>TU Dortmund</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <fpage>3</fpage>
      <lpage>10</lpage>
      <abstract>
        <p>In classical hitting set algorithms for ModelBased Diagnosis (MBD) that use on-demand conflict generation, a single conflict is computed whenever needed during tree construction. Since such a strategy leads to a full “restart” of the conflict-generation algorithm on each call, we propose a divide-and-conquer algorithm called MERGEXPLAIN which efficiently searches for multiple conflicts during a single call. The design of the algorithm aims at scenarios in which the goal is to find a few leading diagnoses and the algorithm can - due to its non-intrusive design - be used in combination with various underlying reasoners (theorem provers). An empirical evaluation on different sets of benchmark problems shows that our proposed algorithm can lead to significant reductions of the required diagnosis times when compared to a “one-conflict-ata-time” strategy.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>In Model-Based Diagnosis (MBD), the concept of conflicts
describes parts of a system which – given a set of
observations – cannot all work correctly. Besides MBD, the
calculation of minimal conflicts is a central task in a number of
other AI approaches [1]. Reiter [2] showed that the minimal
hitting sets of conflicts correspond to diagnoses, where a
diagnosis is a possible explanation why a system’s observed
behavior differs from its expected behavior. He used this
property for the computation of diagnoses in the
breadthfirst hitting set tree (HS-tree) diagnosis algorithm.</p>
      <p>Over time, the principle of this MBD approach was used
for a number of different diagnosis problems such as
electronic circuits, hardware descriptions in VHDL, program
specifications, ontologies, and knowledge-based systems[3;
4; 5; 6; 7]. A reason for the broad utilization of hitting set
approaches is that its principle does not depend on the
underlying knowledge representation and reasoning technique,
because only a general Theorem Prover (TP) – a component
that returns conflicts – is needed.</p>
      <p>The implementation of a TP can be done in different
ways. First, the conflict detection can be implemented as
a reasoning task, e.g., by modifying a consistency
checking algorithm [8; 9]. Second, “non-intrusive” conflict
detection techniques can be used with a variety of reasoning
approaches, since they require only a very limited reasoning
functionality like consistency or entailment checking
without knowing the internals of the reasoning algorithm. Such
methods can benefit from the newest improvements in
reasoning algorithms, such as incremental solving, heuristics,
learning strategies, etc., without any modifications.</p>
      <p>
        A non-intrusive conflict detection algorithm which has
shown to be very efficient in different application
scenarios is Junker’s QUICKXPLAIN [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] (QXP for short) which
was designed to find a single minimal conflict based on a
divide-and-conquer strategy. The algorithm was originally
developed in the context of constraint problems, but since
its method is independent of the underlying reasoner, it was
used in several of the hardware and software diagnosis
approaches mentioned above.
      </p>
      <p>
        In many classical hitting set based approaches, conflicts
are computed individually with QXP during HS-tree
construction when they are required, as in many domains not
all conflicts are known in advance [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. This, however, has
the effect that QXP has to be “restarted” with a slightly
different configuration whenever a new conflict is needed.
      </p>
      <p>In this paper, we propose MERGEXPLAIN (MXP for
short), a divide-and-conquer algorithm which searches for
multiple conflicts during a single decomposition run. Our
method is built upon QXP and is therefore also
nonintrusive. The basic idea behind MXP is that (a) the early
identification of multiple conflicts can speed up the overall
diagnosis process, e.g., due to better conflict “reuses” [2],
and that (b) we can identify additional conflicts faster when
we decompose the original components into smaller subsets
with the divide-and-conquer strategy of MXP.</p>
      <p>The paper is organized as follows. After a problem
characterization in Section 2, we present the details of MXP in
Section 3 and discuss the properties of the algorithm.
Section 4 presents the results of an extensive empirical
evaluation using various diagnosis benchmark problems. Previous
work is finally discussed in Section 5.
2
2.1</p>
    </sec>
    <sec id="sec-2">
      <title>Preliminaries</title>
      <sec id="sec-2-1">
        <title>The Diagnosis Problem</title>
        <p>We use the definitions of[2] to characterize a system,
diagnoses, and conflicts.</p>
        <p>Definition 1 (System). A system is a pair (SD, COMPS)
where SD is a system description (a set of logical sentences)
and COMPS represents the system’s components (a finite set
of constants).
A diagnosis problem arises when a set of logical
sentences OBS, called observations, is inconsistent with the
normal behavior of the system (SD, COMPS). The correct
behavior is represented in SD with an “abnormal” predicate
AB/1. That is, for any component ci ∈ COMPS the literal
¬AB(ci) represents the assumption that the component ci
behaves correctly.</p>
        <p>Definition 2 (Diagnosis). Given a diagnosis problem (SD,
COMPS, OBS), a diagnosis is a minimal set Δ ⊆ COMPS such
that SD ∪ OBS ∪ {AB(c)|c ∈ Δ} ∪ {¬AB(c)|c ∈ COMPS\Δ}
is consistent.</p>
        <p>A diagnosis therefore corresponds to a minimal subset of
the system components which, if assumed to be faulty (and
thus behave abnormally) explain the system’s behavior, i.e.,
are consistent with the observations.</p>
        <p>Two general classes of MBD algorithms exist. One relies
on direct problem encodings and the aim is often to find one
diagnosis quickly, see [12; 13; 14]. The other class relies on
the computation of conflicts and their hitting sets (see next
section). Such diagnosis algorithms are often used when the
goal is to findmultiple or all minimal diagnoses. In the
context of our work, techniques of the second class can
immediately profit when the conflict generation process is done
more efficiently.
2.2</p>
      </sec>
      <sec id="sec-2-2">
        <title>Diagnoses as Hitting Sets</title>
        <p>Finding all minimal diagnoses corresponds to finding all
minimal hitting sets (HS) of all existing conflicts [2].
Definition 3 (Conflict) . A conflict CS for (SD, COMPS,
OBS) is a set {c1, . . . , ck} ⊆ COMPS such that SD ∪ OBS
∪{¬AB(ci) | ci ∈ CS } is inconsistent.</p>
        <p>Assuming that all components of a conflict work correctly
therefore contradicts the observations. A conflict CS is
minimal, if no proper subset of CS is also a conflict.</p>
        <p>
          To find the set ofall minimal diagnoses for a given
problem, [2] proposed a breadth-first HS-tree algorithm with tree
pruning and conflict reuse. A correction to this algorithm
was proposed by Greiner et al. which uses a directed acyclic
graph (DAG) instead of the tree to correctly deal with
nonminimal conflicts [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]. Our work, however, does not
depend on this correction as QXP as well as our proposed
MXP method always return minimal conflicts. Apart from
this, a number of algorithmic variations were suggested
in the literature which, for example, use problem-specific
heuristics [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ], a greedy search algorithm, or apply
parallelization techniques [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ], see also [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ] for an overview.
2.3
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>QUICKXPLAIN (QXP)</title>
        <p>
          QXP was developed in the context of inconsistent constraint
satisfaction problems (CSPs) and the computation of
explanations. E.g., in case of an overconstrained CSP, the
problem consists in determining a minimal set of constraints
which causes the CSP to become unsolvable for the given
inputs. A simplified version of QXP [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ] is shown in
Algorithm 1. The rough idea of QXP is to apply a recursive
procedure which relaxes the input set of faulty constraints
C by partitioning it into two sets C1 and C2 (line 6). If C1
is a conflict the algorithm continues partitioning C1 in the
next recursive call. Otherwise, i.e., if the last partitioning
has split all conflicts in C, the algorithm extracts a conflict
from the sets C1 and C2. This way, QXP finally identifies
single constraints which are inconsistent with the remaining
consistent set of constraints and the background theory.
        </p>
        <sec id="sec-2-3-1">
          <title>Algorithm 1: QUICKXPLAIN(B, C)</title>
          <p>Input: B: background theory, C: the set of possibly
faulty constraints</p>
          <p>
            Output: A minimal conflict CS ⊆ C
1 if isConsistent(B ∪ C) then return ‘no conflict’;
2 else if C = ∅ then return ∅;
3 return GETCONFLICT(B, B, C)
function GETCONFLICT (B, D, C)
if D 6= ∅ ∧ ¬ isConsistent(B) then return ∅;
if |C| = 1 then return C;
Split C into disjoint, non-empty sets C1 and C2
D2 ← GETCONFLICT (B ∪ C1, C1, C2)
D1 ← GETCONFLICT (B ∪ D2, D2, C1)
return D1 ∪ D2
Theorem 1 ([
            <xref ref-type="bibr" rid="ref10">10</xref>
            ]). Let B be a background theory, i.e., a
set of constraints considered as correct, and C be a set of
possibly faulty constraints. Then, QUICKXPLAIN always
terminates. If B ∪ C is consistent it returns ‘no conflict’.
Otherwise, it returns a minimal conflict CS .
2.4
          </p>
        </sec>
      </sec>
      <sec id="sec-2-4">
        <title>Using QXP During HS-Tree Construction</title>
        <p>
          Assume that MBD is applied to find an error in the
definition of a CSP. The CSP comprises the set of possibly faulty
constraints C. These are the elements of COMPS. The
system description SD corresponds to the semantics of the
constraints in C. Finally, the observations OBS are encoded as
unary constraints and are added to the background theory
B. During the HS-tree construction, QXP is called
whenever a new node is created and no conflict reuse is
possible. As a result, QXP can either return one minimal conflict ,
which can be used to label the new node, or return ’no
conflict’, which would mean that a diagnosis is found at the tree
node. Note that QXP can be used with other algorithms,
e.g., preference-based search [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ] or boolean search [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ], in
the same way as with the HS-tree algorithm.
3
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>MERGEXPLAIN (MXP): Algorithm</title>
    </sec>
    <sec id="sec-4">
      <title>Details</title>
      <p>3.1</p>
      <sec id="sec-4-1">
        <title>General Considerations</title>
        <p>
          The pseudo-code of MXP, which unlike QXP can return
multiple conflicts at a time, is given in Algorithm 2. MXP,
like QXP, is generally applicable to a variety of problem
domains. The mapping to the terminology used in MBD (SD,
COMPS, OBS) is straightforward as discussed in the previous
section. In the following, we will use the notation and
symbols from [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ], e.g., C or B, and constraints as a knowledge
representation formalism.
        </p>
        <p>Note that there are applications of MBD in which the
function isConsistent has to be “overwritten” to take the
specifics of the underlying knowledge representation and
reasoning system into account. The ontology debugging
approach presented in [7] for example extends
isConsistent with the verification of entailments of a logical theory.
MXP can be used in such scenarios after the corresponding
adaptation of the implementation of isConsistent.</p>
        <p>Furthermore, MXP can be easily extended for cases in
which the MBD approach has to support the specification
of (multiple) test cases, i.e., sets of formulas that must be
consistent or inconsistent with the system description, e.g.,
[21; 22].
3.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>Algorithm Rationale</title>
        <p>MXP (Algorithm 2) accepts two sets of constraints as
inputs, B as the assumed-to-be-correct set of background
constraints and C, the possibly faulty components/constraints.</p>
        <p>In case C∪B is inconsistent, MXP returns a set of minimal
conflicts Γ by calling the recursive function FINDCONFLICTS
in line 3. This function again accepts B and C as an input and
returns a tuple hC0, Γi, where Γ is a set of minimal conflicts
and C0 ⊂ C is a set of constraints that does not contain any
conflicts, i.e., B ∪ C0 is consistent.</p>
        <p>The logic of FINDCONFLICTS is similar to QXP in that we
decompose the problem into two parts in each recursive call
(lines 7–9). Differently from QXP, however, we look for
conflicts in both splits C1 and C2 independently and then
combine the conflicts that are eventually found in the two
halves (line 10)1. If there is, e.g., a conflict in the first part
and one in the second, FINDCONFLICTS will find them
independently from each other. Of course, there might also be
conflicts in C whose elements are spread across both C1 and
C2, that is, the set C10 ∪ C20 ∪ B is inconsistent. This situation
is addressed in lines 11–15. The computation of a minimal
conflict is done by two calls to GETCONFLICT (Algorithm 1).
In the first call this function returns a minimal set X ⊆ C10
such that X ∪C2∪B is a conflict (line 12). In line 13, we then
0
look for a subset of C20, say Y , such that Y ∪ X corresponds
to a minimal conflict CS . The latter is added to Γ (line 15).
In order to restore the consistency of C10 ∪ C20 ∪ B we have to
remove at least one element α ∈ CS from either C10 or C20.
Therefore, in line 14 the algorithm removes α ∈ X ⊆ CS
from C10.</p>
        <p>Note that MXP allows us to use different split functions
in line 7. In our default implementation we use a function
that splits the set of constraints C into two equal parts, i.e.,
split(n) = n/2, where |C| = n. In the worst case this split
function results in a perfect binary tree with n leaves.
Consequently, the total number of nodes is 2n − 1, which
correspond to 2(2n − 1) consistency checks (lines 5 and 11).
Other split functions might result in a similar number of
consistency checks in the worst case as well, since in any
case MXP has to traverse a binary tree with n leaves. For
instance, the function split(n) = n − 1 results in a tree with
one branch of the depth n − 1 and n leaves, that is, 2n − 1
nodes to traverse. However, while the number of nodes to
explore might be comparable, the important point is that the
computational costs for the individual consistency checks
can be different depending on the splitting strategy.
Under the reasonable assumption that consistency checking of
smaller sets of constraints requires less time, the function
split(n) = n/2 allows MXP to split the set of constraints
faster, thus, improving the overall runtime.
3.3</p>
      </sec>
      <sec id="sec-4-3">
        <title>Example</title>
        <p>Consider a CSP consisting of six constraints {c0, ..., c5}.
The constraint c0 is considered correct, i.e., B = {c0}. Let
{{c0, c1, c3}, {c0, c5}, {c2, c4}} be the set of minimal
conflicts. Algorithm 2 proceeds as follows (Figure 1).</p>
        <p>Since the input CSP (B ∪ C) is not consistent, the
algorithm enters the recursion. In the first step,
FINDCONFLICTS partitions the input set (line 7) into the two subsets
1The calls in line 8 and 9 can in fact be executed in parallel.</p>
        <sec id="sec-4-3-1">
          <title>Algorithm 2: MERGEXPLAIN(B, C)</title>
          <p>Input: B: background theory, C: the set of possibly
faulty constraints</p>
          <p>Output: Γ, a set of minimal conflicts
1 if ¬isConsistent (B) then return ‘no solution’;
2 if isConsistent (B ∪ C) then return ∅;
3 h_, Γi ← FINDCONFLICTS(B, C)
4 return Γ;
function FINDCONFLICTS (B, C) returns tuple hC0, Γi
if isConsistent(B ∪ C) then return hC, ∅i;
if |C| = 1 then return h∅, {C}i;
Split C into disjoint, non-empty sets C1 and C2
hC10, Γ1i ← FINDCONFLICTS(B, C1)
hC20, Γ2i ← FINDCONFLICTS(B, C2)
Γ ← Γ1 ∪ Γ2;
while ¬isConsistent (C10 ∪ C20 ∪ B) do</p>
          <p>X ← GETCONFLICT(B ∪ C20, C20, C10)
CS ← X ∪ GETCONFLICT(B ∪ X, X, C20)
C10 ← C10 \ {α} where α ∈ X
Γ ← Γ ∪ {CS }
return hC10 ∪ C20, Γi
C1 = {c1,c2,c3} and C2 = {c4,c5} and provides them as
input to the recursive calls (lines 8 and 9). In the next level
of the recursion – marked with 2 in Figure 1 – the input is
found to be inconsistent (line 5) and again partitioned into
two sets (line 7). In the subsequent calls, 3 and 4 , the two
input sets are found to be consistent (line 5) and, therefore,
the set {c1, c2, c3} has to be analyzed using GETCONFLICT
(lines 12 and 13) defined in Algorithm 1. GETCONFLICT
returns the conflict { c1,c3}, which is added to Γ. Finally,
FINDCONFLICTS removes c1 from the set C10 and returns the
tuple h{c2,c3}, {{c1,c3}}i to 1 .</p>
          <p>Next, the “right-hand” part of the initial input, the set
C2 = {c4,c5}, is provided as input to FINDCONFLICTS 5 .
Since C2 is inconsistent, it is partitioned into two sets
C1 = {c4} and C2 = {c5}. The first recursive call 6
returns h{c4}, ∅i since the input is consistent. The second
call 7 , in contrast, finds that the input comprises only
one constraint that is inconsistent with the background
theory B. Therefore, it returns h∅,{{c5}}i in line 6. Since
C10 ∪ C20 = {c4} ∪ ∅ is consistent with B, FINDCONFLICTS 5
returns h{c4}, {{c5}}i to 1 .</p>
          <p>Finally, in 1 the set of constraints C10 ∪ C20 = {c2,c3} ∪
{c4} is found to be inconsistent with B (line 11) and
GETCONFLICT is called. The method returns the conflict { c2,c4}
and c2 is removed from C10. The resulting set {c3,c4} is
consistent and MXP returns Γ = {{c1,c3}, {c5}, {c2, c4}}.
Theorem 2. Given a background theory B and a set of
constraints C, Algorithm 2 always terminates and returns
• ‘no solution’, if B is inconsistent,
• ∅, if B ∪ C is consistent, and
• a set of minimal conflicts Γ, otherwise.</p>
          <p>Proof. In the first case, given an inconsistent background
theory B, the algorithm terminates in line 1 and returns ‘no
solution’. In the second case, if the set B ∪ C is consistent,
C1 = {c1, c2, c3} C2 = {c4, c5}
h{c2, c3} , {{c1, c3}}i
1 : h{c4} , {{c5}}i
Γ = {{c1, c3} , {c5}} ∪ {{c2, c4}}
C = {c3, c4}
y
C1 = {c1, c2} C2 = {c3}
h{c1, c2} , ∅i
2 : h{c3} , ∅i
Γ = ∅ ∪ {{c1, c3}}
C = {c2, c3}
%</p>
          <p>C1 = {c4} C2 = {c5}
5 : h{c4} , ∅i
h∅, {{c5}}i
isConsistent X
3 : iBsC∪oCns=ist{ecn0t, cX1, c2}
4 : iBsC∪oCns=ist{ecn0t, cX3}
6 : iBsC∪oCns=ist{ecn0t, cX4}</p>
          <p>B ∪ C = {c0, c5}
7 : isConsistent
|C| = 1
then no subset of C is a conflict. MXP terminates and
returns ∅.</p>
          <p>Finally, if the set B ∪ C is inconsistent, the algorithm
enters the recursion in line 3. The function FINDCONFLICTS
in each call partitions the input set C into two sets C1 and
C2. The partitioning continues until either the found set
of constraints C is consistent or a singleton conflict is
detected. Therefore, every recursion branch ends after at most
log |C|−1 calls. Consequently, FINDCONFLICTS terminates if
the conflict detection loop in lines 11–15 always terminates.</p>
          <p>We consider two situations. If the set C10 ∪ C20 is consistent
with B, the loop terminates. Otherwise, in each iteration at
least one conflict in the set C10 ∪ C20 is resolved. This fact
follows from Theorem 1 according to which the function
GETCONFLICT in Algorithm 1 always returns a minimal
conflict if the input parameter C is inconsistent with B. Since
the number of conflicts is finite and in each iteration one of
the conflicts in C10 ∪ C20 is resolved in line 14, the loop will
terminate after a finite number of iterations. Consequently,
Algorithm 2 terminates and returns a set of minimal
conflicts Γ.</p>
          <p>Corollary 1. Given a consistent background theory B and a
set of inconsistent constraints C, Algorithm 2 always returns
a set of minimal conflicts Γ such that there exists a diagnosis
Δi ⊆ SCSi∈Γ CS i.</p>
          <p>The proof follows from the fact that – similar to the
HStree algorithm – a conflict is resolved by removing one of its
elements from the set of constraints C1 in line 14. The loop
in line 11 guarantees that every conflict CS i ∈ C10 ∪ C20 is
hit. Consequently, FINDCONFLICTS hits every conflict in the
input set C and the set of constraints {α1, . . . , αn} removed
in every call of line 14 is a superset or equal to a diagnosis of
the problem. The construction of at least one diagnosis from
the found conflicts Γ can be done by the HS-tree algorithm.</p>
          <p>
            MXP can in principle use several strategies for the
resolution of conflicts in line 14. The strategy used in MXP
by default is conservative and allows us to find several
conflicts at once. Two additional elimination strategies can be
used in line 14: (1) C10 ← C10 \ X or (2) C10 ← C10 \ CS and
C2 ← C20 \ CS . These more aggressive strategies result in
0
a smaller number of conflicts returned by MXP in each call
but each call returns the results faster. However, for the latter
strategies MXP might not return enough minimal conflicts
for the HS-tree algorithm to compute at least one diagnosis.
For instance, let {{c1, c2} , {c1, c3} , {c2, c4}} be the set of
all minimal conflicts. If MXP returns Γ = {{c1, c2}}, which
is one of the possible valid outputs, then the HS-tree
algorithm fails to find a diagnosis as{c1, c2} must be hit twice.
In this case, the HS-tree algorithm must call MXP multiple
times or another algorithm for diagnosis computation must
be used, e.g., [
            <xref ref-type="bibr" rid="ref23">23</xref>
            ].
          </p>
          <p>Corollary 2. Algorithm 2 is sound, i.e., every set CS ∈ Γ
is a minimal conflict, and complete, i.e., given a diagnosis
problem for which at least one minimal conflict exists,
Algorithm 2 returns Γ 6= ∅.</p>
          <p>The soundness of the algorithm follows from Theorem 1,
since the conflict computation of MXP uses the
GETCONFLICT function of QXP. The completeness is shown as
follows: Let B be a background theory and C a set of faulty
constraints, i.e., B ∪ C is inconsistent. Assume MXP returns
Γ = ∅, i.e., no minimal conflicts are found. However, this is
impossible, since the loop in line 11 will never end.
Consequently, Algorithm 2 will not terminate which contradicts
our assumption. Hence, it holds that MXP is complete.
4</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Evaluation</title>
      <p>We have evaluated the efficiency of computing multiple
conflicts at once with MXP using a number of different
diagnosis benchmark problems. As a baseline for the comparison,
we use QXP as a Theorem Prover, which returns exactly
one minimal conflict at a time. Furthermore, we made
measurements with a variant of MXP called PMXP in which
the lines 8 and 9 are executed in parallel in two threads on a
multi-core computer.</p>
      <sec id="sec-5-1">
        <title>4.1 Benchmark Problems</title>
        <p>
          We made experiments with different benchmark problems.
First, we used the five first systems of the DX Competition
(DXC) 2011 Synthetic Track. For each system, 20 scenarios
are specified in which artificial faults were injected. In
addition, we made experiments with a number of CSP problems
from the CSP solver competition 2008 and several CSP
encodings of real-world spreadsheets. The injection of faults
was done in the same way as in [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ].
In addition to these benchmark problems, we developed
a diagnosis problem generator, which can be configured
to generate (randomized) diagnosis problems with varying
characteristics, e.g., with respect to the number of conflicts,
their size, or their position in the system description SD.
4.2
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>Measurement Method</title>
        <p>
          We implemented all algorithms in a Java-based MBD
framework, which uses Choco as an underlying constraint
solver, see [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. The experiments were conducted on a
laptop computer (Intel i7, 8GB RAM). As a performance
indicator we use the time needed (“wall clock”) for computing
one or more diagnoses. The reported running time
numbers are averages of 100 runs of each problem setting that
were done to avoid random effects. We furthermore
randomly shuffled the ordering of the constraints in each run to
avoid effects that might be caused by a certain positioning
of the conflicts in SD. For the evaluation of MXP we used
the most aggressive elimination strategy (2) as described in
Section 3.4.
        </p>
        <p>Since MXP can return more than one conflict at a time, it
is expected to be particularly useful when the problem is to
find a set ofn first (leading) diagnoses, e.g., in the context of
applying MBD to software debugging [5; 7]. We therefore
report the results for the tasks “find-one-diagnosis” (as an
extreme case) and “find-n-diagnoses”.</p>
        <p>
          The task of finding a single diagnosis is comparably
simple and “direct encodings” or algorithms like
INVERSEQUICKXPLAIN [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ] are typically more efficient for this
task than the HS-tree algorithm. For instance,
INVERSEQUICKXPLAIN requires only O(|Δ| log(|C|/|Δ|)) calls to TP.
If TP can check the consistency in polynomial time, then
one diagnosis can also be computed efficiently. The
problem of finding more than one diagnosis is very different and
computationally challenging, because deciding whether an
additional diagnosis exists is NP-complete [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ]. In such
settings the application of methods that are highly efficient for
finding one diagnosis is not always advantageous. For
instance, the evaluation presented in [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] demonstrates this
fact for direct encodings. Therefore a comparison of our
algorithm with approaches for the “find-one-diagnosis”
problem is beyond the scope of our work, as we are interested
in problem settings in which the HS-tree algorithm is
favorable and no assumptions about the underlying reasoner
should be made. When the task is to find all diagnoses, the
performance of MXP is similar to that of QXP as all
existing conflicts have to be determined.
4.3
        </p>
      </sec>
      <sec id="sec-5-3">
        <title>Results</title>
        <p>DXC Benchmark Problems Table 1 shows the
characteristics of the analyzed and CSP-encoded DXC benchmark
problems. Since we consider multiple scenarios per system,
the number of faults and the corresponding diagnoses can
vary strongly across the experiment runs.</p>
        <p>Table 2 shows the observed performance gains when
using MXP instead of QXP in terms of absolute numbers (ms)
and the relative improvement. For the problem of finding the
first 5 diagnoses (QXP-5/MXP-5), the observed
improvements range from 15% up to 45%. For the extreme case of
finding one single diagnosis, even slightly stronger
improvements can be observed. The improvements when searching
for, e.g., the first 10 diagnoses are similar for cases in which
significantly more than 10 diagnoses actually exist.</p>
        <p>System #C #V #F #D #D |D| #Cf |Cf|
74182 21 28 4 - 5 30 - 300 139 4.66 4.9 3.3
74L85 35 44 1 - 3 1 - 215 66.4 3.13 5.9 8.3
74283 38 45 2 - 4 180 - 4,991 1,232.7 4.42 78.8 16.1
74181 67 79 3 - 6 10 - 3,828 877.8 4.53 7.8 10.6
c432 162 196 2 - 5 1 - 6,944 1,069.3 3.38 15.0 19.8</p>
      </sec>
      <sec id="sec-5-4">
        <title>Constraint Problems / Spreadsheets The characteristics</title>
        <p>
          for the next set of benchmark problems (six CSP
competition instances, five CSP-encoded real-world spreadsheets
with injected faults [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]) are shown in Table 3.
        </p>
        <p>Scenario #C #V #F #D |D| #Cf |Cf|
c8 523 239 8 4 6.25 7 1.6
costasArray-13 87 88 2 &gt;5 3.6 &gt;565 45.6
domino-100-100 100 100 3 81 2 2 15
graceful–K3-P2 60 15 4 &gt;117 2.94 &gt;12 29.2
mknap-1-5 7 39 1 2 1 1 2
queens-8 28 8 15 9 10.9 15 2.8
hospital payment 38 75 4 40 4 4 3
profit calculation 28 140 5 42 4.25 11 9
course planning 457 583 2 3024 2 2 55.5
preservation model 701 803 1 22 1 1 22
revenue calculation 93 154 4 1452 3 3 15.7</p>
        <p>The results for determining the five first minimal
diagnoses are shown in Table 42. Again, performance
improvements of up to 54% can be observed. The obtained
improvements vary quite strongly across the different problem
instances: the higher the complexity of the underlying
problem, the stronger are the improvements achieved with our
new method. Only in the two cases in which only one single
conflict exists (see Table 3), the performance can slightly
degrade as MXP performs an additional check if further
conflicts among the remaining constraints exist.</p>
      </sec>
      <sec id="sec-5-5">
        <title>Systematically Generated MBD Problems To be able to</title>
        <p>systematically analyze which factors potentially influence
the obtained performance improvements, we developed an
MBD problem generator in which we could vary (i) the
2The results for finding one diagnosis follow the same trend.
Scenario
c8
costasArray-13
domino-100-100
graceful–K3-P2
mknap-1-5
queens-8
hospital payment
profit calculation
course planning
preservation model
revenue calculation
overall number of COMPS, (ii) the number of conflicts and
their average size (and as a consequence the number of
diagnoses), and (iii) the position of the conflicts in the database.
We considered the last aspect because the performance of
QXP and MXP can largely depend on this aspect3. If,
e.g., there is only one conflict and the conflict is represented
by the two “left-most” elements in SD, QXP’s
divide-andconquer strategy will be able to rule out most other elements
very fast.</p>
        <p>We evaluated the following configurations regarding the
position of the conflicts (see Table 5): (a) Random: The
elements of each conflict are randomly distributed across
SD; (b) Left/Right: All elements of the conflict appear in
exactly one half of SD; (c) LaR (Left and Right): Conflicts
are both in the left and right half, but not spanning both
halves; (d) Neighb.: Conflicts appear randomly across SD,
but only involve “neighboring” elements.</p>
        <p>One specific rationale of evaluating these constellations
individually is that conflicts in some application domains
(e.g., when debugging knowledge bases) might represent
“local” inconsistencies in SD.</p>
        <p>Since the conflicts are known in advance in this
experiment, no CSP solver is needed to determine the
consistency of a given set of constraints. Because zero
computation times are unrealistic, we added simulated consistency
checking times in each call to the TP. The value of the
simulated time quadratically increases with the number of
constraints to be checked and is capped in the experiments at 10
milliseconds. We made additional tests with different
consistency checking times to evaluate to which extent the
improvements obtained with MXP depend on the complexity
of an individual consistency check for the underlying
problem. However, these tests did not lead to any significant
differences.</p>
        <p>Table 5 shows some of the results of this simulation. In
this evaluation, we also include the results of the parallelized
PMXP variant. The following observations can be made.</p>
        <p>(1) The performance of QXP strongly depends on the
position of the conflicts. In the probably most realistic Random
case, MXP helps to reduce the computation times around
20-30%. In the constellations that are “unfortunate” for
QXP, the speedups achieved with MXP can be as high as
75%. When QXP is “lucky” and all conflicts are clustered
3We assume a splitting strategy in which the elements are
simply split in half in the middle with no particular ordering of the
elements.</p>
        <p>Random</p>
        <p>Left
Right</p>
        <p>LaR
Neighb.</p>
        <p>Random</p>
        <p>Left
Right</p>
        <p>LaR
Neighb.</p>
        <p>Random</p>
        <p>Left
Right</p>
        <p>LaR
Neighb.
in the left part of SD, some improvements or light
deteriorations can be observed for MXP. The latter two situations
(all conflicts are clustered in one half) are actually quite
improbable but help us better understand which factors
influence the performance.</p>
        <p>(2) When comparing the results of the first two blocks
in the table, it can be seen that the improvements achieved
with MXP are stronger when there are more components in
SD and more time is needed for performing the individual
consistency checks. This is in line with the results of the
other experiments.</p>
        <p>(3) Parallelization can help to obtain modest additional
improvements. The strongest improvements are observed
for the LaR configuration, which is intuitive as PMXP by
design explores the left and right halves independently in
parallel. Note that in the experiments with the DXC and the
CSP benchmark problems, in most cases we could not
observe runtime improvements through parallelization. This is
caused by two facts. First, the consistency checking times
are often on average below 1 ms, which means that the
relative overhead of starting a new thread can be comparably
high. Second, the used CSP solver causes some additional
overheads and thread synchronization when used in multiple
threads in parallel.
5</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Related Work</title>
      <p>
        In [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], Junker informally sketches a possible extension of
QXP to be able to compute multiple “preferred
explanations” in the context of Preference-Based Search (PBS). The
general goal of Junker’s approach is partially similar to our
work and the proposed extended version of QXP could in
theory be used during the HS-tree construction as well.
      </p>
      <p>Technically, Junker proposes to set a choice point
whenever a constraint ci is found to be consistent with a partial
relaxation during search and thereby look for (a) branches that
lead to conflicts not containing ci and (b) branches leading
to conflicts in which the removal of ci leads to a solution.</p>
      <p>
        Unfortunately, it is not fully clear from the informal
sketch in [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] where the mentioned choice point should
be set. If applied in line 5 of Algorithm 1, conflicts are
only found in the left-most inconsistent partition. The
method would then return only a small subset of all conflicts
MERGEXPLAIN would return. If the split is done for every
ci consistent with a partial relaxation during PBS, the
resulting diagnosis algorithm corresponds to the binary HS-tree
method [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ], which according to the experiments in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] is
not generally favorable over HS-Tree algorithms, in
particular when we are searching for a limited set of diagnoses.
      </p>
      <p>From the algorithm design, note that QXP applies a
constructive conflict computation procedure prior to
partitioning, whereas MXP does the partitioning first – thereby
removing multiple constraints at a time – and then uses a
divide-and-conquer conflict detection approach. Finally, our
method can, depending on the configuration, make a
guarantee about the existence of a diagnosis given the returned
conflicts without the need of computing all existing conflicts.</p>
      <p>
        In general, our work is related to a variety of
(complete) approaches from the MBD literature which aim to
find diagnoses more efficiently than with Reiter’s original
method. Existing works for example try to speed up the
process by exploiting existing hierarchical, tree-like or
distributed structural properties of the underlying problem [16;
26], through parallelization [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ], or by solving the dual
problem [27; 28; 29]. A main difference to these works
is that we make no assumption about the underlying
problem structure and leave the general HS-tree procedure
unchanged. Instead, our aim is to avoid a full restart of the
conflict search process when constructing a new node by
looking for potentially existing additional conflicts in each
call, and to thereby speedup the overall process.
      </p>
      <p>
        Beside complete methods, a number of approximate
diagnosis approaches have been proposed in the last years,
which for example use stochastic and heuristic search [30;
31]. The relation of our work to these approaches is limited
as we are focusing on application scenarios where the goal
is to find a few first diagnoses more quickly but at the same
time maintain the completeness property. Finally, for some
domains, “direct” and SAT-based, e.g., [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ], or CSP-based,
e.g., [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ], encodings, have shown to be very efficient to find
one or a few diagnoses in recent years. For instance, [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ]
suggests an encoding scheme that first translates a given
diagnosis problem (SD, COMPS, OBS) into a CSP. Then a
specific diagnosis algorithm is applied that searches for conflict
sets with increasing cardinality, i.e., 1, 2, . . . , |COMPS|. The
same method is then used to search for diagnoses in the set
of all found conflict sets. In order to speed up the
computations the author suggests a kind of hierarchical approach
that helps the user spot the relevant components. Generally,
most of the “direct” methods require the use of additional
techniques like hierarchical diagnosis or iterative deepening
that constrain the cardinality of computed diagnoses while
computing minimal diagnoses.
      </p>
      <p>
        The concept of conflicts plays a central role in different
other reasoning contexts than Model-Based Diagnosis, e.g.,
explanations or dynamic backtracking. Specifically, in
recent years a number of approaches were proposed in the
context of the maximum satisfiability problem (MaxSAT),
see [
        <xref ref-type="bibr" rid="ref34">34</xref>
        ] for a recent survey. In these domains the
conflicts are referred to as unsatisfiable cores or Minimally
Unsatisfiable Subsets (MUSes); Minimal Correction Subsets
(MSCes) on the other hand correspond to the concept of
diagnoses in this paper. In [
        <xref ref-type="bibr" rid="ref35">35</xref>
        ] or [
        <xref ref-type="bibr" rid="ref36">36</xref>
        ], for example,
different algorithms were recently proposed to find one
solution to the MaxSAT problem, which corresponds to the
problem of finding one minimal/preferred diagnosis. Other
techniques such as MARCO [
        <xref ref-type="bibr" rid="ref29">29</xref>
        ] aim at the enumeration of
conflicts. In general, many of these algorithms use a similar
divide-and-conquer principle as we do with MXP.
However, such algorithms – including the ones listed above –
often modify the underlying knowledge base by adding
relaxation variables to clauses of a given unsatisfiable formula
and then use a SAT solver to find the relaxations. This
strategy roughly corresponds to the direct diagnoses approaches
discussed above. MXP, in contrast, acts completely
independently of the underlying knowledge representation
language. Moreover, the problem-independent decomposition
approach used by MXP is a novel feature which – to the
best of our knowledge – is not present in the existing
conflict detection techniques from the MaxSAT field.
Specifically, it allows our algorithm to find multiple conflicts more
efficiently because it searches for them within independent
small subsets of the original knowledge base. In addition,
MXP can find conflicts in knowledge bases formulated in
very expressive knowledge representation languages, such
as description logics, which cannot be efficiently translated
to SAT, see also [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ].
6
      </p>
    </sec>
    <sec id="sec-7">
      <title>Conclusions</title>
      <p>We have proposed and evaluated a novel,
general-purpose and non-intrusive conflict detection strategy called
MERGEXPLAIN, which is capable of detecting multiple
conflicts in a single call. An evaluation on various
benchmark problems revealed that MERGEXPLAIN can help to
significantly reduce the required computation times when
applied in a Model-Based Diagnosis setting in which the
goal is to find a defined number of diagnoses and in
which no assumption about the underlying reasoning engine
should be made.</p>
      <p>
        One additional property of MERGEXPLAIN is that the
union of the elements of the returned conflict sets is
guaranteed to be a superset of one diagnosis of the original
problem. Recent methods like the one proposed in [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] can
therefore be applied to find one minimal diagnosis quickly.
      </p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgements</title>
      <p>This work was supported by the Carinthian Science Fund
(KWF) contract KWF-3520/26767/38701, the Austrian
Science Fund (FWF), and the German Research Foundation
(DFG) under contract numbers I 2144 N-15 and JA
2095/41 (Project “Debugging of Spreadsheet Programs”).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <given-names>Ulrich</given-names>
            <surname>Junker</surname>
          </string-name>
          . QUICKXPLAIN:
          <article-title>Conflict Detection for Arbitrary Constraint Propagation Algorithms</article-title>
          .
          <source>In IJCAI '01 Workshop</source>
          on Modelling and
          <article-title>Solving problems with constraints (CONS-1</article-title>
          ),
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <given-names>Raymond</given-names>
            <surname>Reiter</surname>
          </string-name>
          .
          <source>A Theory of Diagnosis from First Principles. Artificial Intelligence</source>
          ,
          <volume>32</volume>
          (
          <issue>1</issue>
          ):
          <fpage>57</fpage>
          -
          <lpage>95</lpage>
          ,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <given-names>Gerhard</given-names>
            <surname>Friedrich</surname>
          </string-name>
          , Markus Stumptner, and
          <string-name>
            <given-names>Franz</given-names>
            <surname>Wotawa</surname>
          </string-name>
          .
          <source>Model-Based Diagnosis of Hardware Designs. Artificial Intelligence</source>
          ,
          <volume>111</volume>
          (
          <issue>1-2</issue>
          ):
          <fpage>3</fpage>
          -
          <lpage>39</lpage>
          ,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <given-names>Cristinel</given-names>
            <surname>Mateis</surname>
          </string-name>
          , Markus Stumptner, Dominik Wieland, and
          <string-name>
            <given-names>Franz</given-names>
            <surname>Wotawa</surname>
          </string-name>
          .
          <article-title>Model-Based Debugging of Java Programs</article-title>
          .
          <source>In Proceedings AADEBUG '00 Workshop</source>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <given-names>Dietmar</given-names>
            <surname>Jannach</surname>
          </string-name>
          and
          <string-name>
            <given-names>Thomas</given-names>
            <surname>Schmitz</surname>
          </string-name>
          .
          <article-title>Model-based diagnosis of spreadsheet programs: a constraint-based debugging approach</article-title>
          .
          <source>Automated Software Engineering</source>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Jules</given-names>
            <surname>White</surname>
          </string-name>
          , David Benavides, Douglas C. Schmidt, Pablo Trinidad, Brian Dougherty, and Antonio Ruiz Cortés.
          <source>Automated Diagnosis of Feature Model Configurations. Journal of Systems and Software</source>
          ,
          <volume>83</volume>
          (
          <issue>7</issue>
          ):
          <fpage>1094</fpage>
          -
          <lpage>1107</lpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <given-names>Kostyantyn</given-names>
            <surname>Shchekotykhin</surname>
          </string-name>
          , Gerhard Friedrich, Philipp Fleiss, and
          <string-name>
            <given-names>Patrick</given-names>
            <surname>Rodler</surname>
          </string-name>
          .
          <article-title>Interactive ontology debugging: Two query strategies for efficient fault localization</article-title>
          .
          <source>Journal of Web Semantics</source>
          ,
          <fpage>12</fpage>
          -
          <lpage>13</lpage>
          :
          <fpage>88</fpage>
          -
          <lpage>103</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>Franz</given-names>
            <surname>Baader</surname>
          </string-name>
          and
          <string-name>
            <given-names>Rafael</given-names>
            <surname>Penaloza</surname>
          </string-name>
          . Axiom Pinpointing in General Tableaux.
          <source>Journal of Logic and Computation</source>
          ,
          <volume>20</volume>
          (
          <issue>1</issue>
          ):
          <fpage>5</fpage>
          -
          <lpage>34</lpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Johan de Kleer</surname>
          </string-name>
          .
          <article-title>A Comparison of ATMS and CSP Techniques</article-title>
          .
          <source>In Proceedings IJCAI '89</source>
          , pages
          <fpage>290</fpage>
          -
          <lpage>296</lpage>
          ,
          <year>1989</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>Ulrich</given-names>
            <surname>Junker</surname>
          </string-name>
          . QUICKXPLAIN:
          <article-title>Preferred Explanations and Relaxations for Over-Constrained Problems</article-title>
          .
          <source>In Proceedings AAAI '04</source>
          , pages
          <fpage>167</fpage>
          -
          <lpage>172</lpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Ingo</surname>
            <given-names>Pill</given-names>
          </string-name>
          , Thomas Quaritsch, and
          <string-name>
            <given-names>Franz</given-names>
            <surname>Wotawa</surname>
          </string-name>
          .
          <article-title>From Conflicts to Diagnoses: An Empirical Evaluation of Minimal Hitting Set Algorithms</article-title>
          .
          <source>In Proceedings DX '11 Workshop</source>
          , pages
          <fpage>203</fpage>
          -
          <lpage>211</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Alexander</surname>
            <given-names>Feldman</given-names>
          </string-name>
          , Gregory Provan, Johan de Kleer, Stephan Robert, and Arjan van Gemund.
          <article-title>Solving Model-Based Diagnosis Problems with Max-SAT Solvers and Vice Versa</article-title>
          .
          <source>In Proceedings DX '10 Workshop</source>
          , pages
          <fpage>185</fpage>
          -
          <lpage>192</lpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Amit</surname>
            <given-names>Metodi</given-names>
          </string-name>
          , Roni Stern, Meir Kalech, and
          <string-name>
            <given-names>Michael</given-names>
            <surname>Codish</surname>
          </string-name>
          .
          <article-title>A Novel SAT-Based Approach to Model Based Diagnosis</article-title>
          .
          <source>Journal of Artificial Intelligence Research</source>
          ,
          <volume>51</volume>
          :
          <fpage>377</fpage>
          -
          <lpage>411</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Iulia</surname>
            <given-names>Nica</given-names>
          </string-name>
          , Ingo Pill, Thomas Quaritsch, and
          <string-name>
            <given-names>Franz</given-names>
            <surname>Wotawa</surname>
          </string-name>
          .
          <article-title>The Route to Success - A Performance Comparison of Diagnosis Algorithms</article-title>
          .
          <source>In Proceedings IJCAI '13</source>
          , pages
          <fpage>1039</fpage>
          -
          <lpage>1045</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>R</given-names>
            <surname>Greiner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B A</given-names>
            <surname>Smith</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R W</given-names>
            <surname>Wilkerson</surname>
          </string-name>
          .
          <article-title>A Correction to the Algorithm in Reiter's Theory of Diagnosis</article-title>
          .
          <source>Artificial Intelligence</source>
          ,
          <volume>41</volume>
          (
          <issue>1</issue>
          ):
          <fpage>79</fpage>
          -
          <lpage>88</lpage>
          ,
          <year>1989</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>Markus</given-names>
            <surname>Stumptner</surname>
          </string-name>
          and
          <string-name>
            <given-names>Franz</given-names>
            <surname>Wotawa</surname>
          </string-name>
          .
          <article-title>Diagnosing tree-structured systems</article-title>
          .
          <source>Artificial Intelligence</source>
          ,
          <volume>127</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>29</lpage>
          ,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Dietmar</surname>
            <given-names>Jannach</given-names>
          </string-name>
          , Thomas Schmitz, and
          <string-name>
            <given-names>Kostyantyn</given-names>
            <surname>Shchekotykhin</surname>
          </string-name>
          .
          <article-title>Parallelized Hitting Set Computation for Model-Based Diagnosis</article-title>
          .
          <source>In Proceedings AAAI '15</source>
          , pages
          <fpage>1503</fpage>
          -
          <lpage>1510</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Johan de Kleer</surname>
          </string-name>
          .
          <article-title>Hitting set algorithms for model-based diagnosis</article-title>
          .
          <source>In Proceedings DX '11 Workshop</source>
          , pages
          <fpage>100</fpage>
          -
          <lpage>105</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>Ulrich</given-names>
            <surname>Junker</surname>
          </string-name>
          .
          <article-title>Preference-Based Search and MultiCriteria Optimization</article-title>
          .
          <source>Annals of Operations Research</source>
          ,
          <volume>130</volume>
          :
          <fpage>75</fpage>
          -
          <lpage>115</lpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>Ingo</given-names>
            <surname>Pill</surname>
          </string-name>
          and
          <string-name>
            <given-names>Thomas</given-names>
            <surname>Quaritsch</surname>
          </string-name>
          .
          <article-title>Optimizations for the Boolean Approach to Computing Minimal Hitting Sets</article-title>
          .
          <source>In Proceedings ECAI '12</source>
          , pages
          <fpage>648</fpage>
          -
          <lpage>653</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Alexander</surname>
            <given-names>Felfernig</given-names>
          </string-name>
          , Gerhard Friedrich, Dietmar Jannach, and
          <string-name>
            <given-names>Markus</given-names>
            <surname>Stumptner</surname>
          </string-name>
          .
          <article-title>Consistency-based diagnosis of configuration knowledge bases</article-title>
          .
          <source>Artificial Intelligence</source>
          ,
          <volume>152</volume>
          (
          <issue>2</issue>
          ):
          <fpage>213</fpage>
          -
          <lpage>234</lpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>Gerhard</given-names>
            <surname>Friedrich</surname>
          </string-name>
          and
          <string-name>
            <given-names>Kostyantyn</given-names>
            <surname>Shchekotykhin</surname>
          </string-name>
          .
          <article-title>A General Diagnosis Method for Ontologies</article-title>
          .
          <source>In Proceedings ISWC '05</source>
          , pages
          <fpage>232</fpage>
          -
          <lpage>246</lpage>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Kostyantyn</surname>
            <given-names>Shchekotykhin</given-names>
          </string-name>
          , Gerhard Friedrich, Patrick Rodler, and
          <string-name>
            <given-names>Philipp</given-names>
            <surname>Fleiss</surname>
          </string-name>
          .
          <article-title>Sequential diagnosis of high cardinality faults in knowledge-bases by direct diagnosis generation</article-title>
          .
          <source>In Proceedings ECAI '14</source>
          , pages
          <fpage>813</fpage>
          -
          <lpage>818</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>Thomas</given-names>
            <surname>Eiter</surname>
          </string-name>
          and
          <string-name>
            <given-names>Georg</given-names>
            <surname>Gottlob</surname>
          </string-name>
          .
          <article-title>The Complexity of Logic-Based Abduction</article-title>
          .
          <source>Journal of the ACM (JACM)</source>
          ,
          <volume>42</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>49</lpage>
          ,
          <year>1995</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>Li</given-names>
            <surname>Lin</surname>
          </string-name>
          and
          <string-name>
            <given-names>Yunfei</given-names>
            <surname>Jiang</surname>
          </string-name>
          .
          <article-title>The computation of hitting sets: Review and new algorithms</article-title>
          .
          <source>Information Processing Letters</source>
          ,
          <volume>86</volume>
          (
          <issue>4</issue>
          ):
          <fpage>177</fpage>
          -
          <lpage>184</lpage>
          , May
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>F</given-names>
            <surname>Wotawa</surname>
          </string-name>
          and
          <string-name>
            <given-names>I</given-names>
            <surname>Pill</surname>
          </string-name>
          .
          <article-title>On classification and modeling issues in distributed model-based diagnosis</article-title>
          .
          <source>AI Communications</source>
          ,
          <volume>26</volume>
          (
          <issue>1</issue>
          ):
          <fpage>133</fpage>
          -
          <lpage>143</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>Ken</given-names>
            <surname>Satoh</surname>
          </string-name>
          and
          <string-name>
            <given-names>Takeaki</given-names>
            <surname>Uno</surname>
          </string-name>
          .
          <source>Enumerating Minimally Revised Specifications Using Dualization. InJSAI '05 Workshop</source>
          , pages
          <fpage>182</fpage>
          -
          <lpage>189</lpage>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Roni</surname>
            <given-names>Stern</given-names>
          </string-name>
          , Meir Kalech, Alexander Feldman, and
          <string-name>
            <given-names>Gregory</given-names>
            <surname>Provan</surname>
          </string-name>
          .
          <article-title>Exploring the Duality in ConflictDirected Model-Based Diagnosis</article-title>
          .
          <source>In Proceedings AAAI '12</source>
          , pages
          <fpage>828</fpage>
          -
          <lpage>834</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Mark</surname>
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Liffiton</surname>
          </string-name>
          , Alessandro Previti, Ammar Malik, and
          <string-name>
            <surname>Joao</surname>
          </string-name>
          Marques-Silva. Fast,
          <source>Flexible MUS Enumeration. Constraints</source>
          , pages
          <fpage>1</fpage>
          -
          <lpage>28</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>Lin</given-names>
            <surname>Li</surname>
          </string-name>
          and
          <string-name>
            <given-names>Jiang</given-names>
            <surname>Yunfei</surname>
          </string-name>
          .
          <article-title>Computing Minimal Hitting Sets with Genetic Algorithm</article-title>
          .
          <source>In Proceedings DX '02 Workshop</source>
          , pages
          <fpage>1</fpage>
          -
          <lpage>4</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>A</given-names>
            <surname>Feldman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G</given-names>
            <surname>Provan</surname>
          </string-name>
          ,
          <article-title>and A van Gemund</article-title>
          .
          <article-title>Approximate Model-Based Diagnosis Using Greedy Stochastic Search</article-title>
          .
          <source>Journal of Artifcial Intelligence Research</source>
          ,
          <volume>38</volume>
          :
          <fpage>371</fpage>
          -
          <lpage>413</lpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Amit</surname>
            <given-names>Metodi</given-names>
          </string-name>
          , Roni Stern, Meir Kalech, and
          <string-name>
            <given-names>Michael</given-names>
            <surname>Codish</surname>
          </string-name>
          .
          <article-title>Compiling Model-Based Diagnosis to Boolean Satisfaction</article-title>
          .
          <source>In Proceedings AAAI '12</source>
          , pages
          <fpage>793</fpage>
          -
          <lpage>799</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>Yannick</given-names>
            <surname>Pencolé</surname>
          </string-name>
          .
          <article-title>DITO: a CSP-based diagnostic engine</article-title>
          .
          <source>In Proceedings ECAI '14</source>
          , pages
          <fpage>699</fpage>
          -
          <lpage>704</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Antonio</surname>
            <given-names>Morgado</given-names>
          </string-name>
          , Federico Heras, Mark Liffiton, Jordi Planes, and
          <string-name>
            <surname>Joao</surname>
          </string-name>
          Marques-Silva.
          <article-title>Iterative and core-guided MaxSAT solving: A survey and assessment</article-title>
          .
          <source>Constraints</source>
          ,
          <volume>18</volume>
          (
          <issue>4</issue>
          ):
          <fpage>478</fpage>
          -
          <lpage>534</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <given-names>Jessica</given-names>
            <surname>Davies</surname>
          </string-name>
          and
          <string-name>
            <given-names>Fahiem</given-names>
            <surname>Bacchus</surname>
          </string-name>
          .
          <article-title>Postponing optimization to speed up MAXSAT solving</article-title>
          .
          <source>In Proceedings CP '13</source>
          , pages
          <fpage>247</fpage>
          -
          <lpage>262</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Alexey</surname>
            <given-names>Ignatiev</given-names>
          </string-name>
          , Antonio Morgado, Vasco Manquinho, Ines Lynce, and
          <string-name>
            <surname>Joao</surname>
          </string-name>
          Marques-Silva.
          <article-title>Progression in Maximum Satisfiability</article-title>
          .
          <source>InProceedings ECAI '14</source>
          , pages
          <fpage>453</fpage>
          -
          <lpage>458</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>