<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A Wiki-based Environment for Constraint-based Recommender Systems Applied in the E-Government Domain</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Stefan Reiterer</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alexander Felfernig</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Michael Jeran</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Martin Stettinger</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Manfred Wundara</string-name>
          <email>manfred.wundara@villach.at</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Wolfgang Eixelsberger</string-name>
          <email>w.eixelsberger@fh-kaernten.at</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Carinthia University of Applied Sciences</institution>
          ,
          <addr-line>A-9524 Villach, Europastra e 4</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>City of Villach</institution>
          ,
          <addr-line>A-9500 Villach, Rathausplatz 1</addr-line>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Institute for Software Technology</institution>
          ,
          <addr-line>A-8010 Graz, In eldgasse 16b</addr-line>
        </aff>
      </contrib-group>
      <abstract>
        <p>Constraint-based recommenders support customers in identifying relevant items from complex item assortments. In this paper we present WeeVis, a constraint-based environment that can be applied in di erent scenarios in the e-government domain. WeeVis supports collaborative knowledge acquisition for recommender applications in a MediaWiki-based context. This paper shows how Wiki pages can be extended with recommender applications and how the environment uses intelligent mechanisms to support users in identifying the optimal solutions to their needs. An evaluation shows a performance overview with di erent knowledge bases.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Constraint-based recommender applications help users navigating in complex
product and service assortments like digital cameras, computers, nancial
services and municipality services. The calculation of the recommendations is based
on a knowledge base of explicitly de ned rules. The engineering of the rules for
recommender knowledge bases (for constraint-based recommenders) is typically
done by knowledge engineers, mostly computer scientists [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. For building high
quality knowledge bases there are domain experts involved who serve the
knowledge engineers with deep domain knowledge [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Graphical knowledge engineering
interfaces like [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] improved the maintainability and accessability and moved the
eld one step further.
      </p>
      <p>
        Other recommendation approaches like collaborative ltering use
information about the rating behavior of other users to identify recommendations [
        <xref ref-type="bibr" rid="ref3 ref4">3, 4</xref>
        ].
Content-based ltering [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] exploits features of items for the determination of
recommendations. Compared to these approaches, constraint-based recommenders
are more applicable for complex products and services due to their explicit
knowledge representation.
      </p>
      <p>In the line of Wikipedia4 where users build and maintain Wiki pages
collaboratively we introduce WeeVis5. WeeVis is a MediaWiki6 based
environment that exploits the properties of MediaWiki and enables community based
development and maintenance of knowledge bases for constraint-based
recommenders. WeeVis is freely available as a platform and successfully applied by
four Austrian universities (in lectures about recommender systems), in the
nancial services domain and in e-government .</p>
      <p>In the e-government domain o cials as well as the community residents can
take numerous advantages of knowledge-based recommenders:
{ WeeVis can be used as an online advisory service for citizens for example
for documents that are necessary to apply for a private construction project.
The online recommendation of necessary documents in advance to on-site
appointments can lead to a time reduction for community residents and
community o cials.
{ WeeVis can be used for modeling internal processes like the signing of travel
applications for example a community o cial wants to visit a conference,
based on di erent parameters like the conference type, or if it's abroad or
in the domestic area, di erent o cials have to sign the travel request. In
WeeVis the appropriate rules for such internal processes can be mapped and
especially for new employees WeeVis recommenders can provide substantial
assistance.
{ WeeVis can be used as an information platform for example with integrated
knowledge-based recommenders for community residents e.g. to identify the
optimal waste disposal strategy for a household (this example is used as a
running example in this paper, see Section 2). Instead of providing plain text
information, like common municipality web pages, the knowledge
representation as a recommender provides an easier way for community members to
identify the optimal solution for their situation.</p>
      <p>
        A recommender development environment for single users is introduced in [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
This work is based on a Java platform and focuses on constraint-based
recommender applications for online selling. Compared to [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], WeeVis provides a
wiki-based user interface that allows user communities to develop recommender
applications collaboratively. Instead of an incremental dialog, where the user
answers one question after the other, like [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], WeeVis provides an integrated
interface where the user is free to answer questions in any order.
      </p>
      <p>
        The WeeVis interface also provides intelligent mechanisms for an instant
presentation of alternative solutions in situations where it is not possible to nd
a solution for a given set of user (customer) requirements, i.e., the requirements
are inconsistent with the recommendation knowledge base and the user is in
the need for repair proposals to nd a way out from the no solution could be
found dilemma. Model-based diagnosis [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] can be applied for the identi cation
4 www.wikipedia.org
5 www.weevis.org
6 www.mediawiki.org
of faulty constraints in a given set of customer requirements. In this context
e cient divide-and-conquer based algorithms [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] can be applied to the diagnosis
and repair of inconsistent requirements. The environment supports the user with
integrated model-based diagnosis techniques [
        <xref ref-type="bibr" rid="ref6 ref8">6,8</xref>
        ]. A rst approach to a con
ictdirected search for hitting sets in inconsistent CSP de nitions was introduced
by [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. With regard to diagnosis techniques, WeeVis is based on more e cient
techniques that make the environment applicable in interactive settings [
        <xref ref-type="bibr" rid="ref10 ref8">8, 10</xref>
        ].
      </p>
      <p>
        A Semantic Wiki-based approach to knowledge acquisition for collaborative
ontology development is introduced, for example, in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Compared to [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ],
WeeVis is based on a recommendation domain speci c knowledge
representation (in contrast to ontology representation languages) which makes the de
nition of domain knowledge more accessible also for domain experts.
      </p>
      <p>The remainder of this paper is organized as follows. In Section 2 we present an
overview of the recommendation environment WeeVis and it's application in the
e-government domain. In Section 3 we present results of a performance evaluation
that illustrates the performance of the integrated diagnosis technologies. With
Section 4 we conclude the paper.
2</p>
    </sec>
    <sec id="sec-2">
      <title>WeeVis Overview</title>
      <p>Since WeeVis is based on the MediaWiki platform, it can be installed on freely
available web servers. On the website www.weevis.org a selection of di erent
WeeVis recommenders is publicly available. For internal processes WeeVis
can be deployed in the local intranet. Standard wiki pages can be complemented
easily by recommender knowledge bases. Currently, WeeVis calculates
recommendations based on previously entered requirements. If the requirements would
result in a no solution could be found message Weevis calculates alternative
solutions based on diagnoses (see Section 2.4). In line with the Wiki idea, WeeVis
provides the ability to build knowledge bases collaboratively, a valuable feature
in e-government domain, because depending on the community department
multiple people are responsible for data management and administration.
Furthermore, WeeVis exploits the basic functionalities provided by MediaWiki and
allows rapid prototyping processes where the result of a change can immediately
be seen by simply switching from the edit mode to the corresponding read mode.
This approach allows an easy understanding of the WeeVis tags and also of the
semantics of the provided WeeVis language.
2.1</p>
      <sec id="sec-2-1">
        <title>WeeVis User Interface</title>
        <p>Since WeeVis is a MediaWiki-based environment the user interface relies on the
common Wiki principle of the read mode (see Figure 1) for executing a
recommender and the write mode (see Figure 2) for de ning a recommender knowledge
base. The development and maintenance of a knowledge base is supported a
textual fashion with a syntax that is similar to the standard Wiki syntax (see Figure
2). In the following we will present the concepts integrated in the WeeVis
environment on the basis of a working example from the e-government domain. More
speci cally we present a recommender that supports households in identifying
their optimal waste disposal strategy. In this recommendation scenario, a user
has to specify his/her requirements regarding, for example, the number of
persons living in the household or how frequently the containers should be emptied.
A corresponding WeeVis user interface is depicted in Figure 1. Requirements
are speci ed on the left hand side and the corresponding recommendations for
the optimal waste disposal plan are displayed in the right hand side.</p>
        <p>For each solution, a so-called support score is determined. If a solution ful lls
all requirements, this score is 100%, otherwise it is lower and, when clicking
on the score value, a corresponding repair action is displayed on the left-hand
side (see Figure 1). Due to the automated alternative determination, WeeVis
is always able to present a solution and users are never ending up in the no
solution could be found dilemma (see Figure 1).</p>
        <p>An example of the de nition of a (simpli ed) e-government recommender
knowledge base is depicted in Figure 2. The de nition of a recommender
knowledge base is supported in a textual fashion on the basis of a syntax similar to
MediaWiki. Basic syntactical elements provided in WeeVis will be introduced
in the next subsection.
2.2</p>
      </sec>
      <sec id="sec-2-2">
        <title>WeeVis Syntax</title>
        <p>A WeeVis recommender consists of three necessary aspects, the de nition of
questions and possible answers, items and their properties, and constraints (see
Figure 2).</p>
        <p>The de nition of an item assortment in WeeVis starts with the
&amp;PRODUCTS tag (see Figure 2). The rst line represents the attributes separated by
the exclamation mark. In our example, the item assortment is speci ed by the
name, sizep, the container size, emptyingp, the emptying frequency, and pricep,
the price of the waste disposal plan. Each of the next lines represents an item
with the values related to the attributes, in our example there are three items
speci ed: Small Plan, Medium Plan, and Large Plan.</p>
        <p>The second aspect starts with the &amp;QUESTIONS tag. In our example the
following user requirements are de ned: persons, speci es the number of persons
living in the household (one to two, three to four, more than four) and maxprice
speci es the upper limit regarding the price of the waste disposal plan.
Furthermore, emptying represents the sequence in which the dustbins will be emptied,
weekly or monthly, and container size, the preferred size of the dust container,
120 or 60.</p>
        <p>The third aspect represents the de nition of the constraints. Starting with
the &amp;CONSTRAINTS tag in WeeVis di erent types of constraints can be
dened. For the rst constraint in our example the &amp;INCOMPATIBLE keyword is
used to describe incompatible combinations of requirements. The rst
incompatibility constraint describes an incompatibility between the number of persons in
the household (persons) and the container size. For example, a waste disposal
plan with (container size) 60 must not be recommended to users who live in a
household with more than four persons. Filter constraints describe relationships
between requirements and items, for example, maxprice pricep, i.e., the price
of a waste disposal plan must be equal or below the maximum accepted price.
2.3</p>
      </sec>
      <sec id="sec-2-3">
        <title>Recommender Knowledge Base</title>
        <p>
          A recommendation knowledge base can be represented as a CSP (Constraint
Satisfaction Problem) [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ] on a formal level. The CSP has two sets of variables
V (V = U [ P ) and the constraints C = P ROD [ COM P [ F ILT where ui 2 U
are variables describing possible user requirements (e.g., persons) and pi 2 P are
describing item properties (e.g., emptyingp). Each variable vi has a domain dj
of values that can be assigned to the variable (e.g., one to two, three to four or
more than four for the variable persons). Furthermore, there are three di erent
types of constraints:
{ COM P represents incompatibility constraints of the form :X _ :Y
{ P ROD the products with their attributes in disjunctive normal form (each
product is described as a conjunction of individual product properties)
{ F ILT the given lter constraints of the form X ! Y
        </p>
        <p>The knowledge base speci ed in Figure 2 can be transformed into a constraint
satisfaction problem where &amp;QUESTIONS represents U , &amp;PRODUCTS
represents P and &amp;CONSTRAINTS represents P ROD, COM P , and F ILT . Based
on this knowledge representation WeeVis is able to determine
recommendations that take into account a speci ed set of user requirements. The results
collected are represented as unary constraints (R = fr1; r2; :::; rkg). Finally the
determined set of solutions (recommended items) is presented to the user.
2.4</p>
      </sec>
      <sec id="sec-2-4">
        <title>Diagnosis and Repair of Requirements</title>
        <p>In situations where requirements ri 2 R (unary constraints de ned on variables
of U such as emptying = monthly ) are inconsistent with the constraints in C,
we are interested in a subset of these requirements that should be adapted to be
able to restore consistency. On a formal level we de ne a requirements diagnosis
task and a corresponding diagnosis (see De nition 1).</p>
        <p>De nition 1 (Requirements Diagnosis Task). Given a set of requirements R
and a set of constraints C (the recommendation knowledge base), the
requirements diagnosis task is to identify a minimal set of constraints (the diagnosis)
that has to be removed from R such that R [ C is consistent.</p>
        <p>
          As an example R = fr1 : persons = morethanf our, r2 : maxprice = 600,
r3 : emptying = monthly; r4 : containersize = 60g is a set of requirements
inconsistent with the de ned recommendation knowledge. The recommendation
knowledge base induces two minimal con ict sets (CS) [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] in R which are CS1 :
fr1; r4g and CS2 : fr1; r3g. For these requirements we can derive two diagnoses:
1 : fr3; r4g and 2 : fr1g. For example, to achieve consistency of 1 at least
r3 and r4 have to be adapted. Such diagnoses can be determined on the basis of
a HSDAG (hitting set directed acyclic graph) (e.g. [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]).
        </p>
        <p>
          Determining con ict sets [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] at rst and afterwards constructing a HSDAG
(hitting set directed acyclic graph) to identify diagnoses tends to become
inefcient especially in interactive settings. Direct diagnosis algorithms like
FastDiag [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] reduce this two-step process to one step by calculating diagnoses
directly without con ict determination. This was the major motivation for
integrating FastDiag [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] into the WeeVis environment. Like QuickXPlain [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ],
FastDiag is based on a divide-and-conquer approach that enables the
calculation of minimal diagnoses without the calculation of con ict sets. In WeeVis
the derived diagnosis are used as a basis for determining repair actions, which
lead to the alternative solutions that are be presented to the user. A repair
action is a concrete change of one or more user requirements in R on the basis of
a diagnosis such that the resulting R0 is consistent with C.
        </p>
        <p>De nition 2 (Repair Task). Given a set of requirements R = fr1; r2; :::; rkg
inconsistent with the constraints in C and a corresponding diagnosis R
( = frl; :::; rog), the corresponding repair task is to determine an adaption
A = frl0; :::; ro0g such that R [ A is consistent with C.</p>
        <p>In WeeVis, repair actions are determined conform to De nition 2. For each
diagnosis determined by FastDiag, the corresponding solution search for
R [ C returns a set of alternative repair actions (represented as adaptation
A). In the following, all solutions that satisfy R [ A are shown to the user
(see the right hand side of Figure 1).</p>
        <p>
          Diagnosis determination in FastDiag is based on a total lexicographical
ordering of the customer requirements [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. This ordering is derived from the
sequence of the entered requirements. For example, if r1 : persons = morethanf our
has been entered before r3 : emptying = monthly and r4 : containersize = 60
then the underlying assumption is that r3 and r4 are of lower importance for
the user and thus have a higher probability of being part of a diagnosis. In our
working example 1 = fr3; r4g. The corresponding repair actions (solutions for
R 1 [ C) is A = fr30 : emptying = weekly; r40 : containersize = 120g, i.e.,
fr1; r2; r3; r4g fr3; r4g[fr30; r40g is consistent. The item that satis es R 1 [A
is fLargeP lang (see in Figure 2). The identi ed items (p) are ranked according
to their support value (see Formula 1).
        </p>
        <p>support(p) =</p>
        <p>#adaptions in A
#requirements in R
(1)
3
3.1</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Performance Evaluation</title>
      <sec id="sec-3-1">
        <title>Description of the evaluation</title>
        <p>
          We have conducted a performance evaluation with the goal to highlight the
ability of WeeVis to calculate repair actions and if no solutions could be found.
Therefore we set up an experiment with three WeeVis recommenders based
on the e-government example presented in Section 2. To illustrate the
performance of WeeVis, the knowledge base was extended and deployed with di
erent complexity regarding the number of solutions (&amp;PRODUCTS tag in
WeeVis), user requirements (&amp;QUESTIONS tag WeeVis), and constraints
(&amp;CONSTRAINTS tag WeeVis) (see Table 1). According to these three attributes the
knowledge bases were classi ed as Small, Medium, and Large. To t the
attributes of knowledge base Small from Table 1, the running example (see Figure
2) was adapted by adding one question, two products and removing the last two
constraints. The Medium and Large knowledge base are extended versions of the
running example.
To provide an optimal user experience a focus of WeeVis is to provide instant
feedback after every interaction. Interacting with a WeeVis recommender starts
with the the entering of new requirements and the subsequent calculation of
solutions for these requirements. If no solution could be found WeeVis calculates
one or more diagnoses and the complementing alternative products. With this
performance evaluation we show that WeeVis can identify at least one
alternative solution even for large knowledge bases within recommended user interface
response times [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]:
{ below 100ms, the user feels that the system reacts instantaneously
{ 1,000ms is the upper limit for keeping the users thought uninterrupted
{ 10,000ms is the upper limit for keeping the user's focus on the dialogue
        </p>
        <p>For the rst performance evaluation the goal was to measure the time needed
for calculating the corresponding solutions to given requirements. After assigning
answers to the questions for the three di erent knowledge bases, the resulting
values are depicted in Table 2. The performance values in Table 2 show that for
each of the knowledge bases WeeVis identi es solutions fast enough to provide
instantaneous feedback from the user interface. If no solution could be found
due to inconsistencies between the requirements and the knowledge base, Table
3 shows the time needed to identify at least one alternative solution on the basis
of one preferred diagnosis, Table 4 shows the time consumption of calculating
all possible solutions. WeeVis is able to calculate either one, two, three or all
diagnoses and the corresponding alternative solutions. By taking the response
time boundaries for user interfaces into account, the experiment shows that
for small and medium knowledge bases it's possible to calculate all minimal
diagnoses within acceptable response times (see Table 3). When it comes to large
knowledge bases the presented alternative solutions can be reduced to increase
the performance of the user interface instead (see Table 4).
In this paper we presented WeeVis which is an open constraint-based
recommendation environment. By exploiting the advantages of Mediawiki, WeeVis
provides an intuitive basis for the development and maintenance of
constraintbased recommender applications. The results of our experiment show that due
to the integrated direct diagnosis algorithms the WeeVis user interface provides
good the response times for common interactive settings.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Burke</surname>
          </string-name>
          .
          <source>Constraint-based Recommender Systems: Technologies and Research Issues. In 10th International Conference on Electronic Commerce, page 3. ACM</source>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Friedrich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Jannach</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Zanker</surname>
          </string-name>
          .
          <article-title>An Integrated Environment for the Development of Knowledge-based Recommender Applications</article-title>
          .
          <source>International Journal of Electronic Commerce</source>
          ,
          <volume>11</volume>
          (
          <issue>2</issue>
          ):
          <volume>11</volume>
          {
          <fpage>34</fpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>G.</given-names>
            <surname>Linden</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and J.</given-names>
            <surname>York</surname>
          </string-name>
          . Amazon.
          <article-title>com recommendations: Item-to-item collaborative ltering</article-title>
          .
          <source>Internet Computing, IEEE</source>
          ,
          <volume>7</volume>
          (
          <issue>1</issue>
          ):
          <volume>76</volume>
          {
          <fpage>80</fpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>J.</given-names>
            <surname>Konstan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Miller</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Maltz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Herlocker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Gordon</surname>
          </string-name>
          , and
          <string-name>
            <surname>J. Riedl.</surname>
          </string-name>
          <article-title>GroupLens: Applying Collaborative Filtering to Usenet News</article-title>
          .
          <source>Communications of the ACM</source>
          ,
          <volume>40</volume>
          (
          <issue>3</issue>
          ):
          <volume>77</volume>
          {
          <fpage>87</fpage>
          ,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>M.</given-names>
            <surname>Pazzani</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Billsus</surname>
          </string-name>
          .
          <article-title>Learning and Revising User Pro les: The Identi cation of Interesting Web Sites</article-title>
          .
          <source>Machine learning</source>
          ,
          <volume>27</volume>
          (
          <issue>3</issue>
          ):
          <volume>313</volume>
          {
          <fpage>331</fpage>
          ,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>R.</given-names>
            <surname>Reiter</surname>
          </string-name>
          .
          <article-title>A Theory of Diagnosis From First Principles</article-title>
          .
          <source>Arti cial intelligence</source>
          ,
          <volume>32</volume>
          (
          <issue>1</issue>
          ):
          <volume>57</volume>
          {
          <fpage>95</fpage>
          ,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>U.</given-names>
            <surname>Junker</surname>
          </string-name>
          . QUICKXPLAIN:
          <article-title>Preferred Explanations and Relaxations for Overconstrained Problems</article-title>
          .
          <source>In Proceedings of the 19th National Conference on Arti cal Intelligence (AAAI'04)</source>
          , pages
          <fpage>167</fpage>
          {
          <fpage>172</fpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Schubert</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Zehentner</surname>
          </string-name>
          .
          <article-title>An E cient Diagnosis Algorithm for Inconsistent Constraint Sets</article-title>
          .
          <article-title>Arti cial Intelligence for Engineering Design, Analysis</article-title>
          and Manufacturing,
          <volume>26</volume>
          :
          <fpage>53</fpage>
          {
          <fpage>62</fpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>R.</given-names>
            <surname>Bakker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Dikker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Tempelman</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P.</given-names>
            <surname>Wogmim</surname>
          </string-name>
          .
          <article-title>Diagnosing and Solving Over-determined Constraint Satisfaction Problems</article-title>
          .
          <source>In 13th International Joint Conference on Arti cial Intelligence</source>
          , pages
          <fpage>276</fpage>
          {
          <fpage>281</fpage>
          ,
          <string-name>
            <surname>Chambery</surname>
          </string-name>
          , France,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10. G. Friedrich.
          <article-title>Interactive Debugging of Knowledge Bases</article-title>
          .
          <source>In International Workshop on Principles of Diagnosis (DX'14)</source>
          , pages
          <fpage>1</fpage>
          <lpage>{</lpage>
          4,
          <string-name>
            <surname>Graz</surname>
          </string-name>
          , Austria,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>J. Baumeister</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Reutelshoefer</surname>
            , and
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Puppe</surname>
          </string-name>
          .
          <article-title>KnowWE: a Semantic Wiki for Knowledge Engineering</article-title>
          .
          <source>Applied Intelligence</source>
          ,
          <volume>35</volume>
          (
          <issue>3</issue>
          ):
          <volume>323</volume>
          {
          <fpage>344</fpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <given-names>A.</given-names>
            <surname>Mackworth</surname>
          </string-name>
          .
          <source>Consistency in Networks of Relations. Arti cial Intelligence</source>
          ,
          <volume>8</volume>
          (
          <issue>1</issue>
          ):
          <volume>99</volume>
          {
          <fpage>118</fpage>
          ,
          <year>1977</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Friedrich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Jannach</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Stumptner</surname>
          </string-name>
          .
          <article-title>Consistency-based Diagnosis of Con guration Knowledge Bases</article-title>
          .
          <source>Arti cial Intelligence</source>
          ,
          <volume>152</volume>
          (
          <issue>2</issue>
          ):
          <volume>213</volume>
          {
          <fpage>234</fpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <given-names>J.</given-names>
            <surname>Nielsen</surname>
          </string-name>
          . Usability engineering. Elsevier,
          <year>1994</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>