=Paper=
{{Paper
|id=Vol-1444/paper2
|storemode=property
|title=Qualitative Probabilistic Inference with Default Inheritance
|pdfUrl=https://ceur-ws.org/Vol-1444/paper2.pdf
|volume=Vol-1444
|dblpUrl=https://dblp.org/rec/conf/ki/ThornEKS15
}}
==Qualitative Probabilistic Inference with Default Inheritance==
https://ceur-ws.org/Vol-1444/paper2.pdf
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
Qualitative Probabilistic Inference
with Default Inheritance
Paul Thorn∗,a , Christian Eichhorn†,b , Gabriele Kern-Isberner† , and Gerhard Schurz∗
∗
: Institute for Philosophy, Heinrich Heine Universität Düsseldorf, Düsseldorf, Germany
†
: Department of Computer Science, Technische Universität Dortmund, Dortmund, Germany
a
:thorn@phil.hhu.de, b :christian.eichhorn@tu-dortmund.de
Abstract. There are numerous formal systems that allow inference of new con-
ditionals based on a conditional knowledge base. Many of these systems have
been analysed theoretically and some have been tested against human reasoning in
psychological studies, but experiments evaluating the performance of such systems
are rare. In this article, we extend the experiments in [19] in order to evaluate the
inferential properties of c-representations in comparison to the well-known Sys-
tems P and Z. Since it is known that System Z and c-representations mainly differ
in the sorts of inheritance inferences they allow, we discuss subclass inheritance
and present experimental data for this type of inference in particular.
1 Introduction
There are systems of conditional reasoning (such as Adams’ System P [2]) that can be
used to make valid (i.e., truth preserving) inferences about conditional probabilities.
More generally, there are systems of conditional reasoning where it is plausible to adopt
a probabilistic interpretation of conditionals, where conditionals of the form (ψ|φ) are
interpreted as expressing that the corresponding conditional probability, P (ψ|φ), is high.
In some cases, it may be plausible to adopt a probabilistic interpretation of conditionals,
for a given system, even when the inferences licensed by the respective system are
ampliative, and not truth preserving, given the probabilistic interpretation. For example,
although inheritance inference (i.e., from (ψ|φ) infer (ψ|φ∧χ)) may fail to preserve
high probability in many cases, inheritance inference is a reasonable form of inference
that one might like to codify within a system of conditional reasoning.
In the present paper, we compare and evaluate the behaviour of two systems of
conditional reasoning that are stronger than System P, but admit of a probabilistic
interpretation, namely: System Z [16], and System MinC (which we define based on the
inductive method of c-representations [8,9]). The two systems are of interest, since they
both license a number of desirable inference patterns, such as inheritance inference and
contraposition, that are not licensed by System P. Nevertheless the two systems differ in
some important respects, such as in their treatment of inheritance reasoning.
Within a system where conditionals are treated as expressing defaults, it is desirable
that subclass inheritance among defaults be licensed, defeasibly. For example, from the
default that birds usually can fly we would like to infer that crows (a subclass of birds) are
usually capable of flight, in the case where we have no background knowledge indicating
16
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
that crows are exceptional birds. Such inferences are assumed to be defeasible, meaning
that there are conditions under which such inferences are defeated (i.e., conditions under
which the inference is not licensed).
Beyond defeasible subclass inheritance, it is controversial whether inheritance in the
case of exceptional subclasses should be licensed, defeasibly [6]. For example, notice
that penguins are exceptional birds inasmuch as they lack the capacity of flight. Given
that penguins represent an exceptional subclass of the class of birds, it is controversial
whether the subclass, penguins, should inherit other characteristics typical of birds.
For example, assuming that birds usually have wings, it is controversial whether it
is reasonable to infer that penguins usually have wings, given that they are (usually)
incapable of flight.
A principal difference between System Z and System MinC is that the latter, and not
the former, permits inheritance inference in the case of exceptional subclasses. Prima
Facie, this fact speaks in favor of System MinC, a point which we briefly discuss in
Section 5. However, as our primary means of evaluation, our paper reports the results
of experiments which test the behaviour of Systems Z and MinC in reasoning about a
simulated stochastic environment. For additional perspective, we also tested the behavior
of System P and System QC [19,23]. The results show that while System MinC makes
many inferences that are not drawn by System Z, System Z rarely makes an inference
that is not drawn by System MinC. Since the two systems are both ampliative with
respect to the probabilistic interpretation of conditionals (in contrast to System P), it is
clear that the conclusions drawn by Systems Z and MinC are more risky than the ones
drawn by System P. It is also plausible to think that conclusions that are drawn by System
MinC and not System Z are more risky than the conclusions that are drawn by both
systems, since such conclusions go “farther out on a limb”. The results presented here
vindicate this thought, and provide a clearer picture of just how risky these inferences
are.
The paper is organised as follows: After introducing the necessary formal preliminar-
ies in Section 2, we introduce Systems P, Z, and QC in Section 3. We define System MinC
via c-representations in Section 4. In Section 5 we discuss subclass inheritance for excep-
tional subclasses. We present the experimental setup and the results of the experiments
in Sections 6 and 7, and conclude in Section 8.
2 Preliminaries
Let Σ = {V1 , ..., Vm } be a propositional alphabet where a literal is a variable V
interpreted to true (v) or false (v). From these we obtain the propositional language L
as the set of formulas of Σ closed under negation ¬, conjunction ∧, and disjunction ∨,
as usual; for shorter formulas, we abbreviate conjunction by juxtaposition (i.e., ab is
equivalent to a ∧ b), and negation by overlining (i.e., a is equivalent to ¬a). We write the
material implication as φ → ψ which is, as usual, equivalent to φ ∨ ψ. Interpretations or
possible worlds are also defined in the usual way; the set of all possible worlds is denoted
by Ω. We often take advantage of the 1-1 association between worlds and complete
conjunctions, i.e., conjunctions of literals where every Vi ∈ Σ appears exactly once.
17
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
Table 1. Evaluation of conditionals in the penguin example (+ indicates verification, − falsifica-
tion, an empty cell inapplicability) (above). Two OCF for the penguin example (below).
pbfw pbf w pbf w pbf w pbfw pbf w pb f w pb f w pbfw pbf w pbf w pbf w pbfw pbf w pb f w pb f w
(f |b) + + − − + + − −
(f |p) − − + + − − + +
(b|p) + + + + − − − −
(w|b) + − + − + − + −
κZΔ (ω) 2 2 1 1 2 2 2 2 0 1 1 1 0 0 0 0
�
κcΔ (ω) 2 3 1 2 4 4 2 2 0 1 1 2 0 0 0 0
A conditional (ψ|φ), φ, ψ ∈ L, is trivalent, with the evaluation: (ψ|φ) is verified iff
ω |= φψ, (ψ|φ) is falsified iff ω |= φψ, and (ψ|φ) is inapplicable iff ω |= φ [5,8]. A
finite set of conditionals Δ = {(ψ1 |φ1 ), . . . , (ψn |φn )} is called a knowledge base.
An Ordinal Conditional Function (OCF, ranking function [21,20]) is a function
Ω → N0 ∪ {∞} that assigns to each world an implausibility rank, such that κ−1 (0),
the preimage of 0, is non-empty. The rank of a formula ψ ∈ L is the rank of the lowest
ranked world that satisfies the formula, formally: κ(φ) = minω|=φ {κ(φ)}. The rank of a
conditional (ψ|φ) is defined as: κ(ψ|φ) = κ(φψ) − κ(φ). A ranking function accepts a
conditional (ψ|φ) (written κ |= (ψ|φ)) iff κ(φψ) < κ(φψ); κ is admissible with respect
to a knowledge base Δ if and only if κ |= (ψ|φ) for all (ψ|φ) ∈ Δ.
Example 1. We illustrate these preliminaries with the well-known penguin example. Let
B indicate whether something is a bird (b) or not (b), let P indicate whether something
is a penguin (p) or not (p), let F indicate whether something is capable of flying (f ) or
not (f ), and let W indicate whether something has wings (w) or not (w). This gives us
the alphabet Σ = {P, B, F, W } with a set of worlds given in the top row of Table 1. We
use the conditionals “birds usually can fly” (f |b), “penguins usually cannot fly” (f |p),
“penguins usually are birds” (b|p), and “birds usually have wings” (w|b) to compose
the knowledge base Δ = {(f |b), (f |p), (b|p), (w|b)}. Table 1 displays the evaluation of
these conditionals within the worlds ω ∈ Ω. Table 1 also displays two ranking functions,
�
κΔZ
and κcΔ that are admissible with respect to this knowledge base. We discuss the
two ranking functions, especially how they are generated inductively from the above
knowledge base, later in the paper.
3 Overview of Systems P, Z, and QC
As described in [7], System P represents the confluence of a number of different semantic
criteria. One feature of System P that is of interest here is its connection with the
following consequence relation (cf. [2]):
Improbability-Sum Preservation: (ψ1 |φ1 ), ..., (ψn |φn ) |=i.s.p. (ξ|χ) iff for all probabil-
ity functions, P , over the appropriate language: I(ξ|χ) ≤ Σi=1
n
I(ψi |φi ), where I(ψ|φ),
the improbability of ψ given φ, is defined as 1 − P (ψ|φ).
As Adams [2] demonstrated, the following calculus (denoted by �P ) is correct and
complete for |=i.s.p. :
18
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
(REF) (reflexivity) �P (ψ|φ)
(LLE) (left logical equivalence) if |= (φ → ψ)∧(ψ → φ), then (χ|φ) �P (χ|ψ)
(RW) (right weakening) if |= φ → ψ, then (φ|χ) �P (ψ|χ)
(CC) (cautious cut) (ψ|φ), (χ|ψ∧φ) �P (χ|φ)
(CM) (cautious monotony) (ψ|φ), (χ|φ) �P (χ|ψ∧φ)
(AND) (φ|χ), (ψ|χ) �P (φ∧ψ|χ)
(OR) (χ|φ), (χ|ψ) �P (χ|ψ∨φ)
System P also has a semantics expressible in terms of ranking functions. In particular,
Δ �P (ψ|φ) if and only if every ranking function that is admissible for Δ accepts
(ψ|φ) [1,13]. Apart from being characterized by reasonable (if mininal) principles, and
plausible semantic theories, empirical studies show that human reasoning makes use of
the principles of System P (c.f. [17,11]), which renders the study of System P especially
worthwhile.
Inference by System Z [16] is based upon the unique ranking function κZ Δ , among
the admissible ranking functions for (consistent) Δ = {(ψ1 |φ1 ), . . . , (ψn |φn )}, that
minimizes the rank of each world in the set of possible worlds ΩΔ defined over the
propositional atoms appearing in Δ. This is achieved by forming �m an ordered partition
(Δ0 , ..., Δm ) of Δ, where each Δi is the maximal subset of j=i Δj that is tolerated
�m
by j=i Δj (where a conditional, (ψ|φ), is tolerated by a set of conditionals, Δ, iff
∃ω: ω |= φψ and ∀(ψi |φi ) ∈ Δ: ω |= φi → ψi ). Due to maximality, such partitions are
unique for every Δ. Given the respective partition, κZ
Δ is defined as the OCF that assigns
the value 0 to a world, ω, if no elment of Δ falsified at ω, and otherwise assigns the
value i + 1, where i is index of the rightmost element of (Δ0 , ..., Δm ) that contains a
conditional falsified by ω. Table 1 (above) presents κZ
Δ for the knowledge base described
in Example 1. Inference by System Z is characterized by the relation �Z , which is
defined in terms of the conditionals accepted by κZ Δ:
Δ �Z (ψ|φ) iff κZ
Δ |= (ψ|φ). (System Z)
By adding the rule Monotony, i.e., (ψ|φ) implies (ψ|φ ∧ χ), to System Z (or merely
to System P), we obtain System QC. We here follow [19], and implement System QC by
reasoning with conditionals as if they were material implications, and define System QC
as follows:
Δ �QC (ψ|φ) iff { φi → ψi | (ψi |φi ) ∈ Δ } |= φ → ψ (System QC)
4 System MinC
System MinC is defined in terms of ranking functions known as c-representations [8,9].
A c-representation assigns an individual impact value κ− i ∈ N0 to each conditional
(ψi |φi ) ∈ Δ. Using these impact values, a ranking function, κΔ
c
, is defined, where each
world ω is assigned the rank κΔ (ω), which is the sum of the impacts of the conditionals
c
falsified by ω:
�
c
κΔ (ω) = κ−
i . (1)
i:ω|=φi ψ i
19
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
The impacts of the conditionals are chosen so that κΔ
c
|= Δ, which is the case if
� � � � � �
κ−i > min κ −
j − min κ −
j . (2)
ω|=ψi φi ω|=ψi φi
j:ω|=φj ψ j j:ω|=φj ψ j
i�=j i�=j
Entailment with respect to a c-representation is defined, as usual, via the OCF κΔ
c
.
c
κΔ |= (ψ|φ) iff c
κΔ c
(φψ) < κΔ (φψ). (3)
Example 2. We illustrate c-representations with the penguin example (Example 1).
Table 1 shows the verification/falsification behaviour of the worlds and the conditionals
in this example, where (2) gives us:
κ− − − − − − −
1 = min{κ2 , κ2 + κ4 , 0, κ4 } − min{0, κ4 , 0, κ4 }
κ− − − − − − − − −
2 = min{κ1 , κ1 + κ4 , κ3 , κ3 } − min{0, κ4 , κ3 , κ3 }
κ− − − − − − − − −
2 = min{κ2 , κ2 + κ4 , κ1 , κ1 + κ4 } − min{κ2 , κ2 , 0, 0}
κ− − − − − − −
1 = min{κ2 , κ1 , 0, κ1 } − min{κ2 , κ1 , 0, κ1 }
This can be solved via the minimal solution κ− − − −
1 = 1, κ2 = 2, κ3 = 2, κ4 = 1 , which,
c�
with (1) gives us the c-representation κΔ shown in Table 1.
The defining system (2) is a system of inequalities. The system defines a schema for
all c-representations of a given knowledge base Δ rather than a unique ranking function
for Δ. To apply the method of c-representations to define a system of conditional
infererence, we introduced an algorythm for selecting a unique c-representation for
each knowledge base. We call the resulting system “MinC” (minimal c-representation).
Following the idea of System Z being the pareto-minimal ranking function admissible to
a knowledge base Δ, we define System MinC via a minimal c-representation that assigns
the smallest possible rank to each world. Since there are no straightforward criteria for
identifying a unique minimal c-representation, we opted for the following hierarchy of
criteria (cf. [15]):
�
(a) minimising the combined rank ω∈Ω κ(ω),
(b) minimising the maximal rank max� ω∈Ω {κ(ω)},
n
(c) minimising the combined impacts i=1 κ− i , and
(d) minimising the maximal impact max1≤i≤n {κ− i }.
Most of the time, these criteria, in this order, select a minimal c-representation within
one or two steps.
To determine our designated minimal c-representation, we order c-representations by
(a), the ones indistinguishable by (a) are then ordered by (b), the ones indistinguishable
by (b) are then ordered by (c), followed by (d). Since ordering by (a) through (d)
does not always yield a unique minimal c-representation, we implemented a practical
measure for identifying our designated c-representation as that c-representation having
the lexicographically smallest vector (κ− 1 , ..., κn ) among the minimal solutions ordered
−
by (a) though (d). To distinguish this uinque c-representation from the general κΔ c
, we
20
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
call the c-representation that is chosen according to the preceding tests, for respective Δ,
�
κcΔ , and define a corresponding inference system as follows:
�
Δ �M inC (ψ|φ) iff κcΔ |= (ψ|φ). (System MinC)
Note that while their is no known axiomatic characterization of �M inC or �Z , both
satisfy all of the principles that characterize �P . In addition, both �M inC and �Z satisfy
rational monotony [12]: from (ψ|φ) and the non-validity of (χ|φ) infer (ψ|φχ).
5 Exceptionality and subclass inheritance
A principal difference between System Z and System MinC is that the latter, and not the
former, permits inheritance inference in the case of exceptional subclasses. This fact is
c�
illustrated by the ranking functions κZ Δ (ω) and κΔ (ω) of Table 1, concering Example 1.
In this case, System minC permits the conclusion that (w|p), whiles System Z does not.
Prima Facie, this behavior speaks in favor of System MinC. Indeed, the range of possible
inheritance inferences to exceptional subclasses is very broad – broader than generally
recognized – and encompasses many inferences that are generally, and correctly, regarded
as reasonable. As a consequence, it appears that abandoning inheritance inference to
exceptional subclasses, as a default, would forsake too much, i.e., too many reasonable
inferences. Systems that do abandon these inferences are described of having a Drowning
Problem [4].
The fact that a prohibition of inheritance inference to exceptional subclasses would
forsake too much can be seen by considering a range of typical inheritance inferences,
where the relevant subclass represents a small proportion of the respective superclass.
For example, suppose it is given that (f |b) (birds are usually able to fly), and we
would like to infer (f |jb) (j-birds are usually able to fly). Assume that we possess no
special information regarding the class j, save that j corresponds to a relatively small
(or improbable) subclass of b. In that case, we are in a position to conclude that j is
exceptional relative to b, since we are in a position to accept (j|b). But it is clear that
the proposed inference should be permitted. Indeed, the proposed inference is no less
reasonable than the most reasonable instances of inheritance reasoning. Moreover, the
fact that j corresponds to a small subclass of b does not speak against the inference. The
latter point is particularly important when we consider cases of classical direct inference,
where inheritance reasoning is used in order to draw a conclusion about a particular
individual (see [18,3]).
6 Experiments
We here extend the experiments conducted in [19], with the aim of evaluating the
performance of System MinC in comparison to System Z. To make the search space
manageable, we restricted the experiments to an alphabet Σ = {A, B, C, D} with a
language L∧ restricted to conjunctions of literals. The language of conditionals (L∧ |L∧ )
is further restricted so that no variable may appear in both the antecedent and the
21
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
consequent of a conditional. This means that (b|a) and (cd|ab) are in (L∧ |L∧ ), but
(bcd|ab) is not.
To generate a stochastic environment, we randomly assigned values from the real-
valued interval [0, 1] to the probabilities: (a|�),(b|ȧ), (c|ȧḃ), and (d|ȧḃċ) with ȧ ∈
{a, a}, ḃ ∈ {b, b}, and ċ ∈ {c, c}. We then generated the probability distribution
P : Ω → [0, 1] by the so called “chain rule”. Based on this distribution, four conditionals
(ψ|φ) with P (ψ|φ) ≥ mp (the minimum probability of the conditionals in the knowledge
base for the respective simulation) were chosen randomly from (L∧ |L∧ ). Given this
knowledge base Δ, the sets of all entailed conditionals C X (Δ) = {(ψ|φ)|(ψ|φ) ∈
(L∧ |L∧ ), Δ �X (ψ|φ)}, for X ∈ {P, Z, MinC, QC}, were computed. The restriction of
our simulations to cases where the systems are provided with four premise conditionals
expressed within (L∧ |L∧ ) partly limits the scope of our results. For some explanation
concerning why these limitations are not so significant, see [24].
The accuracy of the inferences drawn by the four systems was assessed by treating
the systems as asserting that the probability of the inferred conditional was at least the
sum of the improbabilities of the premises upon which the inference was based. This
amounts to treating the systems as licensing inference to inferred lower probability
bounds. According to the present assumption, the precise bound licensed by a respective
system, X, relative to a given knowledge base, Δ, and a probability function, P , is as
follows, where Δ� ranges over the subsets of Δ such that Δ� �X (ψ|φ):
� � �
X(ψ|φ) = max
�
1− (1 − P (ψi |φi )) . (4)
Δ ⊆Δ
(ψi |φi )∈Δ�
While the present assumption is ‘correct’ in the case of System P, it may lead to
overestimation when applied to the other three systems. Precisely, we say that inference
made by a system counts as an overestimation, whenever X(ψ|φ) > P (ψ|φ). For the
moment, we will proceed as if it is reasonable to evaluate the accuracy of inferred
conditionals in the present manner, bearing in mind that any charge of “overestimation”
is based on the assumption that it is correct to propogate lower probability bounds in the
manner of improbability sums. In the conclusion of the paper, when we consider what to
make of our experminental results, we will briefly revisit this assumption.
Beyond attending to cases where a respective system overestimates respective condi-
tional probabilities, our interest is in comparing the accuracy of the bounds licenced by
the Systems Z and MinC. Unfortunately, there are no established and uncontroversial
measures for scoring the accuracy of lower probability bounds. For this reason, we
report the results of a scoring method that has a principled motivation and is pertinent to
assessing accuracy, namely, the advantage-compared-to-guessing measure (ACG) [19]:
1
ACG(X(ψ|φ), P ) = − |P (ψ|φ) − X(ψ|φ)|. (ACG)
3
The idea behind this measure derives from the fact that the mean difference between
two random choices of real values r and s from the unit interval is, provably, 13 . This
means that the ‘strategy’ of setting lower probability bounds by randomly choosing
numbers in [0, 1] is expected to yield an ACG score of zero, on average (assuming that the
22
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
true probabilities are also selected randomly from [0, 1]). Reporting ACG scores, rather
than the linear distance of inferred bounds from the true probabilities, has heuristic value,
since the measure assigns positive scores to judgments that are ‘better than guessing’,
and negative scores to judgments that are ‘worse than guessing’. Given the appearance
of X in the calculation of ACG scores, we once again observe that our proposed
evaluation assumes that it is correct to propogate lower probability bounds in the manner
of improbability sums.
7 Experimental results
The results presented here, regarding systems P, Z, and QC, are similar to those presented
in [19]. The results of this paper are novel inasmuch they permit a comparison of the
performance of Systems Z and MinC. All tested systems do satisfy certain quality criteria,
as noted in Sections 3 and 4, and hence the inferences drawn are sensible with respect to
those criteria.
Table 2 presents the number of inferences made by each of the four systems over the
course of 5,000 simulations, for each of the listed values of mp (the minimum probability
of the conditionals in the knowledge base). Table 2 illustrates that System MinC permits
more inferences than System Z, while both systems permit quite a few more inferences
than System P, and far fewer inferences than System QC. It may also be observed that
the difference between the number of System MinC and System Z inferences decreases
with increases in the value of mp. Indeed, if we exclude those inferences that are
made by System P, then we see that System MinC licenses about 10% more inferences
than System Z, when mp = 0.5. At mp = 0.99, System MinC licenses about 5% more
inferences than System Z. At present, we cannot say whether the behavior of System Z
and System MinC converge as mp goes to 1.
Every inference licensed by System P is included in each of the other systems. On
the other hand, it has been demonstrated that the set of inferences licensed by a minimal
c-representation does not generally include those licensed by System Z, and similarly
the set of inferences licensed by System Z does not generally include those licensed by
a minimal c-representation [10]. Our experiments expand upon this finding, showing
that although there are inferences that are licensed by System Z that are not licensed by
System MinC, such inferences are rare. Indeed, in addition to licensing more conclusions
than System Z, the set of conclusions licensed by System MinC frequently includes the
set of conclusions licensed by System Z, as presented in the right most column of Tbl 2.
Example 3. To show that System Z and System MinC are different in general we
use an Example from [10]. By applying System Z and System MinC to the knowl-
dege base Δ = {(a|b), (a|c), (b|c), (d|b)}, we obtain that ((cb ∨ cb) ∧ ad) �Z cb and
((cb ∨ cb) ∧ ad) �MinC cb, whereas cbd �MinC a and cbd �Z a.
Table 3 shows that both System Z and MinC are somewhat prone to overestimation,
which characterizes the majority of System Z and MinC inferences when the value of
mp is high. Table 3 also shows that the inferences made by System MinC tend to be
less accurate than those of System Z, as measured by the ACG measure. This fact is
partially obscured by the fact that the sets of inferences made by systems Z and MinC
23
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
Table 2. Total number of inferred conditionals.
Number of inferred conditionals Cases where
mp
P Z MinC QC |Z ∩ MinC| |Z\MinC| |MinC\Z| Z⊆MinC
0.5 65 777 258 400 278 366 612 815 257 671 729 20 695 4 535
0.6 51 368 232 926 249 612 508 811 232 404 522 17 208 4 627
0.7 39 354 206 108 218 756 423 102 205 783 325 12 973 4 744
0.8 29 899 175 412 184 751 338 832 175 201 211 9 550 4 813
0.9 24 296 133 602 139 197 218 434 133 566 36 5 631 4 965
0.99 20 690 74 368 76 000 92 904 74 368 0 1 632 5 000
Table 3. Aggregate ACG scores and number of overestimations.
Aggregate ACG scores Overestimations
mp
P Z MinC QC Z MinC QC
0.5 9 413.7 31 385.6 33 019.9 33 880.1 94 512 106 767 368 061
0.6 10 556.5 32 564.0 33 852.3 24 691.5 98 270 109 754 329 044
0.7 10 078.2 30 887.3 31 715.8 15 609.1 101 240 110 764 294 497
0.8 8 973.8 27 711.8 28 144.2 7 497.0 100 573 108 520 255 494
0.9 7 888.6 22 808.0 22 823.5 8 987.9 90 520 95 958 174 652
0.99 6 893.0 19 076.3 19 324.0 16 015.8 56 659 58 291 75 195
both include the inferences made by System P, and by the fact that the set of System Z
inferences is ‘practically’ included in the set of System MinC inferences. In order to
present a clearer picture, Table 4 presents the average ACG score earned for individual
inferences made by System P, inferences made by System Z that were not made by
System P (Z\P), inferences made by System MinC that were not made by System Z
(MinC\Z), and inferences made by System QC that were not made by System MinC
(QC\MinC). Here we see that inferences proper to System MinC (MinC\Z) earned
positive ACG across all values of mp, as with the inferences proper to System Z (Z\P),
and unlike the inferences proper to System QC (QC\MinC).
Finally, since one of our primary concerns was to assess the reasonableness of
inheritance inference in the case of exceptional subclasses, we compared the accuracy
of the inheritance inferences licensed by System Z (which only involve unexceptional
subclasses) with the accuracy of the inheritance inferences licensed by System MinC
(which may involve exceptional subclasses). As a means of assessment, we counted
an inference to a conditional (ψ|φ) as inferred by inheritance from a given premise set
if and only if (i) (ψ|φ) was neither a member of the premise set nor inferred from the
premise set by System P, and (ii) some conditional (ψ|ξ) was also inferred, where φ |= ξ
and ξ �|= φ. Information regarding such inferences is recorded in Table 5. Here we see
that inheritance inferences make up the majority of the inferences licensed by Systems Z
and MinC that are not also licensed by System P, ranging from just over 75% of the total
inferences, for mp = 0.5, to just over 50% of the total inferences, for mp = 0.99. It also
follows from results of Table 5 that the accuracy of the inheritance inferences licensed
by System Z, as measured the average ACG scores per inference, is nearly identical to
24
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
Table 4. Mean ACG scores per inference.
mp P Z\P MinC\Z QC\MinC
0.5 0.143 0.114 0.082 0.003
0.6 0.206 0.121 0.077 −0.035
0.7 0.256 0.125 0.066 −0.079
0.8 0.300 0.129 0.046 −0.134
0.9 0.325 0.136 0.003 −0.175
0.99 0.333 0.227 0.152 −0.196
Table 5. Number of inheritance inferences and their aggregate ACG scores.
Number of inferences Aggregate ACG scores
mp
Z MinC QC Z MinC QC
0.5 145 763 163 532 384 009 15 827.1 17 276.2 20 253.2
0.6 134 338 149 252 314 733 16 587.1 17 749.1 15 043.7
0.7 120 407 131 738 259 069 16 103.8 16 867.5 10 200.4
0.8 102 610 111 088 204 322 14 992.7 15 395.3 6 017.1
0.9 72 671 77 896 120 005 11 752.3 11 770.0 6 291.0
0.99 27 021 28 649 35 261 6 758.9 7 006.0 6 315.4
the accuracy of the non-inheritance inferences among Z\P. Similarly, the accuracy of
the inheritance inferences licensed by System MinC, and not by System Z, is nearly
identical to the accuracy of the non-inheritance inferences among MinC\Z.
In summary, the results of our simulations are as follows (where accuracy claims
assume the correctness of probability propogation by improbability sums):
1. System MinC licensed significantly more inferences than System Z, with a decreas-
ing margin proportional to the value of mp.
2. While neither System Z nor System MinC strictly includes the other (as shown
in [10]), the set of System Z inferences was a subset of the set of System MinC
inferences within a vast majority of our simulations.
3. The accuracy of inferences licensed by System MinC was somewhat less than the
accuracy of inferences drawn by System Z. We also observed that the accuracy of
System MinC inferences tended to decrease with increasing values of mp (excluding
the case where mp = 0.99, whose exceptionality is discussed at length in [19, § 2.5]).
4. The accuracy of the inheritance inferences licensed by System Z was nearly identical
to that of the other inferences licensed by System Z that were not licensed by Sys-
tem P. Similarly, the accuracy of the inheritance inferences licensed by System MinC
was nearly identical to that of the other inferences licensed by System MinC that
were not licensed by System Z.
8 Conclusion
Our results show that for practical purposes, System MinC represents a stronger system
of inference than System Z. Our results also show that inference by System MinC (and
25
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
inheritance for exceptional subclasses, as licensed by System MinC) is more risky than
inference by System Z. These results accord with existing theoretical analyses of the
systems studied here [12,14,10]. Indeed, as measured by the type of monotony that
characterize the systems, we see that inferential strength increases as we proceed from
System P to Systems Z and MinC, and finally to System QC: cautious monotony holds
for System P, rational monotony holds for Systems Z and MinC, and “full” monotony
holds for System QC. As measured by the type of subclass inheritance supported by
the systems, we see that inferential strength increases as we proceed from System P
to System Z to System MinC and finally to System QC: no inheritance inference is
permitted in System P, inheritance inference in the case of unexceptional subclasses is
permitted in System Z, defeasible inheritance for exceptional subclasses is permitted in
System MinC, and unrestricted inheritance inference is permitted in System QC. Our
experimental results show that increasing inferential strength, as described, comes at the
risk of decreased accuracy. Assuming the risk associated with such inferential strength
is too high in the case of System QC (as argued in [19,22,23]), the question remains of
whether inference by System MinC should be favored over inference by System Z.
While inference by System MinC carries greater risk than inference by System Z,
the same claim can be made in comparing inference by System Z to inference by System
P. In the latter case, the riskiness of inference by System Z appears to be small enough,
so that inference by System Z should be preferred to inference by System P (as argued
in [19,22,23]), or better: One should perform the inferences licensed by System Z in
addition to those licensed by System P. Assuming such arguments are cogent in the
case of System Z, are similar arguments cogent in the case of System MinC? In other
words, should one perform the inferences licensed by System MinC in addition to those
licensed by System Z? While we grant that the risks (of overestimation and inaccurate
judgment) are greater in the case of System MinC (in comparison to System Z), we
also observe that inference by System MinC generally yields positive accuracy scores
according to the ACG measure, in the case where probability propogation is determined
by improbability sums.
In addition to evaluating the performance of System MinC, we were keen to evaluate
the accuracy of inheritance inference in the case of exceptional subclasses. In Section 5,
we offered conceptual reasons for rejecting a blanket prohibition of such inferences.
Our argument there proceeded from the fact that the class of inheritance inferences to
exceptional subclasses is very broad and encompasses many inferences that are generally,
and correctly (we maintain), regarded as reasonable. Of course, we do not endorse the
wholesale adoption of all inheritance inferences, which would be tantamount to reasoning
in accordance with System QC. Our hope is rather that there is some systematic way to
move beyond System Z, and a blanket prohibition of inheritance inference in the case
of exceptional subclasses. Our motivation for evaluating the performance of System
MinC experimentally was to determine whether inference by System MinC might serve
as an appropriate means of moving beyond System Z. As things stand (and for the
reasons adduced in the preceding paragraphs), we think that inference by System MinC
represents a promising option.
Finally, it should be mentioned, once again, that the overestimations and accuracy
scores attributed to the studied systems are premised on treating the systems as inferring
26
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
lower probability bounds in accordance with (4), above. We think that application of (4)
is reasonable, since such inferences are valid (i.e., guarenteed to be truth preserving)
for System P, and the other systems represent incremental strengthenings of System P.
Moreover, the fact that such inferences are invalid in the case of Systems Z, MinC, and
QC, is not a decisive objection to the proposed application of (4), since these three
systems all license inheritance inference, for which there is never a guarantee that
high premise probability is preserved (i.e., ∀φ, ψ, χ, r: (r < 1 and φ � χ) ⇒ (∃P :
P (ψ|φ) = r and P (ψ|φ ∧ χ) = 0). Nevertheless, while applying (4) yields a plausible
means of evaluating the four systems, there are certainly possible alternatives. Exploring
such alternatives is an object of present and future research.
Acknowledgment: We thank the anonymous referees for their valuable suggestions that
helped us improve the paper. This work was supported by DFG-Grant KI1413/5-1 of Prof.
Dr. Gabriele Kern-Isberner, and DFG-Grant SCHU1566/9-1 of Prof. Dr. Gerhard Schurz,
both as part of the priority program “New Frameworks of Rationality” (SPP 1516).
References
1. Adams, E.: A note on comparing probabilistic and modal logics of conditionals. Theoria 43,
186–194 (1971)
2. Adams, E.: The Logic of Conditionals: An Application of Probability to Deductive Logic.
Synthese Library, Springer (1975)
3. Bacchus, F.: Representing and reasoning with probabilistic knowledge. MIT Press, Cambridge,
Massachusetts (1990)
4. Benferhat, S., Cayrol, C., Dubois, D., Lang, J., Prade, H.: Inconsistency management and
prioritized syntax-based entailment. In: Proceedings of the International Joint Conferences on
Artificial Intelligence. pp. 640–647 (1993)
5. De Finetti, B.: Theory of Probability, vol. 1,2. John Wiley and Sons, New York, USA (1974)
6. Geffner, H., Pearl, J.: Conditional entailment: Bridging two approaches to default reasoning.
Artificial Intelligence 53(2–3), 209–244 (1992)
7. Hawthorne, J., Makinson, D.: The quantitative/qualitative watershed for rules of uncertain
inference. Studia Logica 86(2), 247–297 (2007)
8. Kern-Isberner, G.: Conditionals in Nonmonotonic Reasoning and Belief Revision – Consider-
ing Conditionals as Agents. No. 2087 in LNCS, Springer (2001)
9. Kern-Isberner, G.: A thorough axiomatization of a principle of conditional preservation in
belief revision. Annals of Mathematics and Artificial Intelligence 40, 127–164 (2004)
10. Kern-Isberner, G., Eichhorn, C.: Structural inference from conditional knowledge bases.
In: Unterhuber, M., Schurz, G. (eds.) Studia Logica Special Issue Logic and Probability:
Reasoning in Uncertain Environments, pp. 751–769. No. 102 (4), Springer Science+Business
Media, Dordrecht, NL (August 2014)
11. Kuhnmünch, G., Ragni, M.: Can Formal Non-monotonic Systems Properly Describe Human
Reasoning? In: Proceedings of the Cognitive Science Conference (COGSCI2014). pp. 1806–
1811 (2014)
12. Lehmann, D., Magidor, M.: What does a conditional knowledge base entail? Artificial Intelli-
gence 55(1), 1–60 (1992)
13. Lewis, D.: Counterfactuals. Blackwell, Oxford, UK (1973)
27
Proceedings of the KI 2015 Workshop on Formal and Cognitive Reasoning
14. Makinson, D.: General patterns in nonmonotonic reasoning. In: Gabbay, D.M., Hogger, C.J.,
Robinson, J.A. (eds.) Handbook of logic in artificial intelligence and logic programming,
vol. 3, pp. 35–110. Oxford University Press, Inc., New York, NY, USA (1994)
15. Müller, C.: Optimierte Wissensdarstellung auf der Basis von Default-Regeln. Masters thesis,
FernUniversität Hagen, Hagen, DE (2003), (in German)
16. Pearl, J.: System Z: A natural ordering of defaults with tractable applications to nonmonotonic
reasoning. In: Proceedings of the 3rd conference on Theoretical aspects of reasoning about
knowledge. pp. 121–135. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1990)
17. Pfeifer, N., Kleiter, G.D.: Coherence and Nonmonotonicity in Human Reasoning. Synthese
146(1–2), 93–109 (2005)
18. Pollock, J.L.: Nomic Probability and the Foundations of Induction. Oxford University Press
(1990)
19. Schurz, G., Thorn, P.D.: Reward versus risk in uncertain inference: Theorems and simulations.
The Review of Symbolic Logic 5, 574–612 (11 2012)
20. Spohn, W.: The Laws of Belief: Ranking Theory and Its Philosophical Applications. Oxford
University Press (2012)
21. Spohn, W.: Ordinal Conditional Functions: A Dynamic Theory of Epistemic States. In:
Causation in Decision, Belief Change and Statistics: Proceedings of the Irvine Conference on
Probability and Causation. The Western Ontario Series in Philosophy of Science, vol. 42, pp.
105–134. Springer Science+Business Media, Dordrecht, NL (1988)
22. Thorn, P.D., Schurz, G.: Ampliative Inference Under Varied Entropy Levels. In: Beierle, C.,
Kern-Isberner, G. (eds.) Proceedings of the 4th Workshop on Dynamics of Knowledge and
Belief (DBK-2013). pp. 46–60. FernUniversität in Hagen (2013)
23. Thorn, P.D., Schurz, G.: A Utility Based Evaluation of Logico-probabilistic Systems. Studia
Logica 102(4), 867–890 (2014)
24. Thorn, P.D., Schurz, G.: Qualitative probabilistic inference under varied entropy levels. Journal
of Applied Logic (forthcoming)
28