In their book, Noise: A Flaw in Human Judgment, the authors Daniel Kanheman, Olivier Sibony and Cass R. Sunstein highlight the importance of minimizing bias, i.e. systematic deviation, and noise, i.e. variability, in judgments in order to reduce error. Bias has long been the subject of many discussions but noise is yet to gain the attention it deserves. In this paper, we discuss noise variables in decision-making, particularly in unique, non-recurrent or singular decisions. For this purpose we introduce and utilize the framework of the Weak Completion Semantics to discuss how noise variables may be identified using counterfactual reasoning.
same person being interviewed by diferent interviewers may be rated and assessed diferently.
This indicates variability and noise. People seeking asylum in the United States are admitted
on a system resembling a lottery [
When speaking of noise in decision-making, one needs to recognize two kinds of decisions:
recurrent and singular. Recurrent decisions are those which we take on a repetitive basis. For
example, deciding on the grades or performance of students appearing for an examination.
Singular decisions are on the other hand unique and not repeated. For example, deciding on
marriage with a particular person, or deciding whether to design a certain kind of mobile
application can be considered as singular decisions. Obama’s decision to send three thousand
healthcare workers and soldiers to West Africa when the Ebola epidemic broke out in 2014 is yet
another example of a historically prominent singular decision [
Recurrent and singular decisions both sufer from noise and variability. It is however not very
easy to identify noise in singular decisions, given their nature. While identifying the presence of
noise in the exemplary situation above, one may imagine the following, in lines with [
The WCS is a formal three-valued, computational, non-monotonic cognitive theory. Till date
it has been used to adequately model the suppression task by [
Counterfactual reasoning, as the name suggest, is reasoning with imagined alternatives
which are contrary to facts. That is, how people reflect on past events which have occurred,
and imagine possibilities like, ”What if . . . had happened? Would the outcome have been any
diferent? What if . . . had not happened?”. People can imagine alternatives to something that
has happened, by either deleting or undoing some aspect of it which had happened in the past
reality, or by adding some new aspects to their simulation of reality [
Summing up, during decision-making or judgment it is advisable that both noise and bias within a system be addressed and minimized as much as possible. In order to reduce noise, one needs to identify it first. Our aim is to propose one such modelling method using the WCS to identify noise variables within a given system. The paper is thus organized as follows: In Section 2 we formally introduce the WCS. A classification of conditional sentences relevant to the goal of this paper is given in Section 3. An example of a singular decision is presented in Section 4. Identification of noise variables in this particular system using the WCS is demonstrated in Section 5. Finally, in Section 6 we conclude and outline further possible research.
We assume the reader to be familiar with logic and logic programming as presented in e.g. [
Throughout this paper, will denote a program. An atom is defined in if and only if
contains a clause of the form ← body . As an example consider the program
= { ← ∧ ¬ab, ab ← ⊥} ,
where , , and ab are atoms. and ab are defined, whereas is undefined. ab is an
abnormality predicate which is assumed to be false. In the WCS, this program represents the
conditional sentence if then . In their everyday lives humans are often required to reason
in situations where the information of all factors afecting the situation might not be complete.
They still reason, unless new information which needs consideration comes to light. The
abnormality predicate in the program serves the purpose of this (default) assumption, as was
suggested in [
Let us now consider the following transformation: (1) For all defined atoms occurring
in a program , replace all clauses of the form ← body 1, ← body 2, . . . by ←
body 1 ∨ body 2 ∨ . . . . (2) Replace all occurrences of ← by ↔. The resulting set of equivalences
is called the weak completion of , denoted by wc(). It difers from the program completion
defined in [
As shown in [
Overall, given premises, general knowledge, and observations, reasoning in the WCS is
modelled in six steps:
1. Reasoning towards a logic program following [
6. Searching for counterexamples [
In this section we introduce a classification of conditionals and their antecedents which we
consider relevant for the identification of noise variables in a system. Pragmatics, life experiences
and cultural diferences play a deciding role in how diferent individuals comprehend or classify
both. Following [
After the above discussion a question that may naturally arise is, what happens when the antecedent of a conditional sentence is not satisfied? To that end, the antecedent of a conditional sentence if then can be classified as necessary with respect to the consequent , if and only if cannot be true unless is true. This implies that if does not hold, cannot either. For example, in case of the conditional if plants get water then they grow, the antecedent plants get water is a necessary one for the consequent plants will grow. If a plant is not watered at all, it will very likely die. This does not imply however, that the antecedent need always be a precondition for the consequent, per se. On the other hand, the antecedent of a conditional sentence if then is said to be non-necessary with respect to the consequent , if can be true despite the falsity of . This implies, if does not hold, may or may not hold. In the conditional, if Maria drinks alcoholic beverages in pubs then she must be over 19 years of age, the falsity of drinking alcoholic beverages in a pub is inconsequential to the truth of the consequent older than 19 years. There are plenty of adults (over 19 years) who do not drink alcohol. The antecedent of the conditional sentence is therefore called non-necessary.
In order to account for the aforementioned classifications in the WCS, given a clause
{ ← ∧ ¬ab} in a program , the previous definition of the set of abducibles (see
Section 2) can be extended to include = { ← ⊤} when the antecedent is deemed
non-necessary, and to include = {ab ← ⊤} when the conditional is deemed factual. An
interested reader may find this discussed in a lot more detail in [
The human race has been threatened by a multitude of diseases, among which Cholera alone has claimed the lives of many over the years. Its patients sufer from mild to severe symptoms, the latter including intense diarrhoea and dehydration which may cause the person’s skin to turn bluish-grey and even be life-threatening. This symptom gives Cholera its colloquial nickname, "the blue death". Although Cholera still poses a problem in certain parts of the world, sewage and water treatment systems have helped curb it in many parts of the globe. This was not however always the case. England for example sufered high Cholera outbreaks in 1831, 1848 and 1854. No efective treatment had yet been in sight, and people did not yet have the knowledge that bacteria transmitted the disease. It was a common (mis)conception that Cholera spread through "miasmas" or toxic gases from open graves, garbage dumps, sewers etc.
During the Cholera outbreak in 1848, Dr. John Snow, who had received his medical degree in London was a practising physician there. The outbreak led Snow to examine many Cholera patients and become heavily involved in the study of the disease. It is also noteworthy that during the 1831 outbreak, young Snow who aimed to be a physician had been working as an apprentice to a physician who treated Cholera patients. These experiences ultimately led Snow to publish a book on specific conclusions he had reached about the nature of the disease which was unfortunately rejected by most of his colleagues in the medical community. This did not discourage Snow however, and in 1854 when Cholera broke out once again Snow conducted his own investigation. His investigations showed that most death cases were approximately concentrated around a hand pump in Broad Street. Furthermore, numerous (fruitful) interviews with the families of the deceased indicated that the victims had consumed water from the aforementioned hand pump. Convinced that the hand pump’s contaminated water played an important role in the epidemic, Snow appeared for an interview with the Board of Guardians of the St. James parish. The board paid heed to Snow’s advice and had the hand pump removed the following day. This helped control the epidemic to a large extent, the repercussions of which echoed through time.
The decision taken by the Board of Guardians to remove the Broad Street pump was a singular one because it was a unique decision with unique circumstances that was never repeated. This singular decision, however, was not entirely free from noise. In other words, had certain parameters in the background story been changed, the singular decision might have had a diferent outcome. Identification of the noise, or more specifically the noise variables can be brought about by reasoning counterfactually about the singular decision. How this can be modelled using the WCS framework will be discussed in the next section. 5. Identifying Noise Variables using Counterfactual Reasoning
Table 2 lists the various conditional statements which may be used to comprehensively summarize the background knowledge presented in Section 4. For the convenience of the reader it also includes the corresponding clauses in a logic program representing the said conditionals, which shall be further used during the modelling of noise variables in the next sub-section. The reader may observe that the abnormality predicate in each clause has been assumed to be false. This may be overridden in the subsequent discussion when we take the classification of the conditionals and their antecedents into account while searching for noise variables.
The idea behind the approach that we demonstrate in this paper is to pick up the thread of the singular decision or the first statement, counterfactually gauge which antecedent could have altered the consequent, which in turn hints at the former being a noise variable. Then, go further back in the chain of events to explore the rules of inference which led to the aforementioned antecedent, thus repeating the exercise.
Beginning with statement (1) in Table 2, upon considering the nature of the antecedent as discussed in Section 3, one may deem the antecedent to have been necessary for the consequent. In other words, one may not easily imagine a possibility where Snow had no interview with the board yet the board decided to remove the hand pump. Had Snow not approached the board for an interview, the very idea of removing the hand pump may have slipped their attention. The logic program 1 which may be constructed from statement (1) consists of the following: { ← ∧ ¬ab, ab ← ⊥ , ← ⊤} , where rem denotes the board removed the hand pump and int denotes Snow had an interview with the board. abint is an abnormality predicate denoting anything that went wrong with regard to the interview, e.g. one of the board members fell sick on the spot and the meeting was adjourned. As there was no such case it is assumed to be false. The last clause represents a fact. The weak completion of 1, (1), is
{ ↔ ∧ ¬ab, ab ↔ ⊥, ↔ ⊤},
which has the least model ℳ(1) = ⟨{, }, {ab}⟩.3 This signifies and
are true, while ab is false. When reasoning with the situation counterfactually, one may
use the conditional if Dr. Snow did not have an interview with the board, then the board would
not have removed the hand pump or, if ¬ then ¬. Upon evaluating ¬ under the
aforementioned least model, the reader may find that it is false. Hence we call the
conditional a subtractive counterfactual conditional, in lines with [
{ ↔ ∧ ¬ab, ab ↔ ⊥, ↔ ⊥} admits the least model ⟨∅, {, , ab}⟩. In particular, which was true is now false. Thus, the variability in the truth of the consequent leads to variability in the outcome, which indicates noise. Consequently, int is a noise variable.
Now we further explore the atom int and hence statement (2). Considering that the antecedent may be deemed necessary for the consequent, any possibility that Snow himself was not convinced that the hand pump should be removed yet had an interview with the board about the same may be discounted. The logic program 2 constructed from the statements (1) and (2) is: { ← ∧ ¬ab, ab ← ⊥ , ← ∧ ¬ab, ab ← ⊥ , ← ⊤} , where conv denotes Snow was convinced about the hand pump, and abconv is an abnormality predicate assumed to be false. The last clause represents a fact. (2) admits the least model ℳ(2) = ⟨{, , }, {ab, ab}⟩. Reasoning counterfactually, one may imagine the conditional if John had not been convinced then he would not have asked for an interview viz. if ¬ then ¬. As ¬ evaluates to false under the aforementioned least model, this is a subtractive counterfactual conditional. Then, rev (2 , {¬conv }) leads to (2 ∖ { ← ⊤} ) ∪ { ← ⊥} . Its weak completion admits the least model ⟨∅, {, , , ab, ab}⟩. This indicates that is a noise variable.
Going further back on the chain of events, we now consider the statements (1) to (4). In case
of statements (3) and (4), both antecedents of the conditionals may be considered necessary for
the consequent. Hence, in such a case the logic program 3 is:
3This model can be computed as follows. Starting with the interpretation ⟨∅, ∅⟩, we obtain Φ1 ⟨∅, ∅⟩ =
⟨{}, {ab}⟩ and Φ1 ⟨{}, {ab}⟩ = ⟨{, }, {ab}⟩ = Φ1 ⟨{, }, {ab}⟩.
← ∧ ¬ab, ← ∧ ¬ab,
ab ← ⊥ , ab ← ¬ , ab ← ⊥ , ab ← ¬ , ← ⊤ , ← ⊤} ,
where exp denotes Snow had a lot of experience with Cholera patients and inv denotes Snow’s
investigations proved fruitful. abexp and abinv are abnormality predicates. The last two clauses
represent facts. One may observe that we have now added the clauses ab ← ¬ and
ab ← ¬ and we attempt to clarify this in what follows. As the antecedents of both
the conditionals if then and if then have been deemed necessary for the
consequent, the possibility that one of the antecedents is false but the consequent is true is
discounted. Not having enough experience with Cholera patients could have prevented Snow from
being convinced about the removal of the hand pump even if his investigations had proved fruitful.
Likewise, an unproductive investigation could have prevented Snow from being convinced about the
pump, despite him having enough experience. Thus given the two conditionals if then
and if then , where both the antecedents are deemed to be necessary, we characterize
¬ as an abnormality with respect to the former conditional and ¬ as an abnormality
with respect to the latter. Hence the additional clauses ab ← ¬ and ab ← ¬ . Such
a characterization is along the lines of so-called enabling relations as discussed in [
Let us now consider statements (1) to (6), and in particular statements (5) and (6). While practising in London during 1848 may be deemed necessary for the level of experience Snow had with Cholera patients, his apprenticeship during 1831 may be deemed non-necessary by some individuals. That is while one may not readily imagine a possibility where Snow did not practise in London, but had experience with Cholera patients, one may imagine Snow not doing the apprenticeship yet having experience. The reader is pointed out that the opposite may also hold for some individuals where they may consider apprenticeship to be necessary but the London practice to be non-necessary. Or some individuals may even consider both antecedents to be non-necessary for the consequent. For the sake of modelling and demonstration purposes, we assume the first case. So, we consider the conditional if then , where is deemed non-necessary for . And consider if then , where is deemed necessary for , owing to which we characterize ¬ as an abnormality with respect to the former conditional.4 Thus, the program 4 has the clauses:
∧ ¬ab, ab ← ⊥ , , ab ← ¬
, ← ⊤ , ← ⊤ , ← ⊤} , where prac denotes Snow was practising in London during the 1848 outbreak and app denotes Snow was an apprentice for a physician during the 1831 outbreak. abprac and abapp are abnormality predicates. The last three clauses represent facts. The reader may note that we now have the additional clause ab ← ¬ , and that the assumption ab ← ⊥ is overridden by in (4). The least model of (4) is: ab ← ¬ ℳ(4) = ⟨{, , , , , , }, {ab, ab, ab, ab, ab, ab}⟩. Here, app and prac are both true and so is rem. Because prac has been deemed necessary for exp, irrespective of whether he worked as an apprentice or not, as long as Snow did not practice in London in 1848 he would not have the level of experience. Reasoning counterfactually about the situation we may thus imagine, if Snow would not have been practising in London then he would not have the experience viz. if ¬ then ¬. Upon evaluating ¬ with respect to ℳ(4), we find that it is false. Hence, this is a subtractive counterfactual conditional. Therefore revision using rev (4 , {¬prac}) leads us to the program (4 ∖ { ← ⊤} ) ∪ { ← ⊥} . Its weak completion admits a least model where prac is false, and so is rem. The variability in the truth of rem indicates that prac is a noise variable.
Now, reconsidering statements (5) and (6) from a diferent angle, we might ask ourselves the following - ”if Snow was a physician’s apprentice in 1831 does it necessarily mean that he would have a high level of experience with Cholera patients?” or that ”if Snow practised in London in 1848, does it necessarily that he would have a high level of experience with Cholera patients?”. In other words, we may imagine a possibility where Snow did practise in London in 1848 but due to some (additional) reasons he could not have a high level of experience with Cholera patients. Or that, he was an apprentice but certain reasons hindered his opportunity to have the needed experience. As discussed in Section 3, imagining such (alternative) possibilities count for statements (5) and (6) being comprehended as factual. Meaning in such a case, given if then and afirming , both and ¬ are deemed possible. For the current moment, we are particularly interested in this latter possibility. Within the WCS it can be modelled by considering as an observation and applying abduction in order to explain it using the set of abducibles for factual conditionals, , mentioned in Section 3. We attempt to clarify the process in the current context in what follows. We consider the program 5 = 4 ∖ { ← ⊤ , ← ⊤} , and consider and to be observations instead, meaning = {, }. Now we apply abduction in order to explain . Following the definition of abducibles as described in Section 2, since both and 4This is in lines with the prior discussion regarding statement (3) and (4). are undefined in 5, 5 = { ← ⊤ , ← ⊥ , ← ⊤ , ← ⊥} . Moreover, comprehending statements (5) and (6) as factual entails extending the set of abducibles that can be derived from 5 to 5 = 5 ∪ 5 , where 5 includes {ab ← ⊤ , ab ← ⊤} . While there is a minimal explanation for , viz. { ← ⊤ , ← ⊤} , there is also a non-minimal explanation viz. { ← ⊤ , ab ← ⊤ , ← ⊤ , ab ← ⊤} . Adding the former to 5 would again result in the program 4. However, the latter results in a program 5′: ← ∧ ¬ab, ab ← ⊥ , ab ← ¬ ,
∧ ¬ab, ab ← ⊥ , ab ← ⊤ ,
, ab ← ¬ , ab ← ⊤ , ← ⊤ , ← ⊤ , ← ⊤} .
Its weak completion, (5′), admits the least model:
ℳ(5′) = ⟨{, , , ab, ab, ab}, {, ab, , ab, , ab, }⟩,
where , , ab and ab are true, but exp is false. The reader is pointed out that
the additional clauses {ab ← ⊤} and {ab ← ⊤} signify that there could be (other
additional) reasons which could have hindered Snow’s experience with Cholera patients.
This is in line with additive counterfactuals as discussed in [
The case of non-necessary antecedents begs a lengthier discussion than the current spatial
constraints of the paper would allow. Given if then , comprehending to be non-necessary
for signifies that in case of ¬, both ¬ and particularly are deemed possible. This
implies two least models, one in which is false, and the other where is true. Both can be
computed by employing the abductive framework using the extended set of abducibles
mentioned in Section 3, as discussed in detail in [
In all that has been demonstrated so far, we have attempted to illustrate the identification
of some of the noise variables in our system such as, , , , , , ab, ab
etc. Thus we have identified the system noise. At this point it must also be acknowledged that
while there are variables which may contribute to noise, there may also be those which do
not. One possible means to identify the latter could be to guage the relevance of the antecedent
of a conditional with respect to the consequent, in line with [
Noise just like bias, may be undesirable in much of our decision-making and judgment. As the
human race journeys further into the age of digitalization and artificial intelligence becomes
more and more involved in our daily lives, creating systems which minimize bias and noise, both
of which contribute to errors in judgement seems important. In order to create any kind of AI
system with decision-making capabilities with minimal noise and bias, it is essential to discuss
how they can be identified in our own, humane judgments, whether it be in economy, judiciary,
education, healthcare, or even personal. Regardless of whether the decisions are recurrent or
singular the aim is to identify and minimize the noise in both. In this paper, we have particularly
looked into singular decisions because in comparison with recurrent decisions the noise in these
systems may be less apparent. Considering the singular decision to remove the hand pump in
Broad Street which not only helped save the lives of many during the 1854 Cholera outbreak
5Unfortunately the current spatial constraints of the paper disallows us from being more detailed about strong
relevance, but an interested reader is encouraged to read [