In the early days of machine learning, Donald Michie introduced two orthogonal dimensions to evaluate performance of machine learning approaches - predictive accuracy and comprehensibility of the learned hypotheses. Later definitions narrowed the focus to measures of accuracy. As a consequence, statistical/neuronal approaches have been favoured over symbolic approaches to machine learning, such as inductive logic programming (ILP). Recently, the importance of comprehensibility has been rediscovered under the slogan 'explainable AI'. This is due to the growing interest in black-box deep learning approaches in many application domains where it is crucial that system decisions are transparent and comprehensible and in consequence trustworthy. I will give a short history of machine learning research followed by a presentation of two specific approaches of symbolic machine learning - inductive logic programming and end-user programming. Furthermore, I will present current work on explanation generation.
Machine learning research began in the 1950s with roots in two di↵erent scientific
communities – artificial intelligence (AI) and signal processing. In signal
processing research mainly statistical methods to generalisation learning from data
were developed under the label of pattern recognition. In AI machine learning
was investigated under this label as a crucial principle underlying general
intelligent behaviour along with other approaches such as knowledge representation,
automated reasoning, planning, game playing, and natural language processing
In the early days of machine learning, Donald Michie introduced two
orthogonal dimensions to evaluate performance of machine learning approaches
– predictive accuracy and comprehensibility of the learned hypotheses
However, in recent years, there is a growing recognition that the holy grail
of performance is not sucient for successful applications of machine learning in
complex real world domains. Under the label of ‘explainable AI’
In the following, I will argue that symbol level approaches to machine learning, such as approaches to inductive programming, o↵er an interesting complement to standard machine learning. I will present ILP and end user programming as two specific approaches of interest. Furthermore, I will present current work on explanation generation. 2
A classic white-box approach which allow the generation of hypotheses in form of
symbolic representations are decisions trees and variants such as decision rules or
random forests. There is empirical evidence that such machine learned
hypothesis can be understood and applied by humans
In contrast to standard machine learning, these approaches are strongly
related to formalisms of computer science. The expressiveness of learned
hypotheses goes beyond conjunctions over feature values. In principle, arbitrary
computer programs – for instance in the declarative language Haskell (for inductive
functional) or Prolog (for inductive logic programming) can be induced from
training examples
Inductive Logic Programming Inductive logic programming has been established in the 1990ies as a symbollevel approach to machine learning where logic (Prolog) programs are used as Target Concepts (Rules): % grandparent without invented pred. p(X,Y) :- father(X,Z), father(Z,Y). p(X,Y) :- father(X,Z), mother(Z,Y). p(X,Y) :- mother(X,Z), mother(Z,Y). p(X,Y) :- mother(X,Z), father(Z,Y). % ancestor without invented predicate p(X,Y) :- father(X,Y). p(X,Y) :- mother(X,Y). p(X,Y) :- father(X,Z), p(Z,Y). p(X,Y) :- mother(X,Z), p(Z,Y).
Background Knowledge (Observations): father(jake,alice). mother(matilda,alice). father(jake,john). mother(matilda,john). father(bill,ted). mother(alice,ted). father(bill,megan). mother(alice,megan). father(john,harry). mother(mary,harry). father(john,susan). mother(mary,susan).
mother(mary,andy). father(ted,bob). mother(jill,bob). father(ted,jane). mother(jill,jane). father(harry,sam). mother(liz,sam). father(harry,jo). mother(liz,jo).
Target Concepts (Rules):
p(X,Y) :- p1(X,Z), p1(Z,Y).
p1(X,Y) :- father(X,Y).
p1(X,Y) :- mother(X,Y).
% ancestor with invented predicate
p(X,Y) :- p1(X,Y).
p(X,Y) :- p1(X,Z), p(Z,Y).
p1(X,Y) :- father(X,Y).
p1(X,Y) :- mother(X,Y).
a uniform representation for examples, background knowledge and hypotheses.
In Figure 2 the well-known family tree example is shown. Here the father and
mother relations are given as background knowledge as observed/known facts
concerning a specific family. Target predicates can be as simple as grandfather
which can be described as the father of a parent of a person or a bit more
general such as grandparent or more complex, such as the ancestor relation
which involves recursion. For learning, some positive and negative examples for
the target predicate are presented. For example ancestor(Jake, Bob) is a positive
example and ancestor(Harry, Mary) is a negative example for ancestor. Given
this information, an ILP system can derive a hypothetical logic program which
entails all the positive and none of the negative examples. Some relaxation of
this brittle criterion is possible
One of the main advantages of ILP over other symbolic approaches to
machine learning is that it allows to consider n-ary relations. Although it is possible
to transform relations into features, such a transformation can be tedious, result
in sparse feature vectors, and cannot be performed without loss of information.
The advantage of ILP in structural domains such as chemistry has for example
been demonstrated for the identification of mutagenic chemical structures
Flashfill is a convincing example for end-user programming which gives users
without background in computer programming the possibility to work more
efficiently with their software applications
Over the last years there is a growing interest in artificial intelligence from
industry and it became a topic of discussion in politics and society in general.
Mostly AI is used a label for machine learning – ignoring all other subject
domains which constitute AI, among them knowledge representation and learning.
Furthermore, machine learning is mostly exclusively referring to artificial neural
networks, especially variants of deep learning. Following the big bang of deep
learning
It is widely recognized that explanations are crucial for comprehensibility and
trust
Often, it might be especially helpful to combine visual and verbal
explanations. An example is given in Figure 4. Here the explanation involves distortions
of specific regions of the face which are typically characterized by so called
action units used in the facial action coding systems (FA CS)
Explanations are important for transparency of a machine learned classifier. Experts might demand an explanation if their own decision deviates from that of the classifier. The explanation might be convincing for the expert or not and consequently, the expert might revise his decision or override the classification.
Explanations can also be helpful in education and training. Here, contrasting
examples might be helpful to make explanations more comprehensible. In many
realistic contexts, it is not so easy to relate ones perception to a specific feature
value. For example, to classify a mushroom it might be necessary to decide
whether the cap has the shape of a bell or whether it is conical
It is an open research question which type of contrast is helpful to support
understanding. In cognitive science research on similarity and analogy it is
recognized that alignment is an important aspect for concept acquisition
It is my strong conviction that future applications of machine learning should
not aim at replacing human decision makers but at supporting them in
complex domains. It is still desirable, in my opinion, to develop machine learning
approaches which fulfill the ultra-strong machine learning criterion proposed by
Michie. Thereby, rather than creating autonomous systems which replace
humans, we can create intelligent companion systems