In this paper we discuss some neuro-symbolic challenges that exist in combining a machine learning model and a symbolic reasoning framework for directional relation prediction. In particular, we consider a recent machine learning approach that predicts the qualitative directional relations between geographical regions, e.g., X is north-west of Y, where each region is a polygon of boundary points, and highlight the challenges of aligning these predicted relations with the inference rules of a well-known qualitative spatial calculus, viz., the Cardinal Direction Calculus.
Translation Extraction Knowledge Base Neural Network Consolidation Training
Neuro-Symbolic Artificial Intelligence is a paradigm that
deals with the combination of Machine Learning models
and Logic-based frameworks; this combination should
ideally lead to unified architectures that aim to
collaboratively utilize both components to their fullest extent
possible. Due to its diverse and human-like nature that
involves data-driven inference and logical reasoning, as
well as its promise in handling problems that pertain to
both large amounts of data and knowledge-based rules,
Neuro-Symbolic Artificial Intelligence is an important
and re-surging topic of research [
In this paper, we discuss some challenges, as well as ways of addressing these challenges, that arise when trying to join together a machine learning model and a symbolic reasoning framework for directional relation prediction, such as X is north-west of Y ; these challenges pertain to problems that arise during this fusion of the two paradigms in the context of qualitative directional relation prediction. Specifically, we consider a machine learning model for directional relation prediction from nothing more than the eight directional relations that we be multiple directional relations between two regions, mentioned earlier, viz., , , , , , , , . e.g., { , } is encoded as (1, 0, 0, 0, 0, 0, 0, 1). The However, unlike the machine learning model, which per- training geometric data are formed of pairs of polygons forms statistical inference, the Cardinal Direction Calcu- (, ), where each pair represents a reference region lus comes with its own logic-based inference rules, and and a target region. Geometric data are pre-processed aligning the two is part of the discussion in the sequel. with a hand-crafted feature extractor to extract quanti
The rest of the paper is organized as follows. In Sec- tative features, such as angles, areas, intersections with
tion 2 we summarize the machine learning model for regions of acceptance, etc. The labels are from binary
direction relation prediction of [
directional relations between geographical regions by using machine learning techniques, where each region is represented as a polygon formed by a sequence of boundary points. Figure 2 introduces the overall idea of the model, including its training and testing process.
In particular, the authors of [
To facilitate discussion, we first recall the formal
definition of a qualitative constraint language, which is a
constraint language that is used to represent and
reason about qualitative information. A binary qualitative
spatial or temporal constraint language is based on a
ifnite set B of jointly exhaustive and pairwise disjoint
relations, called base relations [
to predict the directional relations between geographic
regions (see Section 2.1. However, the machine learning
model did not consider the semantic connections between
diferent relations and between diferent pairs of regions,
Cardinal Direction Calculus and may have issues such as missing or conflicting
direcLet us first introduce the qualitative temporal constraint tional relations in these predictions, as illustrated in the
language of Point Algebra (PA) [
Now, the Cardinal Direction Calculus (CDC) [
−1. i:Cecil Park j:Cecil Hills ← ∩ ( ◇ ).
k:Doonside j: Eastern Creek i: Huntingwood
Figure 4 gives a such example. In this figure, the pre- is {, , } and is { } and is dicted is {}, i.e., region is on east of region . { }. However, the composition of and However, the machine learning model did not predict is {, , } ̸= , so we can update = . By taking the inverse of , we can directly get ∩ ( ◇ ) = {, }. = −1 = {, , }, meaning that the relation of w.r.t. can be , , or . 3.2. Inconsistency handling
Sometimes there exists the prediction of the directional relation from region to region and the directional re- Sometimes there is a contradiction between the predicted lation from region to region , but the relation from and the composition of and , i.e., ̸= ⋆ region to region is missing, i.e., ̸= ⋆ and ̸= ⋆ and ̸= ⋆ and ̸= ⋆ and ∩ ◇ = ∅. In and = ⋆. In this case, we can obtain an approxima- this case, we can resolve contradiction by replacing tion of by composing and : with the composition of and :
For instance, in Figure 5, the predicted is { } and is { }. However, the machine learning model did not predict . By composing and , we can directly get = ◇ = {, , }.
Removing unfeasible relations
a contradiction between and the composition of and , i.e., ̸= ⋆ and ̸= ⋆ and ̸= ⋆ and ̸= ∩ ( ◇ ). In this case, we can obtain an approximation of by taking the intersection of For example, in Figure 7, the predicted is { } and is {} and is {} and ∩ ◇ = {} ∩ { } = ∅. So we can resolve the inconsistency by setting = ◇ = { }.
There can also be a contradiction between the predicted and , i.e., ̸= ⋆ and ̸= ⋆ and ∪ −1 ̸= −1 and looks more reasonable (probably based on criteria including the area in regions of acceptance, the angle of the line connecting center points, etc.). In this case, we can obtain an approximation of by taking the inverse of : ← −1. i:Bronte j:Clovelly
de la Recherche (ANR) for the “Hybrid AI” project that is tied to the chair of Dr. Sioutis, and the I-SITE program of excellence of Université de Montpellier that complements the ANR funding.
In Figure 8, the predicted is { } and is { } and is more reasonable. So we can directly get = −1 = {, , }.
In this paper we discussed some neuro-symbolic
challenges that arise when trying to combine a machine
learning model and a symbolic reasoning framework for
directional relation prediction. Specifically, on one hand,
we considered the machine learning approach of [
It is important to note that we just initiated the
discussions by presenting several example cases where a
symbolic reasoning framework can help with the
predictions of a machine learning model. Much more work
can be done in the future, e.g., when inconsistency is
detected by composition or inverse, how to determine
which predicted relations are more plausible is an
important yet insuficiently researched topic. How to exploit
the predictions of a machine learning model to perform
symbolic reasoning better is also very interesting. For the
problem considered in this paper, an implicit assumption
is that the semantics of symbolic reasoning matches the
semantics of predicted relations, which in real-world
applications is seldom the case. As have been discussed in
[