In this paper, we address the Symbol Grounding Problem (SGP) in the context of neuro-symbolic planning, where the categorical vectors learned to represent high dimensional inputs sufer from instability, which poses a problem of eficiency during the planning phase. One way to alleviate the SGP is to enforce constraints - among the latent variables - by expressing them in the loss function during the learning process. Combining an existing tool for invariant search and ideas from Logic Tensor Networks (fuzzy logic), we propose to automatize the process of finding and enforcing relevant constraints. We apply our idea to LatPlan, a domain independent, image-based classical planner.
The core interest of Domain-Independent Planning [
When the states of the problem are only available as sub-symbolic data (e.g., images),
generating a PDDL description necessarily needs to address the Symbol Grounding Problem (SGP) [
LatPlan [
Another way to address the SGP is to enforce constraints on the categorical vectors. If we consider a vector as an ideally grounded symbol and its unstable version ′, then ′ contains some noise, i.e. variables with undesired values. If we know appropriate constraints to apply during training, the noisy representation corresponding to ′ should converge to .
In this paper, we propose to use an automatic tool to look for invariants in the PDDL generated
by LatPlan and to test their incidence on the learning by expressing them as an auxiliary loss
term. To do so, we borrow ideas from Logic Tensor Networks (LTN) [
LatPlan is a neuro-symbolic architecture that receives pairs of images representing transitions and learns categorical vector representations from them. Once trained, it can generate a PDDL representation of the states and actions of the problem, which can be input to a planner.
For our work, we are interested in improving the learning of two components of LatPlan, the State AutoEncoder network (SAE), which learns to represent images as categorical vectors, and the Action Model Acquisition network (AMA), which learns to represent actions as categorical vectors representing preconditions and efects. These two models are learned end-to-end from a dataset of pairs of images, each pair representing a valid transition occurring in the sub-symbolic world (e.g. the switching of a tile in the case of 8-puzzle).
The SAE learns a bi-directionnal mapping between sub-symbolic raw data and propositional states ∈ {0, 1} (with being the number of variables in the categorical vector). Concretely, it consists of the encoder and the decoder of a VAE that learns DECODE(ENCODE()) = .
The AMA model consists of three networks: ACTION, APPLY and REGRESS. ACTION learns to associate two consecutive states (returned by the SAE) to an action (expressed as a one-hot vector). APPLY learns to predict the next categorical state from the previous one and an action. REGRESS is symmetric of APPLY and learns to predict backwards the previous state from the next one and from the action.
The SAE, APPLY and REGRESS networks all output binary categorical vectors of the same size as the SAE’s output. Each element of these vectors is interpreted as a binary variable and is represented by a unary predicate in the PDDL files generated by LatPlan. This representation is negatively afected by the SGP, as it can break the identity assumption inherent to symbolic reasoning algorithms — in which a state must not change — and can cause disconnections during the search process, i.e. if two states exist that should represent the same state, an action might lead to one of them but not to the other, the latter would then be a dead end.
To address this issue, the authors of LatPlan propose two solutions. First, at test time, they replace the sampling of the categorical vector with an argmax layer, therefore removing stochasticity. Second, during training, they select a version of the prior distribution for the sampling (of the categorical vector) that favours sparsity of truthiness among the binary variables. This leads to more stable latent state vectors, and showed improved performances of the model in terms of next-state prediction accuracy (APPLY network) and planning performances. In our work, we are mainly interested in stabilizing the latent states vectors, generated by the SAE network; to assess it, we use the same metric as in LatPlan paper, i.e. the State Variance. To compute it, we use the state variance for noisy input, i.e., the variance of the latent vectors : ,0 = (,0 + ) where: • ∼ ( = 0, = 0.3) • ,0 : 1 image of the ℎ transition pair of the dataset
The State Variance is computed by iterating over 10 random vectors, then averaged over F bits in the latent space and over the dataset indexed by i. Formally:
E∈0... E Var ∈0..10[ENCODE(,0, ) ] (1)
In classical planning, an invariant is defined as a logical formula over variables of the domain
which is true in any reachable state [
An important part of the research in classical planning focuses on discovering invariants
in planning domains. Today, automatic tools [
Extensive work on integrating logical constraints during the training of neural networks have
been conducted in the last few years. Notably, one can express constraints among neural
networks outputs taken as predicates thanks to t-norm operations and the fuzzy generalisation
of First Order Logic (FOL) [
For example, if we want to express the truth value of ¬1 ∨ ¬2, we can compute it by its average over the dataset, i.e., 1
∑︁ (1 − (1)) + (1 − (2)) || 1,2∈ (2) where (1) is the (continuous) value of truth of one grounding of “1 is true”, and is the set of all the groundings. The inverse of this value can be directly appended as a penalty to the loss function, which eventually forces the network to re-adapt its weights to this new constraint.
We first discuss how we intend to search for invariants of interest among LatPlan binary variables with the help of an automatic tool; then, we discuss how to enforce these invariants during training and we give details about integrating the search in the training loop.
Automatic tools, like Fast Downward (FD) [
In Latplan, internal categorical representations are continuous vectors from which elements converge to binary values during training. At each batch, we can compute the truth value — thanks to fuzzy logic — of any constraint, expressed as a logical forumula, over these binary variables, for instance “1 is true.” Then, by taking the inverse of this value and multiplying it by a normalizing factor, we obtain an additional loss that we can append to the total loss. This way, the network will adapt its weights through backpropagation to comply with the constraint.
Our idea is to integrate an automatic invariant finder in the learning process of LatPlan. More
precisely, we want to perform a search — similar to a hyperparameter search — on the invariants.
Further details can be find in Appendix A. The implementation of this loop builds upon the
t-norm like functions that already exist in Tensorflow [
We discussed the idea of combining an automatic invariants finder with fuzzy logic to progress in solving the SGP that afects the latent representations of a neuro-symbolic architecture (LatPlan). More precisely, the invariants found by the automatic tool are probable invariants since the PDDL they are issued from is learned by statistical inference - and we want to enforce them during training. If they bring a lower state variance, it is probable that they correspond to ground true invariants. The goal is to automatically find invariants that are responsible for more state stability. We are now implementing the search loop introduced in Section 3.
This augmented training loop, GreedyConstraintTraining, is illustrated in Algorithm 1, which uses the following auxiliary functions: • LatPlan, which is a function that embeds the training process of LatPlan: it receives as input a training set, a set of propositional formulae to embed in the loss function, and outputs the learned functions and action descriptions as binary vectors; • Vecs2Pddl transforms the action descriptions learned by LatPlan into PDDL format; • Pddl2Inv is a function that returns a set of invariants of a planning problem in the PDDL format provided in the input. An example is the invariant finder of Fast Downward; • MetricEval receives as input the learned functions and action descriptions of LatPlan and outputs a score value: the higher the score value, the better. In our case, it is the inverse of the State Variance given in Formula 1 (in this case only the SAE function of LatPlan is needed). • PickAnInvariant is a heuristic that identifies the most promising invariant in a set; for instance an invariant with the less number of variables (because it has a higher probability to be a ground true invariant compared to one with more variables).
Algorithm 1 Our proposed GreedyConstraintTraining approach, which receives as input a training set tSet , and returns Ω , the invariants which improve on a chosen metric 1: Ω ← ∅ 2: LPModel ← LatPlan(tSet , ∅) 3: Φ ← Pddl2Inv(Vecs2Pddl(LPModel )) 4: score ← MetricEval(LPModel ) 5: while Φ ̸= ∅ do 6: ← PickAnInvariant(Φ) 7: Φ ← Φ ∖ {} 8: LPModel ′ ← LatPlan(tSet , Ω ∪ {}) 9: if MetricEval(LPModel ′) > score then 10: Ω ← Ω ∪ {} 11: Φ ← Φ ∪ Pddl2Inv(Vecs2Pddl(LPModel ′)) 12: score ← MetricEval(LPModel ′) 13: end if 14: end while 15: return Ω