The imposition of hard constraints on the output of neural networks is a highly desirable capability, as it instills confidence in AI by ensuring that neural network predictions adhere to domain expertise. This area has received significant attention recently, however, current methods typically enforce constraints in a ”weak” form during training, with no guarantees at inference, and do not provide a general framework for diferent tasks/constraint types. We approach this open problem from a neuro-symbolic perspective. Our method enhances a conventional neural predictor with a reasoning module that can correct predictions errors and a neural attention module that learns to focus the reasoning efort on potential prediction errors while leaving other outputs unchanged. This framework provides a balance between the eficiency of unconstrained neural inference and the high cost of exhaustive reasoning during inference.
when and where to reason in order to efectively achieve high prediction accuracy and constraint satisfaction with low computation cost.
The Method.
We introduce a new neuro-symbolic integration method with a novel neuralattention module (Mask-Predictor) that works with any type of constraints/rules. More formally, we consider a set of input data points ( ∈
) representing instances to solve (e.g. the picture of a partially filled Sudoku board), and, a set of multi-dimensional output data points ( ∈ ) that correspond to complete interpretable (symbolic) solutions (e.g. the symbolic representation of a completely filled Sudoku board). The collection of of these pairs of data points will form the task dataset = { , }
=1 . Moreover, we require that the task (e.g. completing a partially filled Sudoku board) can be expressed (fully or partially) by a set of rules ℛ in the Italy form of domain-knowledge constraints (e.g. the rules of the Sudoku game). The goal is to learn a function ∶ → associating a solution to a given input instance, and which satisfies the rules ℛ. To solve this class of problems, we propose NASR, a neuro-symbolic pipeline that works as follows: an input instance is first processed by the Neuro-Solver that outputs an approximate solution. The solution is then analyzed by the Mask-Predictor that has the role of identifying the components of the Neuro-Solver predictions that do not satisfy the set of domain-knowledge constraints/rules ℛ. The masking output of the Mask-Predictor is then combined with the probability distribution predicted by the Neuro-Solver. This is done by deleting the wrong elements of the predictions, leaving the corresponding components “empty” (the component is iflled by an additional class, indicating a masked element). This masked probability distribution is then fed to the Symbolic-Solver that fills the gaps with a feasible solution (satisfying the rules ℛ). In brief, the role of the Symbolic-Solver is to correct the Neuro-Solver prediction errors identified by the Mask-Predictor.
Results. We tested NASR on two tasks: Visual-Sudoku (given an image of a incomplete Sudoku
board, the goal is to provide a complete symbolic solution) and Predicate Classification ( PredCl,
given in input labeled and localized bounding boxes of a set of objects in an image, the goal is
to predict the correct predicate between them). (1) Visual Sudoku. We first compared NASR
with diferent baselines: a Symbolic Baseline that executes a Symbolic-Solver after converting
the input image to a symbolic form; two state-of-the-art neuro-symbolic methods: NeurASP [
To conclude, we presented a neuro-symbolic method that aims to eficiently satisfy domainknowledge constraints at inference. This enables a favourable trade-of between accurate predictions, noise robustness, and computation cost. Our framework is generic and can be applied to diferent types of input (image, text, etc.) and constraints type (logic, arithmetic, etc.).