Matrices via a Neuro-vector-symbolic Architecture

Abstract

Human fluid intelligence is the ability to think and reason abstractly, and make inferences in a novel domain. The Raven's progressive matrices (RPM) test has been a widely-used assessment of fluid intelligence and visual abstract reasoning. Neuro-symbolic AI approaches display both perception and reasoning capabilities, but inherit the limitations of their individual deep learning and symbolic AI components, namely the the so-called neural binding problem and exhaustive symbolic searches explained in the following.

-

Italy

© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). VSA representations can be composed, decomposed, probed, and transformed in various ways using a set of well-defined operations, including binding, unbinding, bundling, permutations, and associative memory.

In [ 1 ], we propose a neuro-vector-symbolic architecture (NVSA) consisting of a visual perception frontend and a probabilisitc reasoning backend, both taping into the rich resources of VSA as a general computing framework (see Fig. 1). The resulting NVSA frontend addresses the binding problem in the neural networks, especially the superposition catastrophe, by efectively mapping the raw image of multiple objects to the structural VSA representations that still maintain the perceptual uncertainty. The NVSA backend maps the inferred probability mass functions into another vector space of VSA such that the exhaustive probability computations and searches can be substituted by algebraic operations in that vector space. The VSA operations ofer distributivity and computing-in-superposition, which significantly reduce the computational costs thus performing probabilistic abduction and execution in real-time manner.

Results. Compared to the state- Table 1: Average accuracy (%) on the I-RAVEN. of-the-art deep neural network [ 3 ] and neuro-symbolic approaches [ 2 ], Method Avg C 2x2 3x3 L-R U-D O-IC O-IG end-to-end training (NVSA e2e tr.) SPCrALE[3[ 2 ]] 8741..31 9893..98 6882..99 4437..04 9984..58 9994..18 9576..76 8327..64 of NVSA achieves a new record of NVSA (e2e tr.) 88.1 99.8 96.2 54.3 100 99.9 99.6 67.1 88.1% in the I-RAVEN dataset (see NVSA (attr. tr.) 99.0 100 99.5 97.1 100 100 100 96.4 Table 1). In a fully supervised setting in which the labels of the visual attributes are given (NVSA attr. tr.), the NVSA frontend can be trained independently of the backend with a novel additive cross-entropy loss, yielding highest accuracy of 99.0%. Moreover, compared to the symbolic reasoning within the state-ofthe-art neuro-symbolic approaches, the probabilistic reasoning of NVSA with less expensive operations on the distributed yet transparent representations is two orders of magnitude faster.

Generalization. Further, we analyze the generalization of the NVSA frontend to unseen combinations of attribute values in novel objects. We observe that the frontend with the multiplicative binding cannot generalize to unseen combinations of the attribute values, hence we enhance it by a multiplicative-additive encoding that can generalize up to 72%. The multiplicative binding-based encoding however generalizes well to unseen combinations of multiple objects. We also evaluate the out-of-distribution generalizability of NVSA frontend and backend with respect to unseen attribute–rule pairs. NVSA outperforms the deep learning baselines by a large margin in all unseen attribute–rule pairs.

References

[1] M. Hersche , M. Zeqiri , L. Benini , A. Sebastian , A. Rahimi , A neuro-vector-symbolic architecture for solving Raven's progressive matrices , Nature Machine Intelligence ( 2023 ).

[2] C. Zhang , B. Jia , S.-C. Zhu, Y. Zhu , Abstract spatial-temporal reasoning via probabilistic abduction and execution , in: IEEE CVPR, 2021 .

[3] Y. Wu , H. Dong , R. Grosse , J. Ba, The scattering compositional learner: Discovering objects, attributes, relationships in analogical reasoning , arXiv preprint arXiv: 2007 . 04212 ( 2020 ).