Integration of symbolic and sub-symbolic approaches is rapidly emerging as an Artificial Intelligence (AI) paradigm. This paper presents a proof-of-concept approach towards an unsupervised learning method, based on Restricted Boltzmann Machines (RBMs), for extracting semantic associations among prominent entities within data. Validation of the approach is performed in two datasets that connect language and vision, namely Visual Genome and GQA. A methodology to formally structure the extracted knowledge for subsequent use through reasoning engines is also ofered.
stronger AI. This integration involves the progress from intuitive, sub-symbolic intelligence
to symbolic, logic-based cognitive processes. Bridging sub-symbolic (connectionist) AI with
symbolic AI has long been considered crucial towards efective AI solutions [
Recognizing the need of symbolic and sub-symbolic integration, the work of this paper
is inspired by the amalgamation of diferent learning and epistemology theories on human
cognitive development. Besides their functional diferences, most theories converge to a
developmental process by which human cognition evolves. In this process, sub-symbolic learning is
fundamental (both in the sense of rudimentary, as well as in the sense of necessary) to obtain
symbolic function, subsequently used for concrete or abstract logic and reasoning [
The current success of NN-based sub-symbolic learning and the plethora of symbolic KRR
solutions long available, signify that AI is now equipped to complete its development cycle,
but with one missing link between two ends: non-manual acquisition of symbolic knowledge
based on the distillation of information hidden in sub-symbolic models. Figure 1 portrays the
analogies with the Piagetean [
The work of this paper corresponds to analogies understanding, i.e. making connections
between concepts. Consequently, we propose an unsupervised approach for extracting knowledge
based on the patterns formed in trained neural networks, namely from Restricted Boltzmann
Machines (RBMs) [
To this end, Section 2 ofers an overview of related work; Section 3 presents the implemented method, along with first experimental results and observations; Section 4 provides a conclusion over the initial approach and a concrete overview of future work.
Integration of sub-symbolic and symbolic methods pertains to three major research directions:
a) symbolic representation learning through neural approaches; b) induction of structured
2Piaget’s theory has received criticism in terms of terminology, stage transition and contents, causation of
gained attributes, etc. This is the reason why the specific stage names are not used in Figure 1. However, it does
provide a concrete flow of the development of cognitive capacities in humans, useful to depict the analogies in
human vs machine intelligence developmental processes.
knowledge and/or some logical inference capacities in neural approaches, e.g. [
The scope of this paper lies in (a) - symbolic representation learning. This can be further divided to: i) recognition of pertinent symbolic representations of salient concepts within a domain and/or their hierarchy; ii) recognition of prominent relations that associate particular concepts; iii) pattern mining for extracting particular associative rules within a domain of discourse.
Representation learning focuses on identifying/automatically constructing the input features
needed to perform a specific ML task. In computer vision, representations learned often do not
have any symbolic manifestation, rather remain unstructured, black box vehicles in ML-based
classification and feature detection [
In NLP, representation learning revolves mostly around the construction of word embeddings,
associating words of high lexical proximity. Symbolic representation of natural language
components is inherent, often accompanied with underlying semantics [
Symbolic representation of non-taxonomic relations, however, is one of the most pertinent tasks towards structured knowledge acquisition. To this end, [13] employ modular neural networks, combining diferent NN models (e.g. image and text) with common features fused in a cumulative model, thus allowing to learn how textual relations associate objects in images.
The Neuro-Symbolic Concept Learner [14], not only learns object representations in scenes through natural supervision – i.e. no labeled data required – but also learns non-specified binary relations between recognized objects, combining visual and textual cues. Ultimately, knowledge learned is formalized and reasoned upon, in a question-answering task. The method is however yet confined to very few specific objects and predefined relations between them. Still, it paves the way towards pragmatic neural extraction of knowledge and subsequently towards symbolic inferencing over new knowledge.
The Neural Symbolic Cognitive Agent (NSCA) [15] uses a Recurrent Temporal Restricted Boltzmann Machine (RTRBM) [16] to learn complex temporal relations between data and subsequently formalize these relations into propositional rules, to be used for subsequent inferencing. The method has been applied to a restricted domain task and only to enrich existing knowledge with non-persistent, non-verified (to be exact, not needing verification) knowledge, however it unveils the capacity of RBMs as a powerful neural method for obtaining associations and rules among entities.
[17] also employ Restricted Boltzmann Machines (RBMs) in order to extract fuzzy rules, encapsulating the uncertainty/vagueness of probabilistic machine learning within logic-compliant rules, something missing from the crisp NSCA method. However, this method lacks comprehensive description of the formalization method for the extracted rules.
The hybrid Diferential Inductive Logic Programming (DILP) method [ 18] combines differential neural-based learning with traditional inductive logic programming to learn and subsequently predict the less than/greater than relationships among visual data of the MNIST digits database and formalizing them into logical rules, however still remaining a crisp and supervised approach.
Inspired by [15] and [17], the presented approach aims at extracting intricate knowledge, i.e. non-predefined associations among data, though RBMs [ 19] and symbolically representing them for further use in knowledge-based inferencing. RBMs were elected due to their capacity for unsupervised learning of probabilities over their input, which can in turn be further analysed to yield fuzzy associations among the input entities.
The proposed approach uses a state-of-the-art algorithm for RBMs, following the benchmarking implementation of RBMs for Deep Belief Nets (DBNs) of [19], but with a more eficient training algorithm than the first implementation, i.e. an extension of the original Contrastive Divergence (CD) algorithm, namely Persistent Contrastive Divergence (PCD) [20].
RBMs compute the probability distribution over pairs of visible and hidden vectors, V and H respectively, by the equation: where and
ℎ (, ℎ ) = ( ⊤ + ℎ⊤ ℎ + ⊤ ℎ )/ , ℎ ∈ hidden vectors respectively,
is the matrix of connection weights and partition function that ensures the validity of the probability distribution [16]. The training = + + algorithm (PCD) for the network finds the states of visible (V) and hidden (H) units that lower the total energy of the network (E), thus maximizing the likelihood of correlation between visible units based on their connection through the network’s hidden units, as described in [20].
,
(, ℎ ) = − ∑ ℎ − ∑ − ∑ ℎ
ℎ
Throughout training we monitored the energy gap between the training and validation sets to
avoid overfitting, as per [
The approach aims at examining the capacities of RBMs to extract persistent, common-sense, semantic relations among classes in datasets of multi-labelled annotated images. The reason why visually-oriented relations’ extraction was opted for the first approach (as opposed to e.g. natural language) is the rudimentary nature and distinguishability of visual data interrelations (as opposed to the complexity and semantic ambiguity of natural language).
In order to examine the domain independence capacities of the approach, the experiment was performed in two distinct datasets.
In order to train our network we used two popular structured image datasets, Visual Genome [21] and GQA [22]. Both contain rich annotations about the objects (object classes) present in an than ≈ 1000 appearances, in order to restrict the computational cost of training, totaling to 300
in [24], as seen in Eq. 3. was assigned a normalized confidence weight ℎ , ∈ The reconstructed input yielded several relations among all ∈ under each ℎ ∈ . However, not all hidden units bore the same significance, since several pertained to connections with wij significantly low activation weights for all ⟷ ℎ . To examine their significance, each
ℎ ∈ , based on the method employed
Based on [25], a cutof mechanism was devised in order to maintain only the "beneficial" i.e. the hidden units whose relative computational cost would still benefit the analysis, while the rest were discarded as noise. To this end, the elbow/knee [25] pertained the cutof point for ℎ the hidden units to be maintained (Eq. 4).
Similarly, a cutof point was devised in order to maintain the "beneficial" visible units per each hidden unit ∀ℎ ∶ ↔ ℎ
. To this end, the elbow/knee ( ), ≥ 0 and ( ), < 0 was computed for all activation weights in the RBM, as the cutof point to prune all visible units within each hidden unit that bore a low, non-beneficial, activation weight.
A graphical representation of the hidden unit confidence and activation distribution curves and their elbow-based cutof points for Visual Genome can be seen in Figure 2. ℎ = ∑ | | (3) ( ) =
′′( ) (1 + ′( )2)15 (4)
To explore the patterns formed in the RBMs’ network structure, spectral analysis was employed.
More precisely, a graph representation = ( , ) was created, per each of the
datasets, based on the pruned RBM network’s structure. The graphs pertain of weighted
vertices, interconnected by weighted edges, where
= ℎ , ∀ ∈ and =
̃ ( ), ∀ ∈ , where ̃ ( ) ∈ [
In order to examine the most prominent associations arising among the interconnected vertices, taking also advantage of the vertex and edge weights, a combinatorial Laplacian matrix with vertex and edge weights [26] was used, with = ∗, where is the weighted incidence matrix and is the weighted edge incidence diagonal of each graph.
In order to explore the RBM results against the ground truth, the results of the RBM’s spectral analysis were compared against the spectral analysis of the co-occurence between object classes in the ground truth. To this end, a graph representation of the ground truth per each dataset was devised, as = ( , ) 3. The graphs again bear weighted vertices and edges, with Ed g e w=eights( are), ∀desi gn∈at e d,wahse re =( ) is the f(r eq, u en)c,ywohf er e ∈ i n( t h, e tra)in∈in{g0d,a1t}as e t. ∑ , denotes the cooccurence between two classes in an image of the dataset.
Lastly, to extract the associations, spectral clustering based on was performed, using DBScan [27] with a relative gap [28], with = 2 as even pair-wise associations are relevant to our goal, = 0.75 set empirically as the optimal radius and = ‖ ‖2 of found to be the optimal relative gap for both RBM-based graphs, but also Ground Truth graphs. Subsequent work aims at also automating retrieval of the optimal based on ’s spectral properties.
The results of this analysis revealed the capacity of RBMs to extract prominent relations among visible units. The extracted clusters of interrelated classes can be seen in the Appendix, in sections A.2 and A.1.
Spectral analysis of the ground truth data graphs already yields results that capture valid common-sense interrelations among input classes. However, spectral analysis over the RBMs has revealed, in many cases, diferent and more intricate relations than in the ground truth data. Ground Truth graphs are found (as expected) to solely depict visual co-occurrence. This is apparent by the diference in number of clusters, as well as from the non-correspondence in semantics that RBM-based spectral analysis produces as opposed to Ground Truth analysis.
Moreover, within RBM-produced clusters, abstractions and specializations that are globally (not only visually) applicable for the observed semantic senses within their domain have been revealed. For example (see Appendix A.2), RBM analysis in Visual Genome was able to discern 3 denotes all classes in the dataset and thus coincide with , efectively making , ∈ between things related to food per se and objects found in an eating area, while the relevant Ground Truth analysis bundled most of these objects under a single cluster related to food. Several abstract relations such as e.g. the ones between logos, letters and design, between words, writing and signs, among diferent body parts, facial parts, animal parts, vehicle parts, etc have only been captured through RBM analysis.
Most interestingly, other non-direct properties were revealed based on seed classes that served as common denominators over related objects. For example, a RBM-produced Visual Genome cluster pertains to lady, dress, skirt, bag, child, male child. Upon further inspection based on VG’s semantic relations among data reveals that this is in fact a two-sense tree, with lady being the common denominator and dress, skirt, bag constituting a semantic branch regarding clothing related to women, while child, male child constituting a disjoint branch4 of types of persons frequently related to women. Such information can enable the unsupervised retrieval of hierarchy or meronymy relations among classes.
Although it is very dificult to evaluate the validity of the produced interrelations due to the lack of a golden standard and to contextual subjectivity, the produced clusters of interrelated objects based on the RBM-graph spectral analysis where compared against the clusters produced from the ground truth-graph spectral analysis, in terms of semantic validity.
Refraining from self-justification of the results, two independent observers studied the produced clusters in order to identify the ’common sense’ semantic contexts of the produced clusters in both RBM and ground truth graphs.
Moreover, the Visual Genome dataset’s RBM-based spectral analysis produced some clusters of similar semantics with its ground truth, which gave the opportunity to measure precision and recall among the semantically similar clusters. The GQA results however did not yield any semantically similar clusters. The results can be seen in Table 1.
Since the semantics of the produced "relation clusters" are not disentangled at this stage, every member of each cluster is considered as generically related to each other member of the cluster.
Therefore, as a first step, a generic isRelatedTo object property can be used to construct generic symbolic rules that express the interrelations among cluster members.
4The members of each branch do not co-occur in the dataset.
To this end, the interrelations extracted can be expressed in Description Logics (DLs) notation as propositional axioms of the form:
⊑ ∀(. ) (5) where C is a class in the dataset, i is a single cluster produced through the spectral analysis of the RBM-based graph and , are classes ∈ cluster i.
In first order logic (FOL), this translates to:
∀. ( ) → (∀. (, ) → ( )) (6)
Based on this representation, several inference tasks can provide richer information while reasoning over given facts in a particular domain. For example, a visual question answering system may use the isRelatedTo property to query, based on a DL or FOL reasoner, additional aspects related to an image, given an instance x of a class ( ) retrieved or explicitly annotated in an image y, by instantiating accordingly isRelatedTo(x,y).
For example, given the VG RBM-produced cluster word.n.01, writing.n.01, sign.n.02, the following axioms may be produced: ⊑ ⊑ ∀(. ∀(. ) ) (7a) (7b) ⊑ ⊑ ∀(. ∀. ) (7d) (7e) ⊑ ∀(. ) (7c) ⊑ ∀(. ) (7f) Based on this knowledge, a question-answering system using a knowledge-based (deductive) inference engine can answer the question "What is related to this image?" given an image imagename, where a sign(infosign) was identified, by grounding the isRelatedTo property for isRelatedTo(infosign, imagename). Through axioms 7d and 7e, we get: sign(infosign) ⊑ ∀ (infosign,imagename. ( )) ⊨ (imagename) sign(infosign) ⊑ ∀ (infosign,imagename). ( ) ⊨ (imagename) Efectively, the contents of the image has some relation to words and writing, given that sign exists in the image.
This paper presented an early neuro-symbolic integration approach for unsupervised symbolic knowledge extraction from trained neural networks, by using RBMs to uncover persistent semantic associations among concepts found in multi-labelled images of the Visual Genome and GQA datasets. Valuable insights arose in the process about the capacity of RBMs and spectral analysis to uncover relational knowledge from data.
The purpose of this first proof-of-concept work is to determine the validity of the assumption that the obscure patterns that are formed in trained NNs may be captured symbolically, to some extent, as structured knowledge, able to construct, update or complement knowledge bases. The goal of this process is to be task-independent and that the knowledge acquired can be further re-used, shared and used for complex reasoning. The approach aims to be applicable to any domain for which data is available, leverage the need for huge data and yield consistent results in terms of extracted propositions’ accuracy.
The next steps will delve into examining whether the same applies in deeper architectures and what more/else deeper networks may reveal, by expanding the approach from RBMs to DBNs. The directed nature of the added layers in DBNs5, in combination with the capacity of the current approach to recognize the k-most significant hidden units of the base RBM layer, are expected to consist the first step of disentangling the top-level semantics of the produced generic relations and improve the accuracy of the RBM results.
Further, a modular NN approach will be sought after in order to project the results of the DBNs to natural language, in order to extract the specific semantic associations among related input classes.
This work has been supported by the European Commission under Grant Agreement No. 787061 ANITA. [13] R. Hu, M. Rohrbach, J. Andreas, T. Darrell, K. Saenko, Modeling relationships in referential expressions with compositional modular networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1115–1124. [14] J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, J. Wu, The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision, arXiv preprint arXiv:1904.12584 (2019). [15] L. de Penning, A. Garcez, L. C. Lamb, J. Meyer, A neural-symbolic cognitive agent for online learning and reasoning, in: Proceedings of the Twenty-Second international joint conference on Artificial Intelligence, volume 2, IJCAI, 2011, pp. 1653–1658. [16] I. Sutskever, G. E. Hinton, G. W. Taylor, The Recurrent Temporal Restricted Boltzmann
Machine, in: Advances in neural information processing systems, 2009, pp. 1601–1608. [17] E. De la Rosa, W. Yu, Data-driven fuzzy modeling using Restricted Boltzmann Machines and probability theory, IEEE Transactions on Systems, Man, and Cybernetics: Systems (2018). [18] R. Evans, E. Grefenstette, Learning explanatory rules from noisy data, Journal of Artificial
Intelligence Research 61 (2018) 1–64. [19] G. E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets, Neural computation 18 (2006) 1527–1554. [20] T. Tieleman, Training Restricted Boltzmann Machines using approximations to the likelihood gradient, in: Proceedings of the 25th international conference on Machine learning, 2008, pp. 1064–1071. [21] R. Krishna, Y. Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma, M. Bernstein, L. Fei-Fei, Visual Genome: Connecting language and vision using crowdsourced dense image annotations, 2016. URL: https://arxiv.org/abs/1602.07332. [22] D. A. Hudson, C. D. Manning, GQA: A new dataset for real-world visual reasoning and compositional question answering, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 6700–6709. [23] G. A. Miller, WordNet: a lexical database for English, Communications of the ACM 38 (1995). [24] S. N. Tran, A. d. Garcez, Knowledge extraction from deep belief networks for images, in:
IJCAI-2013 workshop on neural-symbolic learning and reasoning, 2013. [25] V. Satopaa, J. Albrecht, D. Irwin, B. Raghavan, Finding a" kneedle" in a haystack: Detecting knee points in system behavior, in: 2011 31st international conference on distributed computing systems workshops, IEEE, 2011, pp. 166–171. [26] F. R. Chung, R. P. Langlands, A combinatorial laplacian with vertex weights, journal of combinatorial theory, Series A 75 (1996) 316–327. [27] M. Ester, H.-P. Kriegel, J. Sander, X. Xu, et al., A density-based algorithm for discovering clusters in large spatial databases with noise., in: Kdd, volume 96, 1996, pp. 226–231. [28] P. Kreutzer, G. Dotzler, M. Ring, B. M. Eskofier, M. Philippsen, Automatic clustering of code changes, in: 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR), IEEE, 2016, pp. 61–72.
Table 4 portrays the clusters of interrelated classes produced through spectral analysis of an RBM trained on the Visual Genome dataset (objects) of most prominently co-occurring classes produced through spectral analysis of the Visual Genome dataset’s ground truth respectively.
It is worth mentioning that some Visual Genome object classes are represented with a Wordnet word/term that does not reflect the sense which the Visual Genome data signify. These terms are marked with * in Table 4 and their proper semantics/senses are listed in Table 2. kitchen.n.01, spoon.n.01, paw.n.01, bear.n.01, fur.n.01, cake.n.03, banana.n.01, food.n.01, sauce.n.01, plate.n.04, wave.n.01, ocean.n.01, surfboard.n.01, Things at the ring.n.01, finger.n.01, On a hand beach.n.01, beach telephone.n.01, vegetable.n.01, Related to vege- sauce.n.01, bread.n.01, Related to food root.n.03, tation tomato.n.01, vegetable.n.01, meat.n.01, cheese.n.01, fork.n.01, napkin.n.01, spoon.n.01, tray.n.01, pizza.n.01, knife.n.01, food.n.01, banana.n.01, cake.n.03, root.n.03, room.n.01, drawer.n.01, cabinet.n.01, Things in a room court.n.01, racket.n.04, player.n.01, Related to tennis skirt.n.01, bird.n.01, beak.n.01, Related to birds bat.n.05, new_jersey.n.01*, uni- Related to form.n.01, sports/ baseball sink.n.01, faucet.n.01, bathroom.n.01, Things in a bath- sink.n.01, faucet.n.01, bath- Things in a bathroom room.n.01, toilet.n.01, room uniform.n.01, ball.n.01, Related to animal.n.01, cow.n.01, sheep.n.01, Related to farm new_jersey.n.01*, player.n.01, bat.n.05, sports/baseball animals word.n.01, writing.n.01, sign.n.02, Related to writ- kitchen.n.01, drawer.n.01, Things in a ing kitchen arrow.n.01, trafic_light.n.01, Related to trafic beak.n.01, bird.n.01, Related to birds signs computer.n.01, laptop.n.01, screen.n.01, Things in a sit- hoof.n.01, zebra.n.01, mane.n.01, Related to anitelevision.n.01, wire.n.01, keyboard.n.01, ting room horn.n.01, mals pillow.n.01, desk.n.01, bed.n.01, goggles.n.01, board.n.02, ski.n.01,
Related to snow computer.n.01, keyboard.n.01, lap- Things in a sitsports top.n.01, desk.n.01, screen.n.01, ting/study room television.n.01, skateboard.n.01, ramp.n.01, mountain.n.01,
hill.n.01, Related to path flower.n.01, Related to vege- trafic_light.n.01, arrow.n.01, litation cense_plate.n.01, Farm animals ski.n.01, goggles.n.01, Related to skateboard On the street Related to snow sports
Commonsense semantics bread.n.01, Related to food pizza.n.01, cheese.n.01, RBM cluster Ground Truth cluster