Despite being studied for over twenty years, Recommender Systems (RSs) still sufer from important issues that limit their applicability in real-world scenarios. Data sparsity, cold start, and explainability are some of the most impacting problems. Intuitively, these historical limitations can be mitigated by injecting prior knowledge into recommendation models. Neuro-Symbolic (NeSy) approaches are suitable candidates for achieving this goal. However, the application of such systems to RSs is still in its early stages, and most of the proposed architectures do not really exploit the advantages of a NeSy approach. To this end, we conducted preliminary experiments with a Logic Tensor Network (LTN), a novel NeSy framework. We used the LTN to train a vanilla Matrix Factorization model using a First-Order Logic knowledge base as an objective. In particular, we encoded facts to enable the regularization of the latent factors using content information, obtaining promising results. In this paper, we show our preliminary results with the LTN and propose interesting future works in this novel research area.
CEUR ceur-ws.org
while neural networks struggle. Then, () symbolic methods are usually explainable by design, while neural networks are black-boxes. Finally, () symbolic methods manage to work in the absence of data (i.e., zero-shot learning), while for neural networks, it is impossible. Ideally, by integrating these two paradigms, it could be possible to obtain a recommendation engine that can deal well with sparsity, address cold-start cases, and make its predictions less opaque.
Many Neuro-Symbolic recommenders have been proposed recently [4, 5, 6]. However, these methods do not totally exploit the advantages of the NeSy system, or they have not been applied to overcome the aforementioned limitations, where we believe they are particularly suited. For example, they implement the symbolic part using neural networks or use too simple logic (e.g., propositional logic) that is not expressive enough to model real-world problems [4]. To this end, in a preliminary experiment, we used a Logic Tensor Network (LTN) [7, 8] to encode logical axioms for enabling regularization of a vanilla Matrix Factorization (MF) based on logical reasoning. We obtained promising results, showing that the benefits of the encoded knowledge increase with the sparsity of the user-item ratings.
In our preliminary experiments on Neuro-Symbolic Integration, we selected Logic Tensor Networks (LTN) [7] as the NeSy framework to perform recommendations. LTN allows learning a neural model using the satisfaction of a FOL knowledge base as an objective. Specifically, it defines a FOL language, called Real Logic, that allows mapping every symbolic expression to the domain of real numbers. By doing so, the logical formulas in the knowledge base form a computational graph that can be used for gradient-based optimization. To this end, Real Logic defines the grounding function , which defines the mapping between the symbolic and real domains. Specifically, individuals are mapped to tensors of real values, variable symbols to sequences of individuals, functional symbols to real functions, and predicate symbols to real functions with output in [0., 1.]. Then, connectives (i.e., ∧, ∨, ¬, ⟹ ) are mapped to fuzzy semantics [9], while quantifiers (i.e., ∀, ∃) are mapped to special aggregation functions (e.g., generalized means).
Intuitively, functional and predicate symbols can be represented as neural networks parameterized by . We refer to (| ) as a parametric grounding, meaning symbol depends on some parameters that can be learned. LTN allows learning parametric groundings by maximally satisfying a specified knowledge base = { 1, … , }, where 1, … , are closed formulas. More formally, the objective of the LTN is ∗ = argmax SatAgg∈ (| ), where (| ) means formula includes some functional or predicate symbols parameterized by . SatAgg ∶ [0., 1.]∗ ↦ [0., 1.] is a formula aggregating operator, usually defined with MEp [7], namely the fuzzy operator that represents ∀.
In what follows, we refer to Likes(, ) as a binary predicate returning whether a user likes an item . Note that ( Likes | ) can be any recommendation model returning the prediction for a user-item pair in the dataset. In our experiments, we implemented ( Likes | ) as a Matrix Factorization model. Specifically, ( Likes | ) ∶ , ↦ ( U ⋅ I⊤ + u + i ), where U ∈ ℝ× , I ∈ ℝ× , u ∈ ℝ , and i ∈ ℝ are the users’ and items’ latent factors, and users’ and items’ biases, respectively. denotes the number of users, the number of items, and the number of latent factors. is the logistic function. Specifically, allows Likes to be interpreted as a fuzzy predicate.
LTN allows defining the recommendation objective as a set of logical axioms. Thanks to the
expressiveness of FOL, one can express fine-grained constraints that can represent complex
loss functions. For example, the loss function for training an MF model could be the following.
∀( +, +) Likes( +, +)
∀( −, −)¬ Likes( −, −)
(
Encoding additional information to regularize the recommendation model is straightforward. This can be done by encoding additional axioms that enable logical reasoning based on side or content information. Following this intuition, we conducted a preliminary experiment [6] to see if LTN could enable an underlying MF model reasoning about some additional content information. In particular, an experiment that drastically reduces the density of user-item ratings showed that the benefits of the encoded knowledge increase with the sparsity of the dataset, proving our NeSy approach has been beneficial in dealing with sparsity. Specifically, we added the following axiom to the previous knowledge base. Note the experiment has been conducted on MindReader1, a movie recommendation dataset providing ratings for both movies and movie genres.
∀( ?, ?)(∃¬ LikesGenre( ?, ) ∧ HasGenre( ?, )) ⟹
¬ Likes( ?, ?)
(
Intuitively, Axiom (
tasks.
This section proposes possible extensions of our NeSy model to solve diferent recommendation In Section 2.1, we showed how LTN could be used to implement a recommendation model based on Collaborative Filtering. It is well-known that CF cannot deal with cold-start cases, as collaborative information for newly added users and items is unavailable. For this reason, hybrid recommendation models have been proposed. The idea is to merge CF with contentbased recommendations to deal with cold-start while maintaining all the advantages of CF. Implementing this idea in LTN is straightforward. It involves adding the following axiom to the knowledge base.
∀, ,
Likes(, ) ∧ Sim(, ) ⟹
Likes(,
)
(
Cross-domain recommendation aims at mitigating data sparsity by transferring knowledge acquired from other domains (e.g., books, songs) to the target domain (e.g., movies). The source domain is usually denser than the target domain, as the objective is to compensate for sparsity in the latter. To this end, LTN can be used as an interface for transferring knowledge between domains. To do that, the following axiom can be added to the knowledge base. ∀(, , )
Sim(, ) ⟹
(
LTN has not been designed just for learning. In particular, it also provides the possibility to
query the knowledge base after the training phase. Querying is the process of grounding the
knowledge base with novel data and checking the satisfaction level of its formulas. We believe
this feature can be used to provide explanations for recommendations. For example, after we
train a model with Axiom (
, ) ∧ HasGenre( , ) Intuitively, if the formula is evaluated with a high truth value, it means our test user likes at least one movie genre of . Note that one can go through the computational graph of the formula to understand precisely which genre it is and provide this finding as an explanation.
In this paper, we proposed diferent ways to use a Neuro-Symbolic approach to mitigate important recommender systems’ limitations and solve interesting recommendation tasks. In particular, we showed the flexibility of LTN in designing heterogeneous recommendation objectives.
I would like to thank my supervisors, Luciano Serafini and Fabio Aiolli, for having proposed working on this stimulating and ambitious research area. Then, I thank my colleague Alessandro Daniele, who provided additional supervision during my Ph.D. journey. Furthermore, I am grateful to the University of Padova for the delivery of this interesting doctoral program and for the excellent formation that was provided during my journey. Finally, I thank Fondazione Bruno Kessler, who sponsored my Ph.D. expeditions worldwide and supported my research career.