A knowledge graph (KG) represents a domain of interest with a graph where some of the involved entities are linked with an edge. Knowledge Graph Completion (KGC) is a well-known task for KGs which requires finding missing connections. KGC has been studied for many years with multiple solutions available based on both symbolic and sub-symbolic techniques. In this paper, we would like to answer the question: can parameter learning for Probabilistic Logic Programming be a competitive algorithm to solve the KGC task? An empirical evaluation on the most common KGC datasets allows us to provide a negative answer to such a question.
KGs are graph-based representations of knowledge in terms of relationships between entities. A KG can be represented as a set of triples (ℎ, , ) where r is the relation, and h and t are the head (start) and tail (end) entities of the relation, respectively.
Real-world KGs are usually incomplete and sparse, thus inference of missing information (entities
or relationships) is often required. This task is referred to as KGC. Depending on the information to
be inferred, KGC can be divided into specific tasks [
Given a completion task (ℎ, , ?) on a graph G, the goal is to find an entity t such that (ℎ, , ) ∈/ G is true. Candidate triples are associated with a score and they are sorted in descending order. Given an answer , its rank () is the position in the list of answers. Special care has to be taken in case there is a group with several answers with the same score. In this case, there are various approaches for computing rank(t). For example, minimum and maximum ranking take respectively the lowest and the highest rank in the group, while average ranking assigns instead the average rank of the group. In our implementation, we always consider the average ranking, even if other tested tools may adopt diferent strategies.
The graph G is often split into three datasets, training set , test set , and validation set , used to train the model and evaluate its performance. Given a set of test triples , KGC algorithms are usually evaluated on link prediction tasks in terms of the following metrics: • Mean Rank (MR), which is the average rank of the test triples: • Mean Reciprocal Rank (MRR), which is the average reciprocal individual rank of the test triples: • Hits@K, which represents the proportion of the test triples ranked in the top positions: = 1
∑︁ () || ∈ =
1 ∑︁ 1 || ∈ () @ = |{ ∈ | () ≤ }| ||
Often, the ranking for a test triple (ℎ, , ) includes a candidate ′ such that (ℎ, , ′) ∈/ . Even if
′ is not the correct prediction, a triple (ℎ, , ′) may appear in the training or validation set. To not
penalize the ranking of the correct candidate , predictions that appear in triples of the training or
validation set can be filtered out. By doing so, a filtered version of the metrics will be computed [
AnyBURL [
Given a set of triples {(0, ℎ0, 1), (1, 1, 2), . . . , (, , +1)}, we can construct a ground path
rule of the form ℎ(0, 1) ← 1(1, 2), . . . , (, +1). AnyBURL extracts three types of rules
(listed in Table 1): i) rules that generalize acyclic ground path rules, i.e., rules where 0 ̸= +1 (called
AC2); ii) rules that generalize cyclic ground path rules, i.e., rules where 0 = +1 (called C); iii) rules
that generalize both acyclic and cyclic ground path rules (called AC1). This process continues until
a certain saturation parameter (which depends on the contribution to the overall performance of the
considered rules) is reached. At each iteration, the length of the considered path is increased by 1. The
score of a rule is the confidence, computed as follows [
support () |{ | ∃ }| where is a grounding for variables and appearing in the head ℎ of , is a grounding for the variables appearing in the body of other than and , and is the union of and .
To score entities, AnyBURL adopts the maximum aggregation, in which candidate entities are ordered according to the maximum confidence of all the rules that generated them, i.e., () = {conf (1), . . . , conf ()} where is the entity and are the rules that predicted it.
When the number of rules is high, computing KGC metrics is very expensive. To overcome this,
AnyBURL exploits multiple threads and limits the search for candidates if during the grounding of a
rule a branch with more than 104 children is found [
One drawback of the maximum aggregation is that it considers only a single rule instead of considering a weighted combination of all the rules that predict a certain entity. Noisy-Or aggregation tries to address this by assuming that the confidence of the rule is a probability and by considering the Noisy-Or of the predictions: () = 1 − ∏︁(1 − conf ()) =1
However, Noisy-Or aggregation does not account for rules redundancy, which often occurs in
realworld KGs, resulting in worse performance than Maximum aggregation due to an overestimation
of the confidence because of double counting [
ℎ : Π : − 1, . . . , where the single atom in the head is built over predicate target/a, which is the target of learning, and where is the arity. Bodies of the clauses may contain other predicates than target/a, called input predicates, that are certain, meaning that their facts and rules have a single atom in the head with probability 1. To compute the probability of a query q, it is necessary to find only the number of ground instantiations of clauses so that the body is true and the head is equal to q. If clause has − instantiations, { 1, . . . , }, every corresponds to a random variable and ( = 1) = Π, while ( = 0) = 1
Π. A query q is true if at least one random variable for a rule is true, and is false only if none of the random variables is true. All the random variables are mutually independent, so () = 1
Given a set + = {1, . . . , } of positive examples (i.e., a set of atoms), a set − = {+1, . . . , } of negative examples (i.e., a set of atoms), and a background knowledge defining the input predicates, LIFTCOVER learns the parameters of the clauses using either Expectation-Maximization (EM) or Limited-memory BFGS (LBFGS). The likelihood of the examples can be unfolded as where () is the number of instantiations of whose head is () and whose body is true, and − = ∑︀ =+1 . The gradient of the likelihood is computed as: )︃ ⎞ The equation Π = 0 does not admit a closed-form solution, thus optimization is needed to find the maximum of . Finally, the clauses with a probability below a user-defined threshold are discarded.
The EM algorithm [17] is used to find the maximum likelihood estimates of parameters. First, in the Expectation step, the distribution of the unseen variables in each instance is computed given the observed data and the current value of the parameters. Then, during the Maximization step, the new =1 = ∏︁(1 − Π)− ∏︁ 1 −
∏︁(1 − Π) (︃ =1 =1 Π =
1 − Π ⎛ ⎝ ∑︁ =1 ︂( 1
︂) () − 1 − − ⎠ parameters are computed so that the likelihood is maximized. The algorithm repeats these two steps until there are no more improvements in the likelihood. To use the EM algorithm, the distribution of the hidden variables given the observed ones, ( = 1|) and ( = 1|¬), has to be computed. Given that ( = 1, ) = (| = 1) · ( = 1) = ( = 1) = Π since (| = 1) = 1, Since ( = 1, ¬) = (¬| = 1) · ( = 1) = 0 and (¬| = 1) = 0, ( = 1|) = ( = 1, ) () =
Π 1 −
∏︀=1(1 − Π) ( = 0|) = 1 − 1
Π −
∏︀=1(1 − Π) ( = 1|¬) = 0 ( = 0|¬) = 1
LIFTCOVER+ [18, 19] is a modified version of LIFTCOVER that learns the parameters and the structure of liftable probabilistic logic programs using regularization to prevent overfitting. Moreover, it replaces LBFGS with gradient descent to optimize the likelihood.
Regularization is a widely used strategy to inhibit overfitting, by introducing a penalty term in the loss function to penalize large weights. Since clauses with small weights are removed because they have little influence on the probability of the query, fewer clauses with large weights, i.e., a smaller theory, should be obtained. Bayesian, L1, or L2 regularization can be used in the Maximization step of the EM algorithm. The main diference in L1 and L2 regularization lies in the penalty introduced in the loss function. The L1 penalty term is the sum of the absolute values of the parameters, while the L2 penalty term is the sum of their squares. Furthermore, L1 tries to bring the parameters close to 0 to create a sparse model, i.e., a model with few non-zero parameters. However, L1 is computationally ineficient. On the other hand, L2 is eficient but produces non-sparse solutions. In both cases, the impact of regularization is controlled by a regularization coeficient: the higher it is, the stronger the regularization will be.
The L1 objective function [20] is: while the L2 objective function [20] is: 1( ) = 1 · + 0 · (1 − ) − 2( ) = 1 · + 0 · (1 − ) − 2 2 where = , 0 and 1 are the expected occurrences of = 0 and = 1 respectively computed during the Expectation step, and is the regularization coeficient.
The value of that maximizes the objective function is computed in the Maximization step by solving the equation ( ) = 0 [20]. 1( ) is maximum at: 1 =
41 2( + 0 + 1 + √︀(0 + 1)2 + 2 + 2 (0 − 1)) while 2( ) is maximum at:
2
⎜ √︁ 30+31+ cos ⎜ ⎛ ⎜ ⎝ arccos ⎝ ⎛ √︂ 30+31+
︂( 920 − 91+ ︂) ⎞ 30+31+ 3 ⎠ − 2 =
⎞ 23 ⎟⎟⎟ ⎠ + 1
In Bayesian regularization, parameters are updated assuming a prior distribution in the form of a Dirichlet probability density with parameters [, ], where a and b are the number of extra occurrences observed of = 1 and = 0 respectively. Parameters are shrunk when b is much larger than a.
To apply EM, we need to store the values − for each rule and the values for each clause and each positive example . We can store these values with vector − and matrix + respectively, of size the first and × the latter. We have implemented the operations to be performed in the Expectation and Maximization phases as vector/matrix operations on − and +. LIFTCOVER+ now ofers implementations of the algorithm in Prolog and in Python using the Janus interface [ 21, 22]. The Python version can exploit either the CPU with package numpy or the GPU with package cupy.
In this section, we describe the solutions we considered to generate rules whose probabilities are learnt via LIFTCOVER+.
Here we generate only cyclic rules by generating tuples of relations 0, 1, . . . , and then converting them to rules. The tuples of relations have + 1 elements where the first element is the relation that goes into the head of the clause and the other are those that go in the body, thus obtaining a body with atoms. For this phase, each triple (ℎ, , ) in the dataset is translated into an atom (ℎ, , ). For example, the tuple (r0,r1,r2,r3) is translated into the rule r(A,r0,B):-r(A,r1,C),r(C,r2,D),r(D,r3,B). where /3 is defined as r(S,R,T):
t(S,R,T). r(S,i(R),T):
t(T,R,S). so that both the relation (R) and their inverse (i(R)) are considered. The tuples of relations are obtained by starting from triples in the training set. For example, if we want to extract 4 relations we look for values of R1,R2,R3,R4 such that the query r(h,R1,C),r(C,R2,D),r(D,R3,E),r(E,R4,t). succeeds at least once. Then duplicate tuples are removed.
Removing duplicates can be very expensive, so we looked for diferent ways to do it. In the following, we consider SWI Prolog [23] as inference engine. We tried with tabling [24], which has the property of storing each answer only once, but it was faster to directly use tries1. Moreover, we exploited parallelism in order to speed up the computation. In particular, we used the SWI library predicate concurrent_maplist(Goal,List) that applies Goal to each element of List in parallel using threads. We also wrote a helper predicate chunks(List,N,SubLists) that splits the elements of List into N lists of approximately equal length and returns them in SubLists. For example, to generate cyclic rules with 3 atoms in the body that contain the training tuples, we used the code below: main:tell('tuples3.pl'), rels(Rels), chunks(Rels,32,Chunks), 1https://www.swi-prolog.org/pldoc/man?section=trie trie_new(Tr), concurrent_maplist(genpaths3(Tr),Chunks), trie_gen(Tr,q(R,R1,R2,R3)), write('(tt(A,'), write_canonical(R), write(',B):0.5 :- r(A,'), write_canonical(R1), write(',C), r(C,'), write_canonical(R2), write(',D), r(D,'), write_canonical(R3), writeln(',B)),'), fail. main:
told. genpaths3(Tr,Rels):member(R,Rels), path3(R,R1,R2,R3), trie_insert(Tr,q(R,R1,R2,R3)), fail. genpaths3(_,_). path3(R,R1,R2,R3):r(S,R,T), r(S,R1,I), T\=I, r(I,R2,J), r(J,R3,T), J\=S. where /1 is a predicate that computes the list of all possible relations. These are then split into 32 chunks and each chunk is processed in parallel by a separate thread. Since operations on tries are thread-safe, parallelization does not pose a problem. The reason for the tests T\=I and J\=S is to avoid generating rules such as r(S,r,T):-r(S,r,T),r(T,r1,J),r(J,i(r1),T). r(S,r,T):-r(S,r1,I),r(T,i(r1),S),r(S,r,T).
This approach allows a quick sampling of a fraction of the tuples by generating a random number
in [
With this approach, for each relation in the KG, we find the lists of start (head) and end (tail) nodes. Then, we find a user-defined number of paths starting from each start (end) node up to a maximum length also specified by the user. These are used to generate AC1, AC2, and C rules. Here as well duplicates are removed and each rule is associated with an initial probability value of 10− 4. This set of rules is passed as input theory to LIFTCOVER+ to tune the probabilities. The rankings of the candidates for an entity are computed via Noisy-OR aggregation.
We tried another approach to generate rules: for each dataset, we extract all the relations together with their number of occurrences. Call the list with all the relations . We associate each relation with a probability proportional to its occurrences. To generate a rule, we proceed as follows: we first sample a relation from to consider as head relation. Then, we select a random number between two and four and sample again the same number of relations from , that will be considered for the body. Both sampling phases take into account the probability associated with each relation. Starting from the selected relations, we generate rules by creating an atom with two variables for each relation and then connect the atoms to create a chain rule. To clarify, suppose we have 0 for the head and 1 and 2 for the body. We obtain the rule r(A,r0,B) :- r(B,r1,C), r(C,r2,D). We repeat this process for a ifxed number of rules. Once we have all the rules, we compute the cumulative number of instantiations for all the bodies, call it , and associate each rule with a score given by the ratio between its number of body instantiations and . This method is implemented in Prolog as well.
In this section, we report the experimental evaluation of our approach on well-known KGC datasets.
In our experiments, we consider the following datasets: FB15K-237 [25] which contains entity pairs extracted from FreeBase, WN18RR [26] which contains WordNet entities and explicit relations, NELL [27] which was built with an agent (called Never-Ending Language Learner) that reads the web and learns over time, and Nations [28] which contains data about economic, diplomatic, military, and social dyadic interactions among a reduced set of 182 nation-dyads in the years between 1950 and 1965. The number of tuples, relations, and entities for each dataset are listed in Table 2.
We learn the parameters of the rules using the EM algorithm of LIFTCOVER+ because it gives solutions with better quality than gradient descent and LBFGS, and run it on GPUs to speed up computations. However, the need to compute and store the matrix + for the positive examples (see Section 2.3) requires limiting the number of positive examples and rules. In fact, with rules, positive examples, if we assume that an integer requires 32 bits (4 bytes) of storage, storing the matrix + requires · · 4 bytes. For example, with 105 rules and 105 positive examples, we need 40GB of memory, close to the limit of the available hardware that is available to us (64GB). Thus we need to subsample examples for the FB15k-237 and NELL datasets, since they have an excessive number of examples. In particular, we sampled 81,196 positive examples for FB15k-237 and 4,670 for NELL. Moreover, we randomly generated 105 negative examples for FB15k-237, 2 · 105 for NELL, 2 · 105 for WN18RR, and 5 · 103 for Nations by repeatedly randomly sampling a relation , a head ℎ, and a tail and checking whether the triple (ℎ, , ) is not present in the KG until the required number of examples is obtained.
We implemented in Prolog an algorithm for computing the Hits@1, Hits@3, Hits@5, Hits@10, and MRR metrics. Algorithm 1 shows the pseudocode of function Metrics that, given the test set and the set of rules, returns the metrics. The function, for each triple (ℎ, , ) in the test set, calls function ComputeAns((ℎ, , ), ℎ) that returns a set of pairs (, ′) for answers ′ to (ℎ, , ). ComputeAns ifrst computes the probability of the correct answer . Then, it finds all possible instantiations of the query (ℎ, , ) using the Prolog code setof(r(h,r,T),Prog^find_inst(Prog,r(h,r,T)),Atoms) where find_inst/2 is defined as find_inst(Prog,Ex):member(((Head : _P) :- Body),Prog),
Head = Ex, Body.
After this, ComputeAns removes the instantiations present in the training or validation set and cycles over the remaining ones, computing their probability and keeping a counter of the number of answers with a probability greater or equal to , the probability of the correct answer. If the probability of an instantiation (ℎ, , ′) is greater or equal to , then the pair ( , ′) is added to the current set of scored answers, and the counter is updated. In fact, if < , ′ need not be stored, as it does not influence the ranking of . Then we test : if it is 20 or more, we can stop examining instantiations, as the rank of would be for sure greater than 10. In fact, the worst case is that there are 20 answers all with the same probability: in that case, the rank of each answer would be 20+1 = 10.5 according to the average approach to ranking computation.
2
Function Metrics computes exact values for Hits@1, Hits@3, Hits@5, and Hits@10, while a pessimistic approximation to MRR, as the actual rank of a correct answer could be lower, while we stop as soon as we have found 19 other answers with a probability greater or equal to that of the correct
MRR
Learning Time (s) answer. Since the values of the hits metrics are exact, this function difers from the one of AnyBURL (also implemented in PyClause). Note that function Metrics can also be parallelized by executing concurrently the loop over the triples of the test set.
Parameter learning was performed on the Leonardo HPC system of Cineca, using machines with one Intel Xeon Platinum 8358, 2.60GHz, 32 cores, 481GB of RAM, and Nvidia A100 GPUs with 64GB of memory. The Weighted Sampling algorithm was run on a machine running at 2.40 GHz with 16 GB of RAM with a time limit of 8 hours.
Table 3 shows, for each dataset, the maximum length of the body of the rules and the number of rules generated with the rule generation methods described in Section 3.1, Section 3.2, and Section 3.3, that here we call respectively Paths, AC1+AC2+C and Weighted Sampling respectively.
Table 4 shows the results of the experiments: for each dataset and algorithm, we report the metrics Hits@1, Hits@3, Hits@5, Hits@10, and MRR together with the time taken by parameter learning. The algorithm Paths+Conf means that we used the confidence as the parameters of the rules without performing EM. Note that the Weighted Sampling algorithm does not have an EM phase. That is, the reported values are the ones obtained only by sampling rules and by weighting them as described in Section 3.3. The confidence is computed using PyClause.
From Table 4 we can see that, for the rules found using the method described in Section 3.1 (called Paths), performing parameter learning using EM is not beneficial. This seems surprising, as tuning the parameters to improve the log-likelihood should produce a better classifier. It may be due to the fact that we had to subsample positive examples and to artificially generate negative examples, for which the test of absence from the training and validation set may not ensure absence from the test set or falsity in general. Also Weighted Sampling has limited performance, since it can only handle two of the four datasets. This is possibly due to the high number of instantiations that need to be found to associate weights with rules. Lastly, Table 4 shows that the approach described in Section 3.2, in which an input theory composed of AC1, AC2, and C rules whose parameters are then learned with LIFTCOVER+, could not be used. Specifically, we proved that although the approach works for small datasets, such as Nations, with very large datasets such as the other three considered, parameter learning with LIFTCOVER+ requires too much memory.
In order to compute the metrics for Paths, we used PyClause with the Noisy-Or aggregation strategy because it was significantly faster than our implementation in Prolog of Algorithm 1. This result is surprising as well, as Prolog systems such as SWI are supported by decades of software engineering expertise in improving the speed of answering this kind of queries, with advanced techniques such as argument indexing. This may be due to the fact that the queries are particularly simple and involve terms that are constants, for which the unification algorithm probably requires an excessive overhead because of its generality. Moreover, PyClause implements the optimizations discussed in Section 4.3 that, while leading to approximate results, could be the reason for the speed. Finally, another possible reason is that metrics computation in PyClause is implemented in C++.
The KGC task has been studied for many years and a plethora of techniques has been developed to
address them. A comprehensive survey can be found in [
There is a large body of research also on parameter learning in StarAI literature and in particular
on probabilistic logic languages. Most of them are based on EM, such as [
In this paper, we leveraged Expectation Maximization algorithms for parameter learning in Probabilistic Logic Programming to solve the Knowledge Graph Completion task. We presented three diferent approaches to generate rules and learn their probabilities with LIFTCOVER+. Empirical results show that EM applied to liftable probabilistic logic programs to solve the KGC task seems to be not beneficial. Furthermore, the computation of the full rank is often unfeasible for datasets except for smaller ones. Future work includes approaching the KGC task with other PLP languages as well as studying alternative ways to generate rules.
This work has been partially supported by the Spoke 1 “FutureHPC & BigData” of the Italian Research Center on High-Performance Computing, Big Data and Quantum Computing (ICSC) funded by MUR Missione 4 - Next Generation EU (NGEU), and by TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No. 952215. Damiano Azzolini, Elisabetta Gentili, and Fabrizio Riguzzi are members of the Gruppo Nazionale Calcolo Scientifico – Istituto Nazionale di Alta Matematica (GNCS-INdAM). We acknowledge the CINECA award under the ISCRA initiative, for the availability of high-performance computing resources and support. Elisabetta Gentili contributed to this paper while attending the PhD programme in Engineering Science at the University of Ferrara, Cycle XXXVIII, with the support of a scholarship financed by the Ministerial Decree no. 351 of 9th April 2022, based on the NRRP - funded by the European Union - NextGenerationEU - Mission 4 “Education and Research”, Component 1 “Enhancement of the ofer of educational services: from nurseries to universities” - Investment 4.1 “Extension of the number of research doctorates and innovative doctorates for public administration and cultural heritage. Understanding and Reasoning, URANIA 2016, volume 1802 of CEUR Workshop Proceedings, Sun SITE Central Europe, 2017, pp. 30–37. [13] A. Nguembang Fadja, F. Riguzzi, Probabilistic logic programming in action, in: A. Holzinger, R. Goebel, M. Ferri, V. Palade (Eds.), Towards Integrative Machine Learning and Knowledge Extraction, volume 10344 of Lecture Notes in Computer Science, Springer, 2017, pp. 89–116. doi:10. 1007/978-3-319-69775-8_5. [14] J. Vennekens, S. Verbaeten, M. Bruynooghe, Logic Programs With Annotated Disjunctions, in: 20th International Conference on Logic Programming (ICLP 2004), volume 3132 of Lecture Notes in Computer Science, Springer, 2004, pp. 431–445. [15] A. Nguembang Fadja, F. Riguzzi, Lifted discriminative learning of probabilistic logic programs,
Machine Learning 108 (2019) 1111–1135. [16] D. Poole, First-order probabilistic inference, in: G. Gottlob, T. Walsh (Eds.), IJCAI-03, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, August 9-15, 2003, Morgan Kaufmann Publishers, 2003, pp. 985–991. [17] A. P. Dempster, N. M. Laird, D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society: Series B 39 (1977) 1–38. [18] D. Azzolini, E. Gentili, F. Riguzzi, Link Prediction in Knowledge Graphs with Probabilistic Logic Programming: Work in Progress, in: J. Arias, S. Batsakis, W. Faber, G. Gupta, F. Pacenza, E. Papadakis, L. Robaldo, K. Ruckschloss, E. Salazar, Z. G. Saribatur, I. Tachmazidis, F. Weitkamper, A. Wyner (Eds.), Proceedings of the International Conference on Logic Programming 2023 Workshops co-located with the 39th International Conference on Logic Programming (ICLP 2023), volume 3437 of CEUR Workshop Proceedings, CEUR-WS.org, 2023, pp. 1–4. [19] E. Gentili, Knowledge Graph Completion with Probabilistic Logic Programming, in: V. Poggioni, S. Rossi (Eds.), Proceedings of the AIxIA Doctoral Consortium 2023 co-located with the 22nd International Conference of the Italian Association for Artificial Intelligence (AIxIA 2023), volume 3670 of CEUR Workshop Proceedings, CEUR-WS.org, 2023. [20] A. Nguembang Fadja, F. Riguzzi, E. Lamma, Learning hierarchical probabilistic logic programs,
Machine Learning 110 (2021) 1637–1693. doi:10.1007/s10994-021-06016-4. [21] C. Andersen, T. Swift, The Janus System: A Bridge to New Prolog Applications, in: D. S. Warren, V. Dahl, T. Eiter, M. V. Hermenegildo, R. Kowalski, F. Rossi (Eds.), Prolog: The Next 50 Years, Springer Nature Switzerland, Cham, 2023, pp. 93–104. doi:10.1007/978-3-031-35254-6_8. [22] T. Swift, C. Andersen, The Janus System: Multi-paradigm Programming in Prolog and Python, Electronic Proceedings in Theoretical Computer Science 385 (2023) 241–255. doi:10.4204/eptcs. 385.24. [23] J. Wielemaker, T. Schrijvers, M. Triska, T. Lager, SWI-Prolog, Theory and Practice of Logic
Programming 12 (2012) 67–96. doi:10.1017/S1471068411000494. [24] T. Swift, D. S. Warren, Tabling with answer subsumption: Implementation, applications and performance, in: T. Janhunen, I. Niemelä (Eds.), Logics in Artificial Intelligence - 12th European Conference, JELIA 2010, Helsinki, Finland, September 13-15, 2010. Proceedings, volume 6341 of Lecture Notes in Computer Science, Springer, 2010, pp. 300–312. doi:10.1007/978-3-642-15675-5_26. [25] K. Toutanova, D. Chen, Observed versus latent features for knowledge base and text inference, in: A. Allauzen, E. Grefenstette, K. M. Hermann, H. Larochelle, S. W.-t. Yih (Eds.), Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, Association for Computational Linguistics, Beijing, China, 2015, pp. 57–66. doi:10.18653/v1/W15-4007. [26] A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O. Yakhnenko, Translating embeddings for modeling multi-relational data, in: C. Burges, L. Bottou, M. Welling, Z. Ghahramani, K. Weinberger (Eds.), Advances in Neural Information Processing Systems, volume 26, Curran Associates, Inc., 2013. [27] A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. Hruschka, T. Mitchell, Toward an architecture for never-ending language learning, in: Proceedings of the AAAI conference on artificial intelligence, volume 24, 2010, pp. 1306–1313. [28] R. J. Rummel, Dimensionality of Nations Project: Attributes of Nations and Behavior of Nation
Dyads, 1950-1965, 1992. doi:10.3886/ICPSR05409.v1. [29] R. Zhang, Y. Mao, W. Zhao, Knowledge graphs completion via probabilistic reasoning, Information
Sciences 521 (2020) 144–159. doi:10.1016/j.ins.2020.02.016. [30] L. A. Galárraga, C. Teflioudi, K. Hose, F. Suchanek, AMIE: association rule mining under incomplete evidence in ontological knowledge bases, in: Proceedings of the 22nd international conference on World Wide Web, 2013, pp. 413–422. [31] Z. Wang, J. Zhang, J. Feng, Z. Chen, Knowledge graph embedding by translating on hyperplanes, in: Proceedings of the AAAI conference on artificial intelligence, volume 28, 2014. [32] E. Bellodi, F. Riguzzi, Expectation maximization over binary decision diagrams for probabilistic logic programs, Intelligent Data Analysis 17 (2013) 343–363. doi:10.3233/IDA-130582. [33] D. Fierens, G. Van den Broeck, J. Renkens, D. S. Shterionov, B. Gutmann, I. Thon, G. Janssens, L. De Raedt, Inference and learning in probabilistic logic programs using weighted Boolean formulas, Theory and Practice of Logic Programming 15 (2015) 358–401. [34] D. Azzolini, E. Bellodi, F. Riguzzi, Learning the parameters of probabilistic answer set programs, in: S. H. Muggleton, A. Tamaddoni-Nezhad (Eds.), Inductive Logic Programming, Springer Nature Switzerland, Cham, 2024, pp. 1–14. doi:10.1007/978-3-031-55630-2_1. [35] B. Gutmann, A. Kimmig, K. Kersting, L. D. Raedt, Parameter learning in probabilistic databases: A least squares approach, in: ECMLPKDD-2008, volume 5211 of Lecture Notes in Computer Science, Springer, 2008, pp. 473–488. doi:10.1007/978-3-540-87479-9_49. [36] D. Azzolini, F. Riguzzi, Optimizing probabilities in probabilistic logic programs, Theory and Practice of Logic Programming 21 (2021) 543–556. doi:10.1017/S1471068421000260.