In this paper, we study the efects of diferent node and edge weighting strategies of graph-based semantic representations on the accuracy of a scholarly paper recommendation scenario. Our semantic representation relies on the use of Knowledge Graphs (KGs) for acquiring relevant additional information about concepts and their semantic relations, thus resulting in a knowledge-rich graph document model. Recent studies have used this representation as the basis of a scholarly paper recommendation system. Even when the recommendation is made based on the comparison of graphs, little has been explored regarding the efects of the weights assigned to the edges and nodes in the representation. In this paper, we present the initial results obtained from a comparative study of the efects of diferent weighting strategies on the quality of the recommendations. Three weighting strategies for edges (Number of Paths (NP), Semantic Connectivity Score (SCS), and Hierarchical Similarity (HS)) and two for nodes (Concept Frequency (CF) and PageRank (PR)) are considered. Results show that the combination of the SCS and CF outperform the other weighting strategy combinations and the considered baselines.
In recent years and with the advent of the Semantic Web and
Linked Open Data (LOD), new semantics-aware content
representations have been proposed for the next generation of recommender
systems[
In previous works by the authors, a semantic representation that
exploits the structured knowledge present in KGs for the task of
recommending scholarly papers was proposed [
In the proposed semantic representation weights are assigned to
both nodes and edges. The weights of the nodes consider the
importance of the concept in the document, while the edges’ weights
capture the degree of associativity between concepts in the KG.
Although existing graph similarity measures can exploit these weights,
the efect of diferent weighting strategies in the recommendation
has not been fully explored. In this paper, we present initial results
of a comparative study on the recommendation performance using
diferent weighting strategies for a graph-based representation. The
evaluation is done using a scholarly paper recommendation dataset
that contains the user profiles of eleven professors of computer
science [
The paper is organized as follows: Section 2 reviews some related work in semantics-aware recommender systems. Section 3 presents the semantic representation process and its diverse modules. Sections 4 and 5 present the considered weighting functions. Section 6 describes the evaluation framework and Section 7 the results. Finally, conclusions and directions for future work are discussed in Section 8. 2
Recent contributions to semantics-aware recommender systems
have focused on exogenous semantic representations that
introduce the semantics by linking the recommendation item to a KG
[
To build the semantic representation, we begin with the identification of concepts mentions in the text (i.e. annotations). Diferent tools for entity linking and word sense disambiguation can be used for this task. As an example, consider the short input text in Figure 1. After using an automatic annotation tool a set of URIs corresponding to KG concepts is returned. It is important to mention that no human verification is performed on the set of annotations retrieved by the automatic tool. Then, expansion and filtering processes that consider the semantic relationships found in the KG are applied.
The expansion process is used to enrich the representation with concepts that are not explicitly mentioned in the text or not identified by the annotation tool but are strongly related with annotations. Our previous results show that expanded concepts can be important to reinforce the main topic of the document even if they do not occur in the text. The expansion process adds more discriminative power to the representation. We expand the set of annotations to new related ones following two diferent approaches: category-based and property-based. The category-based expansion incorporates the hierarchical information of the concepts. For the “Artificial_Intelligence” concept, for example, the category “Category:Computational_neuroscience” is retrieved and incorporated into the representation. For property-based expansion, the KG ontology is navigated and a set of related concepts is incorporated. For example, from the “Robotics” annotation the concept “Cooperation” is retrieved by following the “wikiPageWikiLink” property in the ontology. Only outgoing links from a given annotation are considered.
The filtering strategy seeks to eliminate irrelevant and possible
noisy concepts in the representation. The noisy concepts can be the
result of incorrect annotations, of-topic concepts found in the text
or added in the expansion step [
Diferent property path lengths between concepts can be considered in the filtering process, so diferent semantic representations can be produced for the same input text. The result of these processes is a graph whose nodes are concepts and edges express the existence of a linkage between two concepts in the KG. Finally,
the importance of each concept and the strength of the edges are evaluated via diferent weighting functions (see Sections 4 and 5). The resulting representation follows Definition 1.
Definition 1. The semantic representation Gi of a profile/document pi is a directed weighted graph Gi = (Ni , Ei , w(c), w(e)), where both nodes and edges have an associated weight defined by the functions w(c) : N → + and w(e) : E → +. The set of nodes Ni = {c1, c2, ..., ck } are entities/concepts belonging to the space of a KG. The node weight w(c) denotes how relevant the node c is for the profile/document. An edge between two nodes (ca , cb ) represents the existence of at least one statement in the KG that links both concepts. The weight of the edge w(e) denotes how strong this linkage is between the considered concepts. 4
Íτ l=1 |pathsc<il,c>j | , Number of paths (NP). NP is defined as N P (ci , cj ) = M P where |pathsc<il,c>j | is the number of paths of length l between concepts ci and cj in the KG, and τ is the maximum length of path considered. MP is a normalization parameter that is equal to the maximum number of paths found for a pair of concepts in the representation.
The Semantic Connectivity Score (SCS) [
Hierarchical Similarity (HS). We use the hierarchical information of the concepts in the KG to calculate a measure of similarity between the concepts. The higher the similarity, the stronger the link between the concepts. If A is the set of categories for the concept ci and B is the set of categories for the concept cj , HS is defined as:
HS(ci , cj ) =
max (cati ∈A,catj ∈B) taxsim(cati , catj ) (1) taxsim =
δ (root , catlca ) δ (cati , catlca ) + δ (catj , catlca ) + δ (root , catlca )
Given the hierarchy of categories T , the catlca of two categories cati and catj is the vertex of greatest depth in T that is the common ancestor of both cati and catj . δ (cata , catb ) is the number of edges on the shortest path between cata and catb . 5
Concept frequency (CF) + Discounts. Inspired by TF-IDF, CF
analyzes the occurrences of the concept in the input content as
well as the frequency in the set of semantic representations. CF is
defined as: CF (c) = wcf (c) × loд mMc , where wcf (c) represents the
number of times that c appears in the input content, M is the total
number of documents/profiles in the dataset and mc is the number
of documents/profiles with the concept c in their representation.
After the expansion process, the representation could be diverted
towards frequent properties in the set of instances of the KG or
general categories in the hierarchical structure of the KG. For the
categories added through the expansion the following discount is
applied: CFdiscount (cat ) = CF (cat ) × loд1(S P ) × loд(1SC) where SP
is the set of concepts belonging to the category and SC is the set
of sub-categories in the category hierarchy. The idea behind this
discount strategy is that categories that are too broad and generic
are penalizes [
PageRank (PR): PageRank is a well-known node ranking algorithm. We use the PageRank version for directed weighted graphs (i.e. it considers the edge weights). Therefore, the results obtained with this centrality measure depends on the resulting link structure in the semantic graph and the associated edge weight. 6
Our main goal is to analyze the influence of the diferent weighting strategies in the context of scholarly paper recommendation. We compare the quality achieved by the same recommendation algorithm when inputting semantic representations for user profiles and documents using the diferent weighting strategies. In this regard, we embrace the content-based algorithm described in Definition 2.
Definition 2. Recommendation Algorithm: given a user profile u
and a set of candidate scholarly papers SP = {p1, ..., pn }, which are
represented using the graph-based representation in Definition 1,
the recommendation algorithm ranks the candidate items according
to their Graph Similarity (GS) to the user profile. For GS we employ
the edit distance implemented in [
GS(Gi , Gj ) =
GSnodes (Gi , Gj ) + GSedдes (Gi , Gj )
2 αnodes + Íc ∈Ni ∩Nj |wi (c) − wj (c)| GSnodes (Gi , Gj ) = 1 − αnodes + Íc ∈Ni ∩Nj max (wi (c), wj (c)) αedдes + Íe ∈Ei ∩Ej |wi (e) − wj (e)| GSedдes (Gi , Gj ) = 1 − αedдes + Íe ∈Ei ∩Ej max (wi (e), wj (e)) (2) where αnodes and αedдes are defined as: Õ Õ Õ αnodes = αedдes = c ∈Ni Õ e ∈Ei wi (c) + wi (e) + c ∈Nj Õ e ∈Ej wj (c) − wj (e) − c ∈Ni ∩Nj
Õ e ∈Ei ∩Ej wi (c) − wi (e) −
Õ c ∈Ni ∩Nj Õ wj (c) wj (e) e ∈Ei ∩Ej
GS evaluates the similarity between two graphs in terms of the weights diferences of the common nodes/edges compared to the total weight of the nodes/edges in the two graphs. Therefore, two graphs are similar not only if their nodes/edges coincide but also if their weights are close in magnitude. 6.1
We employ the dataset proposed in [
For the construction of the semantic representation we use: (i) DBpedia as KG, (ii) DBpedia Spotlight as annotation service, (iii) a maximum path length of 2 for filtering and edge conformation as well as for the τ parameter, (iv) a minimum degree value α = 1, (v) categories extracted through dct:subject to calculate HS. 7
The performance of the recommender system was evaluated by typical metrics for the evaluation of Top-N recommender tasks: MRR (Mean Reciprocal Rank), MAP@10 (Mean Average Precision), and NDCG@10 (Normalized Discounted Cumulative Gain). We select N=10 as the recommendation objective since it is a common rank and it fits with the minimum number of relevant documents per user in the dataset.
Table 1 presents the results obtained by the diferent combinations of the weighting strategies. We use a classical content-based recommendation algorithm baseline that ranks candidate items according to their cosine similarity with the user profile. In this case, profiles and documents use a standard bag-of-words Vector Space Model (VSM) representation. According to Beel et al. VSM HS HS
CF PR
MRR
0.503
with TF-IDF is the most frequent profiling and weighting scheme in
research paper recommender systems [
We can see that the combination of SCS-CF outperforms other weighting strategies and the baselines in terms of MAP and NDCG. In terms of MRR, the best weighting strategy is SCS-PR. Although SCS and NP consider the number of existing paths, the results show a better performance for SCS. This can be attributed to the fact that in addition to the number of paths, SCS also penalizes the path length through a damping factor. Further experimentation is required to evaluate the efect of the β damping factor. HS presents the worst results among the weighting strategies and only slightly better than the cosine baseline.
Regarding the node weighting strategies, results show that CF is more appropriate. CF is superior to PR in terms of MAP and NDCG. On the other hand, according to the MRR, in average PR ranks higher the first relevant result. The comparison with VEO also shows that the consideration of weights for nodes and edges improves the semantic representation proposed for the task of recommending scholarly papers. 8
In this paper, we explored diferent node and edge weighting strategies for a graph-based semantic model for representing user profiles and papers in the context of the scholarly paper recommendation task. The combination of SCS and CF as weighting strategies for edges and nodes respectively presented the best results. SCS considers the number of existing paths in the KG between the concepts considered, while CF considers the frequency of the concept in the document/profile. In the near future, we plan to explore other measures of centrality for nodes weighting (e.g., betweenness, closeness). We also want to explore the efect of path lengths greater than 2 for the calculation of SCS/NP. Finally, according to our recommendation algorithm and graph similarity measure (Equation 2), the contribution of edges and nodes are considered equivalent. We want to test this assumption by favoring/disfavoring the contribution of nodes and edges.
This work was partially supported by COLCIENCIAS PhD scholarship (Call 647-2014).