=Paper=
{{Paper
|id=Vol-2621/CIRCLE20_08
|storemode=property
|title=Temporal and Topical Profiles for Expert Finding
|pdfUrl=https://ceur-ws.org/Vol-2621/CIRCLE20_08.pdf
|volume=Vol-2621
|authors=Luis M. de Campos,Juan M. Fernández-Luna,Juan F. Huete,Luis Redondo-Expósito
|dblpUrl=https://dblp.org/rec/conf/circle/CamposFHR20
}}
==Temporal and Topical Profiles for Expert Finding==
https://ceur-ws.org/Vol-2621/CIRCLE20_08.pdf
Temporal and Topical Profiles for Expert Finding
Luis M. de Campos
Juan M. Fernández-Luna
Juan F. Huete
Luis Redondo-Expósito
lci@decsai.ugr.es
jmfluna@decsai.ugr.es
jhg@decsai.ugr.es
luisre@decsai.ugr.es
Departamento de Ciencias de la Computación e Inteligencia Artificial,
ETSI Informática y de Telecomunicación, CITIC-UGR,
Universidad de Granada
Granada, Spain
ABSTRACT and automatically extracted from the documents associated to the
We explore the possible advantages of dividing a single heteroge- expert, and could be more easily interpreted.
neous profile into several more homogeneous subprofiles for an The main goal of this work is to study how the results of an
expert finding task. We consider two different dimensions to per- expert recommendation system could be affected by taking into
form this division, topical and temporal, and also a combination of account either the temporal or the topical dimension of data, or both,
both. Topical subprofiles are obtained from a clustering process of in the building process of the profiles. For this purpose, we start
the documents associated to each expert, whereas temporal sub- from a single monolithic profile for each expert, where all terms of
profiles are generated by simply dividing the temporal sequence of the documents associated to this expert are grouped together. Then
documents into several parts. The experiments are carried out in we want to study whether the division of these single profiles into
the domain of political expert finding, using a dataset of parliamen- several subprofiles which are more homogeneous in some sense
tary documents. The results suggest that, although the two types could improve the quality of the recommendations.
of subprofiles increase the quality of the recommendations, topical A way of doing this is to make divisions based on the different
subprofiles and specially the combination of topical and temporal topics in which an expert can be interested, thus obtaining more
subprofiles get the best results. homogeneous subprofiles from a topical point of view. We can
divide the set of documents associated to an expert, for example
KEYWORDS using a clustering technique, in order to get subsets of documents
topically homogeneous, and then generate a subprofile for each of
expert finding, temporal profiles, topical profiles, politics
these subsets.
Another way is to consider groups of documents temporally
1 INTRODUCTION homogeneous, by dividing the temporal sequence into several parts
Expert finding systems are a specific kind of recommender sys- and then to obtain a (temporal) subprofile for each of these groups.
tems [7] where the items to be recommended are people. Given a Finally, we could also combine both dimensions, trying to obtain
query submitted by a user specifying the problem to be considered, subprofiles simultaneously homogeneous both temporally and top-
the expert finding system will return an ordered list of possible ically.
candidates having the required expertise to tackle this problem Although the methods to get homogeneous subprofiles proposed
[4]. Expert finding systems have many applications (in community in this work to improve the expert finding task can be applied to
question answering, the academic world, industry, social media,...) any domain, the experiments will be carried out with a collection of
and there is a growing interest on them [1, 21, 22, 29]. parliamentary documents, where the experts to be recommended
A key point for the operation of an expert finding system is how are politicians working in a parliament.
to learn and how to represent the knowledge/information of the The remainder of this paper is organized as follows: Section 2
system about the possible candidates. A way of representing the briefly discusses some related works. In Section 3 we introduce
expertise of a candidate is to use a user/expert profile [17]. The most monolithic profiles and how they can be used by the recommender
common and simple type of profile associated to a candidate expert system. Sections 4 and 5 describe topical and temporal subprofiles,
is composed of a set of (weighted) terms or keywords describing respectively, whereas in Section 6 combined subprofiles are dis-
their areas of interest and expertise [16], although there are also cussed. Section 7 is devoted to the experimental part of the work,
profiles based on semantic networks or concepts [25]. The key including experimental setting, results and discussion. Finally, Sec-
advantage of term-based profiles is that these terms can be easily tion 8 contains the concluding remarks.
"Copyright ©2020 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0)."
de Campos, et al.
2 RELATED WORK to each document (and also a probability distribution of terms to
There is an increasing interest about how to include temporal infor- each topic). LDA needs an input parameter, k, representing the
mation within recommender systems, although most of the works number of topics to be used. In order to use LDA as a clustering
are focused only on collaborative filtering. One of the basic ideas is method, once LDA has been applied, each document is assigned to
to weight the ratings using a decaying factor based on the time gap the cluster associated to its most probable topic [14], thus obtaining
[15]. The incorporation of time in a latent factor model is also a key a partition of the document collection into k clusters.
factor in the performance of timeSVD++ [20] (the winner of the In our case the document collection to be clustered is the one
Netflix prize). An interesting survey of time-aware recommender formed by all the documents associated to all the possible experts,
systems is provided in [9] (also centered in collaborative filtering). D = ∪ri=1 D ei . We do it in this way in order to find a single set of
In [23] a content-based filter for tweets is studied, where a specific topics common for all the experts. Each cluster, Dl , l = 1, . . . , k, is
time frame is learned for each user, thus recommending to her only formed by the documents of the experts which are associated to
tweets within this personalized frame. Temporal discounting (ex- the l-th topic, xl (those documents whose most probable topic is
ponential and hyperbolic) is used in [28], together with an expert xl ), that is to say:
finding approach for question routing, in the context of Community e e
Dl = {di j | l = arg max p(x s |di j ), j = 1, . . . , r , i = 1, . . . , ne j }.
Question Answering (CQA) systems. s=1, ...,k
There are many papers that consider compound profiles instead (1)
of monolithic ones, for example using long-term and short-term sub- As these clusters contain documents from different experts, a spe-
profiles [18], subprofiles based on positively and negatively judged cific local clustering for each expert e and each topic xl , Dle , is
documents [8] or hierarchical profiles (using a fixed taxonomy) obtained by grouping the documents within each global cluster
[24]. Other methods consider topical (sub)profiles generated using that are associated to the given expert, Dle = Dl ∩ D e . Then each
cluster methods: some works group terms or tags (not documents) expert e will have associated as many subprofiles as local clusters
for profiling in recommender systems [2, 3], whereas other group have been generated for her (at most k). These subprofiles are then
documents, either for search personalization purposes [11, 26] or documents, d e,l , which are built by concatenating the documents
for filtering and expert recommendation [14]. within each local cluster Dle , d e,l = ∪die ∈D e die . Figure 2 illustrates
l
We are not aware of any previous work simultaneously dealing all the process for generating the topical subprofiles.
with the temporal and topical dimensions of profiles, except [24], The recommender system is thus obtained by indexing this sub-
which is centered on expert profiling but not on (expert) recom- profile document collection D t sp and using again an IRS, where
mendation.
D t sp = {d e1,1 , . . . , d e1,k , d e2,1 , . . . , d e2,k , . . . , d er ,1 , . . . , d er ,k }.
3 MONOLITHIC PROFILES AND THE However, in this case the result returned by the IRS for a given
RECOMMENDER SYSTEM query is a ranked list of subprofiles (and now there is not a one-to-
Let E = {e 1 , . . . , er } be the set of candidate experts. Given a candi- one association of experts and subprofiles). As we need a ranking of
date expert e ∈ E, we have a set of ne documents D e = {d 1e , d 2e , . . . , dne e } experts, a fusion strategy to combine the scores of the subprofiles
which are associated to e. These documents in some sense repre- associated to the same expert is required, in order to rerank the
sent (possibly in an implicit way) the interests and expertise of e. combined scores and recommend the top-ranked experts. We will
For example, in an academic setting the documents could be the use the so-called CombLgDCS fusion method [13], which aggre-
scientific articles written by each author or, for lawyers, the court gates the scores of all the expert subprofiles but reducing them
cases they have worked on. In the political setting where we are proportionally to the logarithm of their positions in the ranking.
focusing in this paper, these documents could be the transcriptions
of the interventions of politicians in parliamentary sessions. 5 TEMPORAL SUBPROFILES
Given a candidate expert e, the monolithic profile for e is built The other option to divide the monolithic profiles into more ho-
by simply concatenating all the documents in D e into a single mogeneous subprofiles is to use the temporal dimension instead
macro document, d e = ∪ni=1 e
die . The process is illustrated in Figure of the topical dimension of the documents. Then we are going to
1. Then, we have a collection of monolithic documents/profiles divide the temporal line into h intervals and will group together
Dm = {d e1 , . . . , d er }. The recommender system will be obtained the documents associated to an expert which belong to the same
from this collection using an information retrieval system (IRS): temporal interval. More formally, let us consider the h temporal
the profiles collection will be indexed for use by the IRS. When a intervals Iu , determined by h + 1 time points t 0 < t 1 < . . . < th ,
query representing the expertise required by a user is submitted to Iu = [tu−1 , tu ), u = 1, . . . , h. We define in this case the h global
the IRS, this will generate a ranking of profiles, and the top-ranked temporal clusters Tu as follows:
experts will be returned to the user. e e
Tu = {di j | tu−1 ≤ date (di j ) < tu , j = 1, . . . , r , i = 1, . . . , ne j }.
(2)
4 TOPICAL SUBPROFILES
In order to get subprofiles topically homogeneous, we are going to Alternatively, we could repeatedly apply clustering only to the documents associated
use a clustering method based on LDA (Latent Dirichlet Allocation) to each expert e , D e , thus obtaining topics specific for each expert, but we are not
going to explore this option in this paper.
[5]. LDA is a non supervised method which finds latent topics in a Expert e may have less than k local clusters in case that some global clusters do not
document collection and assigns a probability distribution of topics contain any documents associated to e .
Temporal and Topical Profiles for Expert Finding
Monolithic
Document Collection profiles
XX
XX Expert 1 PP1
1
XX
YY YY
PP2
Expert 2 2
YY YY
SS SS
PPS
SS Expert s S
SS SS
Figure 1: Building the monolithic profiles.
Experts’
Document Collection Clustering subprofiles
Expert 1
XX XX
XX XX PP11 PP21
XX Expert 1 XX
11 21
XX XX
XX Expert 2
YY YY YY YY
YY YY PP12 PP22
12 22
Expert 2
YY YY YY
YY YY
YY
Expert s
SS SS SS SS
SS SS
PP1S PP2S
SS SS 1S
SS Expert s SS SS SS SS
2S
SS SS
Figure 2: From the document collection to the topical subprofiles using clustering.
where date (d ) is a function that returns the date of document d. documents within each global temporal cluster which are associated
As in the case of topical clusters, we extract the local temporal to e, Tue = Tu ∩ D e . Also, the (at most h) temporal subprofiles for
clusters for each expert e in the same way, i.e. by grouping the each expert e are built by concatenating the documents within each
de Campos, et al.
Tue , d e,u = ∪die ∈Tue die (i.e. each expert e would be represented by Concerning the implementation details of the recommender sys-
at most h monolithic profiles, one per temporal period), which are tems, the base IR system is built using the Lucene library and its
then indexed by the IRS. CombLgDCS will also be used to obtain a default implementation of the Language Model as retrieval model.
ranking of experts. Previous to indexing the (sub)profiles of each MP, stop words were
removed, stemming performed and any remaining terms appear-
6 TEMPORAL AND TOPICAL SUBPROFILES ing in fewer than 1% of the interventions were also deleted. For
We can try to combine both the temporal and the topical dimensions those experiments that require LDA to cluster documents, we used
in order to obtain subprofiles which are simultaneously topically the R implementation (topicmodels package), with hyperparame-
and temporally homogeneous. ters α and β fixed to 50/k and 0.1, respectively (these are the by-
A way of doing this is first to obtain temporal subprofiles and default values), where k is the number of topics. For the parameter
next further subdivide them topically, in order to get sub-subprofiles k, we have tested two classical alternatives in cluster analysis: (a)
thematically homogeneous. In this case we would have to cluster k = m ∗ n/t [10], where m is the number of terms in the collection
separately the documents within each temporal cluster. This would (m = 4, 208), n is the number of documents/interventions in the
imply to apply LDA to each of the h temporal subcollections of training set (n = 10, 025) and t is the number of nonzero entries
documents (in this way obtaining specific topics for each time in the document-term√ matrix (t = 1, 702, 296). The value of k is
period). Another option, which is the one that we are going to use then 24. (b) k = n/2 [19], which only considers the number of
in this paper, is first to carry out the topical division, obtaining interventions in the collection. In this case, k = 70. With respect to
topical subprofiles and later to subdivide them temporally. In this the temporal division, in this case we use h = 4 temporal intervals,
way we only have to apply LDA once to the complete document each one roughly corresponding to one year.
collection. While the interventions within the initiatives in the training
More precisely, let Dl as defined in eq.(1), l = 1, . . . , k, and Tu set are used to build the (sub)profiles of the MPs, the initiatives
as defined in eq.(2), u = 1, . . . , h. Then the global topical-temporal in the test set are used to obtain queries and relevance judgments.
clusters, DT lu are defined as More precisely, we use the title (which is a short description of the
initiative) and its subjects (which are terms from a controlled vo-
DT lu = Dl ∩ Tu , l = 1, . . . , k, u = 1, . . . , h. (3) cabulary assigned to each initiative by Parliament staff) to simulate
It should be noticed that the total number of topical-temporal clus- a query representing a real expert finding task. If we focus only on
ters generated can be lesser than the product k ∗ h, because some the test initiatives corresponding to committee sessions (i.e. exclud-
combinations of topics and temporal intervals may be empty (i.e. ing initiatives in the test set discussed in plenary sessions, where
there are no documents about a given topic at certain time inter- all the MPs participate, which are more general and political and
vals). This process is illustrated in Figure 3, where k = 6 topics less specific than those from committees), we have a very simple
and h = 4 temporal intervals give rise to only 13 topical-temporal way of fixing the ground truth to evaluate the different approaches:
clusters. any MP who is a member of the committee where the initiative
The local clusters for each expert e are obtained, as in the previ- generating the query has been discussed is relevant (is a potential
ous cases, by grouping together the documents within each topical- expert for this query). There are twenty-six different committees
temporal cluster that are associated to e, i.e. DTlue = DT
lu ∩ D .
e covering different areas (as for example Education, Health, Culture,
Also, the documents within each local cluster are concatenated and Environment,...), having on average 15.2 MPs per committee.
the corresponding macro documents are indexed by the IRS. We have used three classic IR metrics to measure the perfor-
mance of the different recommender systems: precision@10, NDCG@10
7 EXPERIMENTS (normalized discounted cummulative gain), both focusing on the
7.1 Experimental settings top 10 MPs retrieved and recall@nr , where nr is the total number
of relevant MPs for each query. To compute these measures, we
The experimental work to test our proposals has been carried out have only considered those MPs having at least 10 interventions in
in the domain of political expert finding [12, 13]. We have used the training set (a total of 132 persons).
the Records of Parliamentary Proceedings (in Spanish) from the
Andalusian Parliament in its 8th Term of Office (which covers 7.2 Results
four years of parliamentary activity, from march 2008 to march
2012). These records contain the transcriptions of the speeches of We have experimented with four types of profiles: monolithic
the Members of Parliament (MPs) in the initiatives discussed in profiles (MONP), topical subprofiles (TOPS), temporal subprofiles
committee and plenary sessions. There are a total of 5258 initiatives (TEMPS) and topical-temporal subprofiles (TOPTEMPS), as de-
in this term and 12633 interventions of MPs (which are the experts scribed in the previous sections 3, 4, 5 and 6, respectively. For both
to be recommended). We randomly partitioned the set of initiatives, TOPS and TOPTEMPS subprofiles we have used the two methods
using 80% for the training set (to build the subprofiles from the of selecting the number of topics previously described. For TEMPS
interventions contained in these training initiatives) and 20% for we fixed the number of temporal intervals to 4. The results are dis-
the test set (to obtain the queries), and this sampling process is played in Table 1. The percentages of improvement of each method
repeated five times, reporting then the average results of these five https://lucene.apache.org/
partitions. In this case we still have on average 612 test initiatives per partition.
We do not use recall@10 because usually the number of relevant experts for each
Available at http://irutai2.ugr.es/ColeccionPA/legislatura8.tgz query is greater than 10.
Temporal and Topical Profiles for Expert Finding
Figure 3: k=6 topical and h=4 temporal clusters generate 13 topical-temporal clusters.
with respect to the base monolithic profiles are displayed in Table It can be seen that using subprofiles of any type is better than
2. using the monolithic profiles. The differences are always statistically
significant, using a paired t-test with the results of the five training-
Method ndcg@10 prec@10 recall@nr test partitions of the document collection, with significance level
MONP 0.67622 0.65146 0.45806 of 0.01.
RAND4 0.69035 0.66465 0.47869 Although the results obtained by the temporal subprofiles are
TEMPS 0.70907 0.68139 0.49444 better than those of the monolithic profiles, as the percentages
TOPSsqr t (n/2) 0.73911 0.71351 0.52740 of improvement are rather small (although significant), we want
TOPSmn/t 0.73721 0.71313 0.53491 to test whether this improvement is really due to the temporal
TOPTEMPSsqr t (n/2) 0.75416 0.72456 0.53072 influence or merely to the fact that we are using several (four in
TOPTEMPSmn/t 0.76136 0.73283 0.54520 this case) smaller subprofiles instead of a single big profile. In order
Table 1: Results of the experiments (best results in bold). to do so we have also randomly divided the interventions of each
MP in the training set into four parts and generated a subprofile
for each part. We have repeated this process 10 times and averaged
the results obtained from these random subprofiles. The results are
also shown in Tables 1 and 2, under the name RAND4.
Method % ndcg@10 % prec@10 % recall@nr We can observe that the (averaged) results of the random parti-
RAND4 2.09 2.02 4.50 tions are also better than monolithic profiles (and they are statisti-
TEMPS 4.86 4.59 7.94 cally significant too), although they are worse than the temporal
TOPSsqr t (n/2) 9.30 9.52 15.14 subprofiles. The fact that random partitions are somewhat better
TOPSmn/t 9.02 9.47 16.78 than monolithic profiles is probably due to the smaller size of these
TOPTEMPSsqr t (n/2) 11.53 11.22 15.86 subprofiles. This suggest that a part of the improvement obtained by
TOPTEMPSmn/t 12.59 12.49 19.02 TEMPS over MONP is not due to temporal influence but to the use
Table 2: Percentages of improvement with respect to mono- of smaller subprofiles. Therefore, we can conclude that the contribu-
lithic profiles. tion of temporal subprofiles alone to improve the recommendation
results is positive but rather scarce.
On the other hand, the gains obtained by the topical subprofiles
are more important (around 9%), and the differences with temporal
First, we can observe that the behavior of the different systems
with respect to the three metrics is essentially the same (the rank- In these tables only the averages of the ten random partitions appear. The individual
ings of the systems for the three metrics are almost identical). For values are all quite similar, having very low standard deviations, on the order of 0.001.
The differences between random and temporal subprofiles are small but also statisti-
that reason we are going to focus on the ndcg@10 metric which, cally significant.
from the perspective of expert finding is probably the most relevant. The IRS may tend to favor smaller documents.
de Campos, et al.
subprofiles are also statistically significant. However, there is not [12] L.M. de Campos, J.M. Fernández-Luna, J.F. Huete, Profile-based recommendation:
significant differences between the two versions of TOPS using A case study in a parliamentary context, Journal of Information Science 43(5):665-
682, 2017.
different number of topics. This is interesting because it suggests a [13] L.M. de Campos, J.M. Fernández-Luna, J.F. Huete, Committee-based profiles for
robust behavior of TOPS. politician finding, International Journal of Uncertainty, Fuzziness and Knowledge-
Based Systems 25(Suppl. 2):21-36, 2017.
The best results are obtained when combining the two types of [14] L.M. de Campos, J.M. Fernández-Luna, J.F. Huete, L. Redondo-Expósito, Automatic
subprofiles (with gains around 12% with respect to MONP). The construction of multi-faceted user profiles using text clustering and its application
results are significantly better than those obtained by topical sub- to expert recommendation and filtering problems, Knowledge-Based Systems
190, article number 105337, 2020.
profiles alone. Again, there are not significant differences between [15] Y. Ding, X. Li, Time weight collaborative filtering, In Proceedings of the 14th
the two versions of topical-temporal subprofiles. ACM International Conference on Information and Knowledge Management, pp.
485-492, 2005.
[16] C.I. Eke, A.A. Norman, L. Shuib, H.F. Nweke, A survey of user profiling: state-of-
8 CONCLUDING REMARKS the-art, challenges, and solutions, IEEE Access 7:144907-144924, 2019.
In this paper we have studied the effects of using subprofiles tem- [17] S. Gauch, M. Speretta, A. Chandramouli, A. Micarelli, User profiles for person-
alized information access. In: The Adaptive Web, Lecture Notes in Computer
porally and/or topically homogeneous, in comparison with single Science 4321:54-89 2007.
heterogeneous profiles, for the problem of expert recommendation. [18] J.A. Gulla, A.D. Fidjestøl, X. Su, H. Castejon, Implicit user profiling in news
recommender systems, In Proceedings of the 10th International Conference on
The experiments have been carried out in a context of political Web Information Systems and Technologies - Volume 1: WEBIST, pp. 185-192,
expert finding, using a document collection of parliamentary docu- 2014.
ments. [19] L. Kaufman, P.J. Rousseeuw, Finding groups in data: An introduction to cluster
analysis, John Wiley, 1990.
The obtained results show that the use of homogeneous subpro- [20] Y. Koren, Collaborative filtering with temporal dynamics, Communications of
files instead of monolithic profiles is always positive to improve the the ACM 53:89-97, 2010.
recommendation quality, although the improvement degree varies [21] S. Lin, W. Hong, D. Wang, T. Li, A survey on expert finding techniques, Journal
of Intelligent Information Systems 49:255-279, 2017.
depending on the different methods: temporal subprofiles are only [22] N. Nikzad-Khasmakhi, M.A. Balafar, M.R. Feizi-Derakhshi, The state-of-the-art
a bit better than random subprofiles (of the same size), whereas in expert recommendation systems, Engineering Applications of Artificial Intelli-
gence 82:126-147, 2019.
topical subprofiles get much better results. So it seems that the [23] C. Ramos Casimiro, I. Paraboni, Temporal aspects of content recommendation
topical dimension is more important than the temporal dimension. on a microblog corpus. In: Baptista J., Mamede N., Candeias S., Paraboni I., Pardo
Nevertheless, there is a sinergy between both dimensions, as their T.A.S., Volpe Nunes M..G. (eds) Computational Processing of the Portuguese
Language. PROPOR 2014. Lecture Notes in Computer Science 8775:189-194, 2014.
combination produces the best results. [24] J. Rybak, K. Balog, K. Nørvåg, Temporal expertise profiling, In de Rijke, M., Kenter,
For future work we are planning to explore other ways of com- T., de Vries, A. P., Zhai, C., de Jong, F., Radinsky, K., and Hofmann, K., editors,
bining temporal and topical dimensions to get better subprofiles, Proceedings of the 36th European Conference on IR Research, pp. 540-546, 2014.
[25] A. Sieg, B. Mobasher, R. Burke, Web search personalization with ontological user
as for example using temporal topic models [6, 27]. profiles. Proceedings of the 16th ACM International Conference on Information
and Knowledge Management, pp. 525-534, 2007.
[26] G.L. Somlo, A.E. Howe, Incremental clustering for profile maintenance in informa-
ACKNOWLEDGMENTS tion gathering web agents, In Proceedings of the Fifth International Conference
This work has been funded by the Spanish Ministerio de Economía on Autonomous Agents, pp. 262-269, 2001.
[27] X. Wang, A. McCallum, Topics over time: a non-Markov continuous-time model
y Competitividad under project TIN2016-77902-C3-2-P, and the of topical trends, Proceedings of the 12th ACM SIGKDD international conference
European Regional Development Fund (ERDF-FEDER). on Knowledge discovery and data mining, pp. 424-433, 2006.
[28] R. Yeniterzi, J. Callan, Moving from static to dynamic modeling of expertise for
question routing in CQA sites, Proceedings of the Ninth International AAAI
REFERENCES Conference on Web and Social Media, pp. 702-705, 2015.
[1] M.Z. Al-Taie, S. Kadry, A.I. Obasa, Understanding expert finding systems: domains [29] S. Yuan, Y. Zhang, J. Tang, W. Hall, J.B. Cabota, Expert finding in com-
and techniques, Social Network Analysis and Mining 8:57, 2018. munity question answering: a review, Artificial Intelligence Review, 2019
[2] B. Amini, R. Ibrahim, M.S. Othman, A. Selamat, Capturing scholar’s knowledge https://doi.org/10.1007/s10462-018-09680-6.
from heterogeneous resources for profiling in recommender systems, Expert
Systems with Applications 41:7945-7957, 2014.
[3] C. Au Yeung, N. Gibbins, N. Shadbolt, Multiple interests of users in collaborative
tagging systems, In King, I. and Baeza-Yates, R., editors, Weaving Services and
People on the World Wide Web, pp. 255-274, Springer, 2009.
[4] K. Balog, Y. Fang, M. de Rijke, P. Serdyukov, L. Si, Expertise retrieval, Foundations
and Trends in Information Retrieval 6:127-256, 2012.
[5] D.M. Blei, A.Y. Ng, M.I. Jordan, Latent Dirichlet allocation, Journal of Machine
Learning Research 3:993-1022, 2003.
[6] D.M. Blei, J.D. Lafferty, Dynamic topic models, Proceedings of the 23rd interna-
tional conference on Machine learning, pp. 113-120, 2006.
[7] J. Bobadilla, F. Ortega, A. Hernando, A. Gutiérrez, Recommender systems survey,
Knowledge-Based Systems 46:109-132, 2013.
[8] C. Bouras, V. Tsogkas, Improving news articles recommendations via user clus-
tering, International Journal of Machine Learning and Cybernetics 8:223-237,
2017
[9] P.G. Campos, F. Díez, I. Cantador, Time-aware recommender systems: A compre-
hensive survey and analysis of existing evaluation protocols, User Modeling and
User-Adapted Interaction 24:67-119, 2014.
[10] F. Can, E. Ozkarahan, Concepts and effectiveness of the cover-coefficient-based
clustering methodology for text databases. ACM Trans. Database Syst. 15(4):483-
517, 1990.
[11] L. Chen, K. Sycara, Webmate: A personal agent for browsing and searching, In
Proceedings of the Second International Conference on Autonomous Agents, pp.
132-139, 1998.