We study the publication venue recommendation problem, where a recommender system must help a researcher to decide where to submit a given target article for possible publication. We focus on content-based recommendation approaches, where we explicitly look for a good match between the topics discussed in the article and the recommended venue. But, in addition to this topical dimension, we also want to include a temporal dimension, in such a way that we prefer those venues where the articles which are more related with the target article have been published more recently. We use an information retrieval system to obtain separate topical and temporal recommendations and then combine them by means of diferent fusion strategies. The experimental results obtained on a collection of biomedical journal articles confirm the efectiveness of our proposals.
Every researcher in any discipline has to face the problem of deciding where to publish the scientific article he/she has written. This decision, among other factors, should be based on the suitability of the topics that the article deals with respect to those usually treated by the tentative publication venues (either journals or conferences). This is known as the publication venue recommendation problem [28, 45].
Although there are also approaches to this problem based on collaborative filtering (recommending venues where either similar researchers or coauthors have published in the past [25, 31]), in this paper we are going to adopt a content-based approach, where the decision about which venue to recommend is mainly based on the textual content of the paper to be published [20, 42].
The information that the recommender system possesses about each possible publication
venue can be represented by means of a profile [19]. Although there are diferent types of
profiles, for example based on semantic networks or concepts [ 39], the most common and
simple type that can be associated to an item in general, or to a venue in our particular scenario,
is based on a set of terms or keywords (perhaps weighted) [
In [
However, we did not consider other information source that could also help to improve the recommendation results, namely the temporal information. The motivation for using this source for publication venue recommendation is that we do not only want that the thematic content of our new (target) article matches with the recommended venues, we would also prefer that the themes this article deals with have been recently considered in these venues: If venues A and B both have published articles related with the main topic of the target article but the articles in A are more recent than the articles in B, then probably we would prefer to select venue A instead of B.
In this work we aim to build a publication venue recommendation system considering both the temporal and the topical dimension of the available data (the articles published in the venues). Therefore, on the one hand, we want to study how the temporal information alone can be used in the publication venue recommendation problem and the benefits it may ofer; on the other hand, we also want to study diferent ways of combining the temporal information with the topical information, hoping that these two complementary sources reinforce each other.
The remainder of this paper is organized as follows: Section 2 briefly discusses some related
works. Section 3 describes how we build the topical subprofiles by means of Latent Dirichlet
Allocation (LDA) [
In the literature about content-based publication venue recommendation there are diferent
ways of representing the information within the profiles (terms, n-grams, noun-phrases, topics)
but in all the cases each venue has either a single profile [28, 40, 45] or as many subprofiles as
published articles [
Moreover, there are not much works explicitly using temporal information: [
The situation is diferent in the context of general recommender systems, where there are
many works which consider organizations of the information diferent from single profiles.
These multi-faceted profiles can be trees [ 30], graphs of clusters [47] or hierarchies [21, 36]. Other
methods generate topical subprofiles using clustering (grouping either terms or documents)
[
Within general recommender systems there are also many works considering the inclusion of
temporal information, although most of them are focused on collaborative filtering techniques
[
Two basic organizational schemes to represent information about the diferent publication venues are that we call monolithic profiles and atomic subprofiles, which represent the two extremes in terms of heterogeneity or homogeneity of the information. In monolithic profiles all the articles published in a given venue are grouped together in a single (heterogeneous) macro-document. On the contrary, in atomic subprofiles each individual article published in a venue forms a (very homogeneous) subprofile for this venue, having as many subprofiles as articles it contains. An intermediate way of organizing the information would be to build subprofiles around the diferent thematic areas covered by each venue.
In [
We propose to apply a global LDA to the collection of all the articles of all the venues1. It is known that LDA tries to find latent topics in a document collection. Each of these latent topics is characterized by a diferent probability distribution of terms (given the topic). Moreover, in this non-supervised method, for each document a probability distribution of topics is obtained, which describes to which extent each document is about each topic. LDA requires the number of topics to be used, , as an input parameter.
Once LDA has been applied to our articles collection and we have computed the probability distributions of topics, we associate each article with its most probable topic. Therefore, we have a partition of the articles in groups, where all the articles in each group deal mainly about the same global topic.
The next step is to obtain the homogeneous subprofiles for each publication venue. We simply divide each group of articles associated with the same topic into local groups of articles related to this topic and each venue. All the articles in each local group associated with a venue are then concatenated to form a single document/subprofile. In this way each venue will have at most subprofiles, probably much less than , because it is quite possible that the vast majority of the venues do not deal with all the topics but are especialized in a more reduced number of specific topics (and therefore there will be topics which are not the most probable topic for any article in a given venue).
Finally, the document collection formed by the subprofiles of all the venues is indexed by an IRS that will be used to match the content of the target article (the query) with the subprofiles, providing a score, (), for each subprofile and hence a ranking of subprofiles.
We assume that a researcher normally will prefer to publish in a venue where the topics related to their article are dealt with recently, i.e. they are topics of current interest of the venue. In other words, a venue that has recently published articles which are closer to the target article is preferable. Although reasonable and intuitive, this is only a working hypothesis, which our experiments in Section 7 will confirm 2.
A simple but efective way of incorporating this criterion in the recommendation is through the use of a temporal decay function (∆ ) [22]. This function considers the diference, ∆ , between the current year3 and the publication year of an already published article, in such a way that the greater ∆ is the less important becomes this article (from a temporal perspective).
Then, the recommender system based on temporal information will work in the following way: we index the collection of articles (the atomic collection) using an IRS. Given the target article, the IRS obtains a score () for each article in the collection, which represents the degree of similarity between the target and . Then we compute a modified score () by using also the decay function, () = () × (∆ ), which reduces the similarity of the article as the temporal distance between the target and increases. Finally, we obtain a ranking of articles according to this modified score.
In the previous models, the ranking obtained by the IRS, given the target article as the query, is
either a ranking of individual articles (for the temporal and the atomic models) or a ranking of
2However, in these experiments, we only have access to the venues where the articles were in fact published but
we do not know which venues the authors considered in the first place.
3In our context this is the year of publication of the test article.
subprofiles (for the topical model) 4. However, a ranking of publication venues is required, as
we are going to recommend venues to publish the target article. To combine the scores of the
diferent documents associated with each venue (i.e. for those articles ∈ published in
and for those subprofiles ∈ associated to ) and generate a final venue ranking, we use the
CombLgDCS method [
, () () = ∑︁ ∈
() log2(() + 1) , (1) (2) where () and () denote the original score values obtained by the IR systems based on subprofiles and articles, respectively, and () (respect. ()) is defined as 1 in the case that (respect. ) being the first occurrence of a subprofile (respect. an article) of in the original ranking and, otherwise, it is the raw value of the position of the subprofile (respect. the article) in this ranking.
Once we have developed publication venue recommenders based on both topical and temporal information, the next logical step is to combine them to try to improve performance. As the previous methods are based on IR systems and each generates a ranking of venues given the target article, it seems reasonable to use some fusion/aggregation methods which are usually employed to combine the rankings of diferent IR systems [ 43]. There are two basic types of fusion methods: score-based and rank-based methods [34].
Score-based methods combine the scores generated by each IRS for a document, thus obtaining a combined score that is used to rerank the documents. However, as we are combining the scores of diferent IRSs with diferent characteristics (in our case scores obtained from diferent types of documents, subprofiles in one case and articles in other case), previous to the combination a score normalization is necessary to make the scores comparable to each other. We normalize the scores of each IRS by dividing by the corresponding maximum score. Rank-based methods do not use the scores (in case these scores exist) but only the information provided by the rankings.
Among the score-based methods, we are going to use the so-called CombSUM and CombMNZ, proposed originally in [18], and LC, a linear combination of scores.
CombSUM simply computes the sum of the scores of the diferent IRSs, in our case the combined score of venue is
() = () + (),
where and are the normalized scores generated by the topical and the temporal recommenders, respectively (and assuming that if a venue does not appear in some ranking, its corresponding score is equal to 0).
CombMNZ is similar to CombSUM but tries to promote those venues appearing more frequently in the rankings, by multiplying the sum of scores by the number of rankings where the document appears. In our case the score of venue is
⎧ 2 * (() + ()) if appears in the two rankings () = ⎨ () if appears only in the topical ranking ⎩ () if appears only in the temp ranking
() = * () + (1 − ) * (), where is a parameter controlling the relative importance given to the topical recommender.
Among the rank-based fusion methods, we shall use Borda Count [
Borda methods transform the rankings into scores which are later combined using diferent functions. Each document is associated with its position in the ranking: the first document gets score 1, the second document obtains score 2, and so on until the last document in the ranking, which gets score (if there are documents in the ranking). If a document does not appear in the ranking (but we are going to consider it because it appears in another ranking), its score is + 1. Then the scores obtained from each ranking are combined with some functions and a new ranking is generated (in this case sorting from least to greatest). In our case, having only two rankings and hence two scores, and , the diferent functions used give rise to the following expressions:
BordaSUM: () = () + () BordaPROD: () = () * () BordaL2: () = ()2 + ()2
Another common option is to associate with each document some transformation of the ranking instead of the ranking itself. For example, we can link with a document in position the score 1/, thus obtaining (using the same functions to combine the scores as before) the Reciprocal Borda methods, BordaRSUM, BordaRPROD and BordaRL2.
The Condorcet method is based on pairwise comparisons between the documents in the ranking: given a ranking, if document is previous to (is prefered to) document in the ranking then we add 1 to the element in a matrix. This process is repeated for all the rankings, thus obtaining at the end in the number of times that was prefered to . If this number is strictly greater than half the number of existing rankings (one in our case), we count a victory of on . In this way we are accumulating the number of victories of over the other documents. This number is the score associated to and based on these scores a new ranking is generated5. 5Ties are resolved in favor of the document having greater ∑︀ .
In this section we are going to experimentally tests our proposals for publication venue recommendation. We explain first the experimental setting and next the obtained results.
We used in the experimentation a set of 309,551 biomedical journal papers extracted from the
PMSC-UGR test collection [
As mentioned previously, firstly we used LDA to find the latent topics in the training set and secondly to build the subprofiles associated to each journal. More precisely, we employed the implementation of LDA in the Gensim Python library, with default parameters. Before applying LDA, in order to reduce dimensionality, we performed stopwords removal and stemming; also, the terms appearing in more than 90% of the articles and those appearing in fewer than 750 articles were removed6. Concerning the LDA parameter fixing the number of topics, , we proposed a value equal to the number of descriptors or categories in the second level of the MeSH thesaurus, which is 110.
Concerning the decay function used by the temporal model, we have considered two options: 1 1 a linear decay, (∆ ) = 1+Δ , and a power decay, (∆ ) = (1+Δ)1/4 . In the first case, the linear decay imposes a penalization more severe to older articles, whereas the power decay penalizes them more smoothly.
All the recommendation models being considered are supported by an IRS. As the original individual articles and also the diferent subprofiles are text documents (in the case of subprofiles these documents are formed by the concatenation of the articles within each subprofile), they form a training document collection which can be indexed and searched for. We have used the Lucene library7 for these purposes, removing stopwords and performing stemming before indexing and using the Language Model (with Jelinek-Mercer smoothing) as retrieval model to compute the ranking of documents.
Each article from the test set is considered as the article for which we are seeking a recommendation (i.e. the target article), and its textual content is used to form a query to the IRS.
In order to evaluate the quality of the recommendations ofered by the diferent models, we adopt a conservative but objective approach: only one journal is relevant for each query (test article) and this is the journal where the paper has actually been published. As evaluation measures we use accuracy@X (with X=1,5) and mean reciprocal rank, MRR. Accuracy@X computes the ratio between the number of recommendations where the true journal at which a test article was published is among the first recommended journals and the number of all the 6All these reductions of terms are only used to apply LDA and determine the topics; the subprofiles obtained from these topics will contain all the terms appearing in the original documents associated with the subprofile. 7https://lucene.apache.org/ recommendations. MRR tries to reflect how high in the ranking the only relevant journal is recommended: it is computed as the average of the inverse of the positions in the ranking at which the actual journal where each test paper was published is found (0 if the actual venue does not appear in the top 40 positions in the ranking).
We have experimented with all the fusion methods explained in Section 6, as well as the original Topical and Temporal models and also the baselines Monolithic and Atomic models. The results of our experiments with the diferent models are displayed in Table 1 and in Figures 1, 2 and 3. The decay function used for the temporal model (and hence for all the fusion models) in these results is always the power decay. We do not show the results obtained when using the linear decay because they are quite poor, even worse than the baselines. This implies that a smooth penalization of older articles is positive but it becomes self-defeating when the penalization is abrupt.
Although the diferences between the diferent models are not quantitatively very big, the tendencies are clear and the three metrics being considered essentially move in the same direction (the rankings of the systems for all of them are very similar). All the fusion strategies improve the results of the two individual components, Topical and Temporal, which in turn both improve the results of the two baselines, Atomic and Monolithic, which do use neither topical nor temporal information. This confirms two facts: first, the use of topics to build more homogeneous subprofiles, as well as the use of temporal information (in the form of a decay function) is positive to get better venue recommendations; second, the topical and temporal information are complementary and their combination through a fusion method further improves the results. Moreover, the individual results for topical and temporal models are quite similar, although topics performs slightly better than temporal.
It can also be observed that the fusion methods based on scores (CombSUM, CombMNZ and
LinearComb) perform better than those ones based only on the ranking information (Borda and
Condorcet). This confirms findings obtained in other contexts [
There is almost no diference between the diferent methods based on scores, although the linear combination performs somewhat worse if the weights are not uniform (considering the three weight combinations being used, taking into account that CombSUM is equivalent to LinearComb with both weights equal to 0.5). In other words, we should attribute the same importance to the topical and the temporal information (although perhaps it would be preferable to give somewhat more importance to topics than to time).
Within the methods based on ranking, the diferences are also quite small, although the best method is always a variant of Borda (depending on the metric, either direct Borda or Reciprocal Borda is better) and Condorcet produces the worst results.
To assess the robustness of our main findings, we have used a statististical significance test,
namely the McNemar test [
The results of these pairwise tests (the p-values) are displayed in Table 2.
These results confirm that CombSUM is the best method, showing statistically significant diferences with all the other methods. BordaPROD is the second best, although its diferences with Topical and Temporal are not significant (but they are with all the other methods). Topical is somewhat better than Temporal (but without significant diferences), which in turn is better than Monolithic and Atomic (having significant diferences). Finally, Atomic is significantly better than Monolithic.
We have tackled the publication venue recommendation problem by building two contentbased recommender systems using diferent dimensions: One system is based on a process that analyzes the articles published in the diferent venues through latent Dirichlet allocation and generates homogeneous subprofiles for each venue associated with the diferent discovered topics, which are then indexed by an IRS. The other system uses the temporal information of each published article, by means of a decay function, to modify the output of another IRS. The lists of recommended venues for publishing a given target article generated by each method are then combined by using several score-based and ranking-based fusion strategies.
We have carried out experiments with a collection of biomedical journal articles. The results obtained indicate that both the topical and the temporal recommenders improve the performance of the baseline models which do not use these types of information. Moreover, all the fusion strategies further improve the performance, thus showing that the two dimensions, temporal and topical, are complementary and their combined use is always beneficial.
We have considered the topical and temporal recommenders essentially in a separate way and
then we have combined them by means of fusion strategies. However, for future work it would be
interesting to consider other ways of merging these two dimensions, for example through the use
of time-based topic models [
This work was supported by the Spanish “Agencia Estatal de Investigación” [grant number PID2019-106758GB-C31]; the Spanish “Programa operativo FEDER Andalucía 2014-2020 de la Junta de Andalucía y la Universidad de Granada” [grant number A-TIC-146-UGR20]; and the European Regional Development Fund (ERDF-FEDER). reviewers, appropriate journals and similar publications, Nucleic Acids Research 35 W12W15, 2007. [18] E.A. Fox and J.A. Shaw. Combination of multiple searches. In Proccedings of the Second
Text REtrieval Conference (TREC-2), pp. 243-252, 1994. [19] S. Gauch, M. Speretta, A. Chandramouli, A. Micarelli, User profiles for personalized information access. In: The Adaptive Web, Lecture Notes in Computer Science 4321:54-89 2007. [20] N. Kang, M. Doornenbal, B. Schijvenaars, Elsevier journal finder: recommending journals for your paper, RecSys’15, 261-264, 2015 [21] H.J. Kook, Profiling multiple domains of user interests and using them for personalized web support, Proceedings of the International Conference on Intelligent Computing, 512-520, 2005. [22] S. Larrain, C. Trattner, D. Parra, E. Graells-Garrido, K. Nørvåg, Good times bad times: A study on recency efects in collaborative filtering for social ragging, Proceedings of the 9th ACM Conference on Recommender Systems, 269-272, 2015. [23] L. Li, L. Zheng, T. Li, LOGO: a long-short user interest integration in personalized news recommendation. Proceedings of the 5th ACM conference on Recommender systems, 317–320, 2011. [24] S. Lin, Rank aggregation methods, Wiley Interdisciplinary Reviews: Computational Statistics 2(5) 555-570, 2010. [25] H. Luong, T. Huynh, S. Gauch, L. Do, K. Hoang, Publication venue recommendation using author networks publication history, in Intelligent Information and Database Systems, LNCS 7198, pp. 426-435, 2012. [26] J.P. McGowan, N. Kushmerick, B. Smyth, Who do you want to be today? Web personae for personalised information access, Web-Based Systems: Second International Conference, 514–517, 2002. [27] Q. McNemar, Note on the sampling error of the diference between correlated proportions or percentages, Psychometrika 12:153–157, 1947. [28] E. Medvet, A. Bartoli, G. Piccinin, Publication venue recommendation based on paper abstract, Proceedings of the 26th IEEE International Conference on Tools with Artificial Intelligence, 1004-1010, 2014. [29] M. Montague, J.A. Aslam, Condorcet fusion for improved retrieval, In Proceedings of the
ACM CIKM Conference, pp. 538–548, 2002. [30] M. Pavan, E.W.D. Luca, Semantic-based expert search in textbook research archives, Proceedings of the 5th International Workshop on Semantic Digital Archives, CEUR Workshop Proceedings, 1529:18-29, 2015. [31] M.C. Pham, Y. Cao, R. Klamma, M. Jarke, A clustering approach for collaborative filtering recommendation using social network analysis, Journal of Universal Computer Science 17:583-604, 2011. [32] T. Pradhan, S. Pal, CNAVER: A Content and Network-based Academic VEnue Recommender system, Knowledge-Based Systems 189, 105092, 2020. [33] C. Ramos Casimiro, I. Paraboni, Temporal aspects of content recommendation on a microblog corpus. In: Baptista J., Mamede N., Candeias S., Paraboni I., Pardo T.A.S., Volpe Nunes M..G. (eds) Computational Processing of the Portuguese Language. PROPOR 2014.
Lecture Notes in Computer Science 8775:189-194, 2014. [34] M.E. Renda, U. Straccia, Web metasearch: rank vs. score based rank aggregation methods,
In Proceedings of the 2003 ACM symposium on Applied computing, pp. 841–846, 2003. [35] J. Rollins, M. McCusker, J. Carlson, J. Stroll, Manuscript Matcher: a content and bibliometric-based scholarly journal recommendation system, 5th International Workshop on Bibliometric-enhanced Information Retrieval (BIR 2017), CEUR Workshop Proceedings vol. 1823, 2017. [36] J. Rybak, K. Balog, K. Nørvåg, Temporal expertise profiling, In de Rijke, M., Kenter, T., de Vries, A. P., Zhai, C., de Jong, F., Radinsky, K., and Hofmann, K., editors, Proceedings of the 36th European Conference on IR Research, pp. 540-546, 2014. [37] M.J. Schuemie, J.A. Kors, Jane: suggesting journals, finding experts, Bioinformatics 24:727728, 2008. [38] Y. Si, F. Zhang, W. Liu, CTF-ARA: An adaptive method for POI recommendation based on check-in and temporal features, Knowledge-Based Systems 128:59-70, 2017. [39] A. Sieg, B. Mobasher, R. Burke, Web search personalization with ontological user profiles.
Proceedings of the 16th ACM International Conference on Information and Knowledge Management, pp. 525-534, 2007. [40] T. Silva, J. Ma, C. Yang, H. Liang, A profile-boosted research analytics framework to recommend journals for manuscripts, Journal of the Association for Information Science and Technology 66:180-200, 2015. [41] I. Vayansky, S.A.P. Kumar, A review of topic modeling methods, Information Systems 94, 101582, 2020. [42] D. Wang, Y. Liang, D. Xu, X. Feng, R. Guan, A content-based recommender system for computer science publications. Knowledge-Based Systems 157:1-9, 2018. [43] S. Wu, Data Fusion in Information Retrieval, Adaptation, Learning, and Optimization, vol.
13, Springer, 2012. [44] L. Xiang, Q. Yuan, S. Zhao, L. Chen, X. Zhang, Q. Yang, J. Sun, Temporal recommendation on graphs via long- and short-term preference fusion, Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, 723-732, 2010. [45] Z. Yang, B.D. Davison, Venue recommendation: submitting your paper with style, Proceedings of the 11th International Conference on Machine Learning and Applications, vol. 1, pp. 681-686, 2012. [46] R. Yeniterzi, J. Callan, Moving from static to dynamic modelling of expertise for question routing in CQA sites, Proceedings of the 9th International AAAI Conference on Web and Social Media, 702-705, 2015. [47] H.-J. Zeng, Z. Chen, W.-Y. Ma, A unified framework for clustering heterogeneous web objects, Proceedings of the 3rd International Conference on Web Information Systems Engineering, 161-172, 2002.