Editorial for the 8th Bibliometric-enhanced Information Retrieval Workshop at ECIR 2019

Abstract

The Bibliometric-enhanced Information Retrieval workshop series (BIR) at ECIR tackles issues related to academic search, at the crossroads between Information Retrieval and Bibliometrics. BIR is a hot topic investigated by both academia (e.g., ArnetMiner, CiteSeer , DocEar) and the industry (e.g., Google Scholar, Microsoft Academic Search, Semantic Scholar). This editorial presents the 8th iteration of the oneday BIR workshop held at ECIR 2019 in Cologne, Germany.

Keywords

• Academic Search
• Information Retrieval
• Digital Libraries
• Bibliometrics
• Scientometrics

-

Bibliometric-enhanced IR systems must deal with the multifaceted nature of scienti c information by searching for or recommending academic papers, patents [ 11 ], venues (i.e., conferences or journals), authors, experts (e.g., peer reviewers), references (to be cited to support an argument), and datasets. The underlying models harness relevance signals from keywords provided by authors, topics extracted from the full-texts, coauthorship networks, citation networks, and various classi cations schemes of science.

Bibliometric-enhanced IR is a hot topic whose recent developments made the news|see for instance the Initiative for Open Citations [ 33 ] and the Google Dataset Search [ 7 ] launched on September 4, 2018. We believe that BIR@ECIR is a much needed scienti c event for the retrievalists and citationists to meet and join forces pushing the knowledge boundaries of IR applied to literature search and recommendation. 2

Past Related Activities

The BIR workshop series was launched at ECIR in 2014 [ 26 ] and it was held at ECIR each year since then [ 25,20,21,22 ]. As our workshop has been lying at the crossroads between IR and NLP, we also ran it as a joint workshop called BIRNDL (for Bibliometric-enhanced IR and NLP for Digital Libraries) at the JCDL [ 5 ] and SIGIR [ 18,19 ] conferences. All workshops had a large number of participants, demonstrating the relevance of the workshop's topics. The BIR and BIRNDL workshop series gave the community the opportunity to discuss latest developments and shared tasks such as the CL-SciSumm [ 13 ], which was introduced at the BIRNDL joint workshop.

The authors of the most promising workshop papers were o ered the opportunity to submit an extended version for a Special Issue for the Scientometrics journal [ 27,6 ] and of the International Journal on Digital Libraries [ 24 ].

The target audience of our workshop are researchers and practitioners, junior and senior, from Scientometrics as well as Information Retrieval. These could be IR researchers interested in potential new application areas for their work as well as researchers and practitioners working with, for instance, bibliometric data and interested in how IR methods can make use of such data. 3

Objectives and Topics for BIR@ECIR 2019

We called for original research at the crossroads of IR and Bibliometrics. Thirteen peer-reviewed papers were accepted: 9 long papers, 3 short papers and 1 demo paper. These report on new approaches using bibliometric clues to enhance the search or recommendation of scienti c information or signi cant improvements of existing techniques. Thorough quantitative studies of the various corpora to be indexed (papers, patents, networks or else) were also contributed. The papers are as follows: Long papers: { An interactive visual tool for scienti c literature search: Proposal and algorithmic speci cation [ 2 ] { A searchable space with routes for querying scienti c information [ 8 ] { Discovering seminal works with marker papers [ 12 ] { How do computer scientists use Google Scholar?: A survey of user interest in elements on SERPs and author pro le pages [ 14 ] { Feature selection and graph representation for an analysis of science elds evolution: An application to the digital library ISTEX [ 15 ] { Optimal citation con-text window sizes for biomedical retrieval [ 17 ] { Bibliometric-enhanced arXiv: A data set for paper-based and citation-based tasks [ 30 ] { Mining intellectual in uence associations [ 32 ] { Citation metrics for legal information retrieval systems [ 40 ] Short papers: Demo: { Finding temporal trends of scienti c concepts [ 9 ] { A preliminary study to compare deep learning with rule-based approaches for citation classi cation [ 28 ] { Improving scienti c article visibility by neural title simpli cation [ 34 ] { Recommending multimedia educational resources on the MOVING platform [ 37 ]

The topics of the workshop are in line with those of the past BIR and BIRNDL workshops (Fig. 1): a mixture of IR and Bibliometric concepts and techniques. More speci cally, the call for papers featured current research issues regarding three aspects of the search/recommendation process: 1. User needs and behaviour regarding scienti c information, such as: { Finding relevant papers/authors for a literature review; { Measuring the degree of plagiarism in a paper; { Identifying expert reviewers for a given submission; { Flagging predatory conferences and journals. 2. The characteristics of scienti c information: { Measuring the reliability of bibliographic libraries; { Spotting research trends and research fronts. 3. Academic search/recommendation systems: { Modelling the multifaceted nature of scienti c information; { Building test collections for reproducible BIR.

Peer Review Process and Organization

The 8th BIR edition ran as a one-day workshop, as it was the case for the previous editions. Dr. Iana Atanassova delivered a keynote entitled "Beyond Metadata: the New Challenges in Mining Scienti c Papers" [ 1 ] to kick o the day.

Two types of papers were presented: long papers (15-minute talks) and short papers (5-minute talks). As the interactive session introduced last year was generally acclaimed, we decided to organize a interactive session to close the workshop. Two weeks earlier, we invited all registered attendees to demonstrate their prototypes or pitch a poster during ash presentations (5 minutes). This was an opportunity for our speakers to further discuss their work and for the public to showcase their work too.

We ran the workshop with peer review supported by EasyChair5. Each submission was assigned to 2 to 3 reviewers, preferably at least one expert in IR and one expert in Bibliometrics. The stronger submissions were accepted as long papers while weaker ones were accepted as short papers, and demo. All authors were instructed to revise their submission according to the reviewers' reports. All accepted papers were included in the workshop proceedings hosted at ceur-ws.org, an established open access repository with no author-processing charges.

5 https://easychair.org

As a follow-up of the workshop, the co-chairs will write a report summing up the main themes and discussions to SIGIR Forum [23, for instance] and BCS Informer6, as a way to advertise our research topics as widely as possible among the IR community. All authors are encouraged to submit an extended version of their papers to the Special Issue of the Scientometrics journal launched in Spring 2019.

6 https://irsg.bcs.org/informer/

References

1. Atanassova , I. : Beyond metadata: the new challenges in mining scienti c papers . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 8 { 13 . CEUR-WS.org ( 2019 )

2. Bascur , J.P. , van Eck, N.J. , Waltman , L. : An interactive visual tool for scienti c literature search: Proposal and algorithmic speci cation . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 76 { 87 . CEUR-WS.org ( 2019 )

3. Beel , J. , Langer , S. , Gipp , B. , Nurnberger, A.: The architecture and datasets of docears research paper recommender system . D-Lib Magazine 20 ( 11 /12) ( 2014 ). https://doi.org/10.1045/november14-beel

4. Bohannon , J.: A computer program just ranked the most in uential brain scientists of the modern era . Science ( 2016 ). https://doi.org/10.1126/science.aal0371

5. Cabanac , G. , Chandrasekaran , M.K. , Frommholz , I. , Jaidka , K. , Kan , M.Y. , Mayr , P. , Wolfram , D . (eds.): BIRNDL'16: Proceedings of the Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries co-located with the Joint Conference on Digital Libraries , vol. 1610 . CEUR-WS , Aachen ( 2016 )

6. Cabanac , G. , Mayr , P. , Frommholz , I. : Bibliometric-enhanced information retrieval: Preface. Scientometrics 116 ( 2 ), 1225 { 1227 ( 2018 ). https://doi.org/10.1007/s11192-018-2861-0

7. Castelvecchi , D. : Google unveils search engine for open data [News & Comment] . Nature ( 2018 ). https://doi.org/10.1038/d41586-018-06201-x

8. Fabre , R.: A searchable space with routes for querying scienti c information . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 112 { 124 . CEUR-WS.org ( 2019 )

9. Farber, M. , Jatowt , A. : Finding temporal trends of scienti c concepts . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 132 { 139 . CEUR-WS.org ( 2019 )

10. Gar eld, E.: Citation indexes for science: A new dimension in documentation through association of ideas . Science 122 ( 3159 ), 108 { 111 ( 1955 ). https://doi.org/10.1126/science.122.3159.108

11. Gar eld, E.: Patent citation indexing and the notions of novelty, similarity, and relevance . Journal of Chemical Documentation 6 ( 2 ), 63 { 65 ( 1966 ). https://doi.org/10.1021/c160021a001

12. Haunschild , R. , Marx , W. : Discovering seminal works with marker papers . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 27 { 38 . CEUR-WS.org ( 2019 )

13. Jaidka , K. , Chandrasekaran , M.K. , Rustagi , S. , Kan , M.Y.: Insights from CL-SciSumm 2016: The faceted scienti c document summarization shared task . International Journal on Digital Libraries 19 ( 2 {3), 163 { 171 ( 2018 ). https://doi.org/10.1007/s00799-017-0221-y

14. Kim , J. , Trippas , J.R. , Sanderson , M. , Bao , Z. , Croft , W.B.: How do computer scientists use Google Scholar?: A survey of user interest in elements on SERPs and author pro le pages . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 64 { 75 . CEUR-WS.org ( 2019 )

15. Lamirel , J.C. , Cuxac , P. : Feature selection and graph representation for an analysis of science elds evolution: An application to the digital library ISTEX . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 88 { 99 . CEUR-WS.org ( 2019 )

16. Leydesdor , L. , Milojevi , S. : Scientometrics. In: Wright, J.D . (ed.) International Encyclopedia of the Social & Behavioral Sciences , vol. 21 , pp. 322 { 327 . Elsevier , 2nd edn. ( 2015 ). https://doi.org/10.1016/b978-0 -08-097086-8 . 85030 - 8

17. Lykke Nielsen , B. , Lavlund Skau , S. , Meier , F. , Larsen , B. : Optimal citation context window sizes for biomedical retrieval . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 51 { 63 . CEUR-WS.org ( 2019 )

18. Mayr , P. , Chandrasekaran , M.K. , Jaidka , K . (eds.): BIRNDL'17: Proceedings of the 2nd Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries co-located with the Joint Conference on Digital Libraries , vol. 1888 . CEUR-WS, Aachen ( 2017 )

19. Mayr , P. , Chandrasekaran , M.K. , Jaidka , K . (eds.): BIRNDL'17: Proceedings of the 3rd Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries co-located with the Joint Conference on Digital Libraries , vol. 2132 . CEUR-WS , Aachen ( 2018 )

20. Mayr , P. , Frommholz , I. , Cabanac , G . (eds.): BIR'16 Proceedings of the 3rd Workshop on Bibliometric-enhanced Information Retrieval co-located with the 38th European Conference on Information Retrieval , vol. 1567 . CEUR-WS , Aachen ( 2016 )

21. Mayr , P. , Frommholz , I. , Cabanac , G . (eds.): BIR'17 Proceedings of the 5th Workshop on Bibliometric-enhanced Information Retrieval co-located with the 39th European Conference on Information Retrieval , vol. 1823 . CEUR-WS, Aachen ( 2017 )

22. Mayr , P. , Frommholz , I. , Cabanac , G . (eds.): BIR'18 Proceedings of the 7th Workshop on Bibliometric-enhanced Information Retrieval co-located with the 40th European Conference on Information Retrieval , vol. 2080 . CEUR-WS ( 2018 )

23. Mayr , P. , Frommholz , I. , Cabanac , G. : Report on the 7th International Workshop on Bibliometric-enhanced Information Retrieval (BIR 2018 ). SIGIR Forum 52 ( 1 ), 135 { 139 ( 2018 ). https://doi.org/10.1145/3274784.3274798

24. Mayr , P. , Frommholz , I. , Cabanac , G. , Chandrasekaran , M.K. , Jaidka , K. , Kan , M.Y. , Wolfram , D.: Special issue on bibliometric-enhanced information retrieval and natural language processing for digital libraries . International Journal on Digital Libraries 19 ( 2 {3), 107 { 111 ( 2018 ). https://doi.org/10.1007/s00799-017-0230-x

25. Mayr , P. , Frommholz , I. , Mutschke , P. (eds.): BIR'15 Proceedings of the 2nd Workshop on Bibliometric-enhanced Information Retrieval co-located with the 37th European Conference on Information Retrieval , vol. 1344 . CEUR-WS , Aachen ( 2015 )

26. Mayr , P. , Schaer , P. , Scharnhorst , A. , Larsen , B. , Mutschke , P. (eds.): BIR'16 Proceedings of the 1st Workshop on Bibliometric-enhanced Information Retrieval co-located with the 36th European Conference on Information Retrieval , vol. 1143 . CEUR-WS , Aachen ( 2014 )

27. Mayr , P. , Scharnhorst , A. : Scientometrics and information retrieval: weak-links revitalized . Scientometrics 102 ( 3 ), 2193 { 2199 ( 2015 ). https://doi.org/10.1007/s11192-014-1484-3

28. Perier-Camby , J. , Bertin , M. , Atanassova , I. , Armetta , F. : A preliminary study to compare deep learning with rule-based approaches for citation classi cation . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 125 { 131 . CEUR-WS.org ( 2019 )

29. Pritchard , A. : Statistical bibliography or bibliometrics? [Documentation notes] . Journal of Documentation 25 ( 4 ), 348 { 349 ( 1969 ). https://doi.org/10.1108/eb026482

30. Saier , T. , Farber, M.: Bibliometric-enhanced arXiv: A data set for paper-based and citation-based tasks . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 14 { 26 . CEUR-WS.org ( 2019 )

31. Salton , G.: Associative document retrieval techniques using bibliographic information . Journal of the ACM 10 ( 4 ), 440457 ( 1963 ). https://doi.org/10.1145/321186.321188

32. Shah , T. , Pudi , V. : Mining intellectual in uence associations . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 100 { 111 . CEURWS.org ( 2019 )

33. Shotton , D. : Funders should mandate open citations . Nature 553 ( 7687 ), 129 ( 2018 ). https://doi.org/10.1038/d41586-018-00104-7

34. Shvets , A. : Improving scienti c article visibility by neural title simpli cation . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 140 { 147 . CEUR-WS.org ( 2019 )

35. Sinha , A. , Shen , Z. , Song , Y. , Ma , H., Eide , D. , Hsu , B.J.P. , Wang , K. : An overview of Microsoft Academic Service (MAS) and applications . In: Gangemi, A. , Leonardi , S. , Panconesi , A . (eds.) WWW'15: Proceedings of the 24th International Conference on World Wide Web . pp. 243 { 246 . ACM , New York, NY, USA ( 2015 ). https://doi.org/10.1145/2740908.2742839

36. Tang , J. , Zhang, J., Yao , L. , Li , J. , Zhang, L. , Su , Z. : ArnetMiner: Extraction and mining of academic social networks . In: KDD'08: Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining . pp. 990 { 998 . ACM , New York, NY, USA ( 2008 ). https://doi.org/10.1145/1401890.1402008

37. Vagliano , I. , Nazir , S. : Recommending multimedia educational resources on the MOVING platform . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 148 { 158 . CEUR-WS.org ( 2019 )

38. Van Noorden , R. : Google Scholar pioneer on search engines future . Nature ( 2014 ). https://doi.org/10.1038/nature. 2014 .16269

39. White , H.D. , McCain , K.W. : Visualizing a discipline: An author co-citation analysis of Information Science, 1972 { 1995 . Journal of the American Society for Information Science 49 ( 4 ), 327 { 355 ( 1998 ). https://doi.org/b57vc7

40. Wiggers , G. , Verberne , S. : Citation metrics for legal information retrieval systems . In: Proc. of the 8th Workshop on Bibliometric-enhanced Information Retrieval . pp. 39 { 50 . CEUR-WS.org ( 2019 )

41. Williams , K. , Wu , J. , Choudhury , S.R. , Khabsa , M. , Giles , C.L. : Scholarly big data information extraction and integration in the CiteSeer digital library . In: ICDE'14: Proceedings of the 30th IEEE International Conference on Data Engineering Workshops . pp. 68 { 73 . IEEE ( 2014 ). https://doi.org/10.1109/icdew. 2014 .6818305