The German Information Retrieval community is located in two diferent sub-fields: Information and computer science. There are no current studies that investigate these communities on a scientometric level. Available studies only focus on the information scientific part of the community. We generated a data set of 401 recent IR-related publications extracted from six core IR conferences from a mainly computer scientific background. We analyze this data set at the institutional and researcher level. The data set is publicly released, and we also demonstrate a mapping use case.
2000. More recent studies are scarce and mostly look at the information science side of the community or do not look at the German community specifically.
Therefore in this work, we will investigate some recent characteristics of the German IR community by building a data set of recent publications from six core IR conferences. We will analyze this data set at the institutional and the individual researcher levels to learn more about central actors in the field and to show first and preliminary results of this study.
An example of a German-focused study is the work of Baumgartner and Schlögl [
In 2015, Lewandowski and Haustein [
More tailored explicitly to the IR community, the bibliometric studies of Ding et al. [
While conducting scientometric studies one has to keep in mind that these kinds of studies are often limited. Selection bias and inclusion/exclusion criteria are usual suspects when it comes to the validity of these kinds of studies [7]. These are not only introduced on the researcher level, who chooses which publication to include in the analysis but also on the data provider level [8]. Database curators can influence the coverage of the databases scientometric researchers work on and therefore influence the results.
We compiled a data set of research publications by analyzing the proceedings of six major IR-related and peer-reviewed conferences (CHIIR, CIKM, CLEF, ECIR, SIGIR, WWW) available in the ACM Digital Library. The publication dates ranged between January 2020 and June 2023. We only considered those publications that had at least one German author or an author that was afiliated with a German research institute. Other central publication venues of the community, like the Springer Information Retrieval Journal or ACM TOIS were not considered, as we wanted to focus on conferences. The TREC and CLEF workshop notes were also not included due to the missing peer review.
For the six conferences mentioned above, we gathered the following publications’ metadata: • author names, • afiliation, • titles, • DOI of the publication.
As afiliation names were given in many diferent variants, we harmonized them manually. We added detailed information about the working group as well as the complete postal address and geo-location. In the case of diferent departments of research groups within an institute, we kept all sub-groups separate as long as the sub-group names were explicitly mentioned in the publications. In total, we discovered 401 publications and 195 diferent afiliations. The data set is publicly available in a GitHut repository3.
From the 195 research groups we found in the data set, we extracted the ten most productive ones (see Table 1). The most productive group is the Webis Group from Weimar, Leipizg, Jena, and formerly Halle, followed closely by the Databases and Information Systems group from Max Planck Institute and the L3S research center. Webis and L3S are “virtual groups” whose members also have co-afiliations with universities or non-university research centers (like TIB Hannover). We decided to not split up these publications, as the publishing collaboration within these institutes is intense, and authors publicly afiliate with them. Due to the size of these groups, it’s not surprising to see them at the top of the list. If we would have split up these groups, the ordering would not have changed a lot, as the two central locations of Webis Weimar and Leipzig would still have made it to the top.
The list also includes one commercial research institute (Bosch Center for Artificial Intelligence), one university of applied sciences (TH Köln), and two non-university research centers (GESIS and TIB), showing the interdisciplinary and heterogeneous constitution of the German IR community.
Next to the sum of all publications within the time frame, we separated the count for each of the six core conferences. We also see a mixed set of publication profiles. The University of Regensburg only published at CHIIR and ECIR, while Max Planck, Webis, and L3S had a more heterogeneous coverage.
3https://github.com/irgroup/LWDA2023-IR-community
Webis Group Max Planck Institute - Databases and Inf. Systems Forschungszentrum L3S GESIS - Leibniz-Institut für Sozialwissenschaften TIB - Forschungsgruppe Visual Analytics U Bonn - Data Science & Intelligent Systems U Regensburg - Chair of Information Science U Mannheim - Data and Web Science Group Bosch Center for Artificial Intelligence TH Köln - Information Retrieval Research Group Total
In contrast to the previously mentioned observations from the field of information science, we wanted to check on the characteristics of co-authorship and collaboration in more computer science-related IR conferences. In Table 2 the number of authors ranges from 1 to 17 for single publications. On average, the top ten research groups published papers with 4.83 authors, while on the whole data set the average number of authors was 4.98. With eight authors the Webis Group has the highest number of authors per paper on average, while the second most productive group (Max Planck) has the lowest number of authors per paper (three on average). The number of authors alone can therefore not explain the publication success of a group. We should nevertheless keep in mind that the number of co-authors per paper can introduce distortions for the calculation of additional network-based performance metrics [9].
Given these limitations, we analyzed the publications with the help of a co-authorship network. All author collaborations form a network of 1159 nodes (authors) and 4907 edges (co-authorship Author Lucie Flek Martin Potthast Ralph Ewerth Benno Stein Jens Lehmann Stefan Dietze Gerhard Weikum Avishek Anand Rishiraj Saha Roy Daniel Hienert Kuldeep Singh Megha Khosla Norbert Fuhr Andrew Yates Axel-Cyrille Ngonga Ngomo Matthias Hagen Ran Yu Chris Biemann Sherzod Hakimov Maria-Esther Vidal Endri Kacupaj Henning Wachsmuth David Elsweiler Andreas Both Janek Bevendorf Maria Maleshkova Philipp Schaer Timo Breuer Alexander Bondarenko Yvonne Kammerer
Afiliation U Bonn / U Marburg Webis Group TIB Hannover Webis Group Amazon GESIS, Köln Max Planck Institute L3S Max Planck Institute GESIS, Köln Cerence L3S U Duisburg-Essen Max Planck Institute U Paderborn Webis Group GESIS, Köln U Hamburg U Potsdam L3S Cerence Webis Group U Regensburg
HTWK Leipzig Webis Group
U der Bundeswehr, Hamburg TH Köln TH Köln Webis Group HDM Stuttgart 9 relations). To allow a more nuanced impression in comparison to simple publication counts, we calculated betweenness centrality on the network. Betweenness centrality tells us how important a node is to form the network by connecting diferent parts. In a co-authorship network, these are authors that bridge diferent communities of working groups and are important for the connectivity of the network.
In Table 3, we see the top thirty best-connected authors of the field. We only selected those researchers who work at German research institutes. However, we can see in the co-authorship network that many other well-connected researchers are from outside of Germany. The topranked researcher is Lucie Flek (U Bonn and U Marburg), with only three publications in total, which might be surprising, but these papers were published at WWW, SIGIR, and ECIR and had no single overlap in co-authors. Therefore she is a well-connected author in the co-authorship
We applied a simple topic modeling of the top ten research institutes based on the titles of the publications published by each institute. The titles are combined into documents and the terms in these documents up to bi-grams are weighted by TF-IDF after a basic text processing. The top terms per institute with the highest TF-IDF give insights about the topics the institute mainly focuses on, and which diferentiates it from the other groups. Table 4 gives an overview of the top 3 terms per research institute. While most terms like knowledge, question answering or search are related to IR, some terms appear unexpected. The highest ranked term for the Bosch Center for Artificial Intelligence is, for example, welding. Since this term is rarely used in the context of IR, in two publications the institute describes how welding can be monitored through the help of IR techniques [10, 11].
Likewise, the title terms of the publications with German authors were analyzed by conference. By comparing the top terms of the groups with the top terms of the conference, a group mainly publishes some correlations could be found. For example, for the Max Planck Institute with the Databases and Information Systems group question answering has a high TF-IDF, which ranks high for the SIGIR conference in which the group mainly publishes. Other indirect similarities can be observed for example between the L3S with machine learning-related terms and the CIKM. Since the initial data set for the topics of the conferences and groups is the same and, therefore productive groups significantly influence the top terms per conference, a correlation is not surprising.
We use the available data to draw the geo-locations of each research group on a map using the OpenStreetMap platform. The map is available online4. The map includes all groups and institutes that published at least two times in the six previously mentioned conferences. To extend the map with additional institutes that might be missed, we added groups that were active in academic societies, like the German Special Interest Group on Information Retrieval between 2020 and 2023. Each data point on the map includes the name of the group, the postal address, and a link to the institute’s homepage. Figure 1 shows a sample of the map.
We conducted a small-scale scientometric study of the German IR community using publications from 2020 till mid-2023. For a more in-depth investigation, we would need a more extended time coverage and should investigate more conferences to reflect the field’s heterogeneity. Additionally, the chosen time frame mainly consists of the COVID-19 pandemic, which might have introduced some uncommon publication patterns (like submitting to conferences without traveling to these conferences). The six conferences are relevant to the field, but other related conferences like JCDL or the ICTIR might also be included in a later version of the data set. Including CIKM might have introduced some topical shift in the data set, as the main focus of CIKM is not information retrieval (although some relevant IR papers are located there). A more ifne-tuned topical selection process might increase the quality of the data set. Additionally, the decision to discard TREC and CLEF lab notebooks also excluded some potentially interesting publications and, consequently, may be the reason for missing research groups.
While all these limitations are known and are valid complaints regarding the preliminary results of this scientometric study, it’s the only available data set for the German IR community. The dataset in its current form therefore only allows us to get some first and preliminary insights into the community, and its actors on the institutional and researcher levels. Nevertheless, it gives us an idea of the rich collaborations happening in the field.