Faceted search and exploration is efective and desirable when confronted with bibliographic data sets. As this data is mostly extensive, a powerful and flexible information retrieval tool needs to be provided. We propose FacetSearch; a scalable prototype especially designed for 1) comparing and visualizing trends in corpora via a trend visualization, as well as 2) exploring data and relations between elements with a pivot operation. Metadata can be accessed in multiple dimensions with facets and filter rules on them to help refine search queries or navigate through a corpus. An evaluation showed the potential and user acceptance of our approach when compared to the textual search engine Semantic Scholar.
With the ever-growing amount and richness of bibliographic data, exploring
and interpreting this information with help of tools is becoming increasingly
important. A simple textual search is not suficient to access the collection in
its full dimension if a user has no prior knowledge of the data being looked
for. Faceted search can help overcome this by providing attributes with which
queries can be refined or a corpus can be explored. Some works even point out,
that users prefer searching with facets instead of only textual search [
When comparing applicants for an open position in science, multiple aspects of researchers need to be considered. Not only their number of publications but also their h-index, their domain, the conferences and journals they published in and their publication development over the years could be relevant. In literature research, examining quality and impact of citations as well as getting a brief insight into papers is essential to estimate their usefulness and centrality. For scientists exploring a corpus is important to detect interesting researchers and publications associated with certain topics. All these tasks require a powerful information retrieval system. We introduce a prototype tailored to tackle such information needs: FacetSearch, a system espacially directed at visualizing and
P
R1 R2
C1 C2 comparing trends and relations in large bibliographic collections as well as giving the opportunity to explore them by usage of faceted search.
This work is structured as follows: in Section two, we introduce some basic definitions and our data set. Afterwards, the FacetSearch interface is described in detail and evaluated in the following chapter. We close with related work and the description of possible improvements for our preliminary system.
There are diferent information needs when working with an information system. A simple search is about refining a result set from many to few elements with the goal of finding the most relevant element for a query. Exploration is a much more imprecise task with no certain ambition except to casually examine an information space and navigate between elements.
Metadata can be clustered in alternative hierarchical categories in the information space. These categories are called facets. Facets contain elements, which are the aggregated potential values that occur in the corresponding metadata dimension. Queries on elements of facets are called filter rules , they generate and refine results. A rule set is composed of all currently active filter rules. Faceted search interfaces support exploration and search by suggesting fitting facet filter options based on the respective rule set to progressively specify the results.
In the context of a publication-centered faceted search system, facets could be author names, journals or keywords attributed to publications, elements in a facet containing keywords could be database, machine learning and algorithm. A iflter rule on a facet containing keywords could query for algorithm and retrieve publications attributed with this keyword.
For our use case of bibliographic data, we need to specify linkages between publications. We distinguish between citations and references as visualized in Figure 1. Citations are the publications connected by ingoing links to a paper from its citing publications, references are the papers connected by outgoing links from a publication to papers which are cited. 2.2
Our prototype uses data from the dblp computer science bibliography. The dblp
corpus contains bibliographic metadata concerning authors, publications as well
as venues (journals and conferences) from the field of computer science and
adjacent areas [
In dblp, author names, publication titles and related venues are stored, the extended collection additionally contains citations, references, abstracts, full texts, keywords and topics of publications. All supplementary data is partial. Except for abstracts and full text, available metadata is used as elements in facets. Topics are merged into one facet with keywords as these attributes are similar but often missing. The additional data is useful in indicating quality of papers (citations and references) as well as providing further aspects to explore (abstracts, full texts) and redefine (keywords) the data. – Papers By Author – Authors By Paper – Papers By Collection – Authors By Collection – Papers By Keyword – Cited Papers By Author – Authors Cited By Author – Referenced Papers By Author – Papers Referencing Paper – Papers Cited By Paper – Papers Referencing Collection – Papers Cited By Collection – Authors Referencing Author The FacetSearch interface can be partitioned into five main components as seen in Figure 2: At the top (A), a menu is given with which a user can switch rule sets and select which facets are being displayed. In the facet area (B), currently chosen facets are depicted to explore the data with. Above in the filter (C), the current rule set is given. On the right in the item view (D), elements retrieved by the query donated by the active rule set are listed. Lastly, in the trend visualization (E), the temporal development of elements from rule sets is pictured.
The arrangement of areas in the interface is highly influenced by FacetLens
[
The facet Papers By Author contains all author names as elements. If the elements Ralf Schenkel and Gerhard Weikum are selected and conjugated in a filter rule, a query for papers which are co-authored by Ralf Schenkel and Gerhard Weikum is initiated and publications fitting all requirements of the current rule set are retrieved and listed in the item view. The elements in all facets are updated according to the restricted result set.
In addition to the traditional facets, two facets Papers By Year and Papers By Type are always visible, they do not contain dynamically generated elements. These facets refine the time interval and the desired publication type of results.
Users can switch between multiple rule sets. Filter rules contained in them are conjugated and resulting entries are listed in the item view. It can be switched between a publication and an author view of the results, the default view is denoted by the X -component of the facet, on which the last filter rule was generated. A search is available on the item view. If an element from the item view is selected, a detail view opens as pictured in Figure 4. It provides all available information for an author or a publication.
Next to some attributes in the detail view, a button with an arrow symbol (marked red on the left detail view in Figure 4) ofers a shortcut-functionality of our prototype, the pivot operation. When clicking this button, a user breaks free from the current environment and creates a new rule set without dropping the old one. The only active filter rule in the new rule set contains the elements, which a user pivoted on. By pivoting, users can quickly explore the corpus.
In the case shown in the left of Figure 4, clicking on the pivot button next to the five authors (Ralf Schenkel, Andreas Broschart, Seung-won Hwang, Martin Theobald and Gerhard Weikum) of the publication Eficient Text Proximity Search generates a new rule set. A user has to decide whether the authors are combined conjunctive or disjunctive. In the new rule set, the only active filter rule would be the selection of the five authors as elements from the facet Papers By Author. The item view then holds all publications fitting these requirements.
The development of publications over time from diferent rule sets can be visualized in the trend visualization. With two drop-down menus, the depicted interval can be determined. There are several trend types available for display: number of publications, number of citations and number of references. With a click on a trend bar, a window with publications fitting the rule set from the
Fig. 5: Comparison of publications for re- Fig. 6: Comparison of citations of publications searchers R1 (blue) and R2 (green) for key- from researchers R1 (blue) and R2 (green) words database, inf ormation retrieval and for publications with keywords database, data mining. Hight of bars indicates number inf ormation retrieval and data mining. of publications per year. Hight of bars gives number of citations per year corresponding year opens. Trends of up to three rule sets can be compared in the trend visualization at the same time.
The trend visualization can be used for the estimation of keywords’ relevance. If the keyword logic programming is chosen in facet Papers By Keyword, the trend visualization shows the decreasing number of publications attributed with this term which could hint at a vanishing importance of the topic. The selection of the keyword smartphone instead set could show an increasing number of publications over the years which could indicate the growing significance of this area at a current time frame. 3.2
One of the tasks FacetSearch is suited for is the comparison of applicants for vacancies in academia. Manning these positions is challenging, as only experts in the vacant field can fully access quality and impact of candidates research. Let’s assume there are two researchers R1 and R2 who run for a professor position in database systems. In a first step, the number of publications of each applicant could be compared in this area which can be described by keywords database, inf ormation retrieval and data mining. When searching for publications from these authors (via facet Papers By Author) with OR-ed keywords (via facet Papers By Keyword), the comparison of results in the trend visualization is depicted in Figure 5. Based on these publication trends R1 would be the better candidate as he publishes more papers. Figure 6 shows the outcome if the number of citations of the publications of the authors is compared. Even though R1 did publish more than R2, R2 is cited much more often which indicates that R2 is more important than R1 and would be a more suitable population for the open position if no other aspects influence this decision.
FacetSearch is a prototype implemented in Java 7 using local indices which store bibliographic information. Elasticsearch is used for saving and searching data. Q1 Who are the co-authors of author A in his most cited publication? How many publications do they have together? Q2 What is the most frequent keyword associated with author A between years Y1 and Y2? Q3 How many publications has author A in collection C? Q4 Which of the publications citing publication P is the most cited one? Q5 Compare collection C1 and C2: which one contains more publications with keyword K between years Y1 and Y2? Q6 Compare authors A1 and A2: who was cited more between years Y1 and Y2?
One index stores information concerning authors (unique name, possible further names, h-index), the other one contains data regarding publications (title, unique author names, publication year, collection in which it was published, keywords, full text, citations, references, average h-index of authors) as far as available.
In Elasticsearch, the maximum result size of a query is set to 10,000 elements by default. As we did not change that value, it is the top number of elements returned in facets where Y = Author or Y = Paper if a search is performed on them. A search on authors could query for the primary unique name or a further name. Fitting names are looked up in the author index. For every author with good scoring, the unique name is used in the publication index to test if the author has at least one paper matching the current rule set and if so, the author is considered further. Search terms with high frequency such as John or the should be avoided or rendered more precisely to prevent the case of an element existing and fitting the current rule set but not being retrieved. 4
A direct comparison of FacetSearch with its predecessors PaperLens [
Our comparative study used FacetLens as a faceted information retrieval system
and Semantic Scholar[
Between participants, the order of the presented systems was alternated to
compensate for learning efects. Test users were given tutorials of both systems
before they started working on the respective tasks. A pre-evaluation with two
diferent subjects showed the need for another training step between tutorial
and evaluation with FacetSearch as the time to learn [
X × dif mean X × dif mean Q1 8 2 3.5 9 1 4.1 Q2 9 1 2.4 2 8 4.3 Q3 9 1 2.4 9 1 3.2 Q4 8 2 4.9 6 4 4.9 Q5 9 1 3.2 8 2 3.4
Q6 3 7 3.9 7 3 4.4 eaS ceS 1FQ 1SQ eaS ceS 2FQ 2SQ eaS ceS 3FQ 3SQ eaS ceS 4FQ 4SQ eaS ceS 5FQ 5SQ eaS ceS 6FQ 6SQ (b) Number of right and wrong answers per system and question (a) Time per question answered correctly in seconds with mean of dificulty given by for FacetSearch and Semantic Scholar. test users. primarily underestimated. Two basic training tasks were included before users worked on the evaluated questions. Participants solved these in training with help of an instructor. The whole session lasted about one hour for each member of the evaluation group.
Figure 7 gives an overview of the six types of questions posed for each of the systems, variables were changed between them. The first two questions were directed at correct handling of authors, the next two aimed at exposure with publications and the last tasks required usage of the trend visualisation. Not only the correctness of responses was measured, answering time was also taken. Additionally, users were asked to rate the dificulty of solving the posed questions with the respective systems on a scale from 1 to 7 (1 being very easy, 7 being very dificult) and judge functionalities of FacetSearch in the end. 4.2
FacetSearch Some works observe a change in search habits when a grouping
mechanism such as faceted search is available [
Semantic Scholar With Semantic Scholars intuitive operational concept, most
users had no dificulties in answering Q1 and Q3 correctly. The second task Q2
was solved wrong by eight users because in this search engine, keywords are
ordered without apparent criteria. Users were confronted with the challenge of
being unable to sort citations by attribute in Q4. The next question Q5 hardly
posed a problem except in the search for a collection as it cannot be searched for
directly in Semantic Scholar. The last task Q6 caused come confusion, several
users had problems in finding and comparing the citation diagrams.
Comparison In Figure 8, evaluation results from FacetSearch and Semantic
Scholar are compared. Figure 8a gives an overview of time needed for correctly
answering each of the six questions split by system. The plot indicates over all
participants were faster when using Semantic Scholar. Only when solving Q2
and Q6, FacetSearch was the faster option. Figure 8b quantifies the correct and
incorrect answers as well as the subjective mean of dificulty for the questions.
With FacetSearch, 76.67% of all tasks were finished correctly, with Semantic
Scholar, only 68.33% of results were correct. The rating of subjective dificulty
of solving the problems by test users lead to a median of 3 for FacetSearch
while resulting in 4 for Semantic Scholar. Participants did consider using the
faceted search prototype as easier when confronted with our exemplary tasks
even though their completion took longer. This phenomenon of initial problems
but successful and easy usage when confronting users with faceted search has
also been observed in user studies as the users’ ability to use and understand a
system evolves due to learning processes [
There are classes of tasks which cannot be solved by systems such as Semantic Scholar but are manageable with FacetSearch. Such tasks were not put in our evaluation as it would have resulted in failure for the text-based search engine. They include the following example problems: The best publication (e.g. with most citations) from a set of publications is hardly found as sorting options are missing. Exploration of publications from collections is dificult as no direct search is implemented for venues. Trends for referenced publications from an author or publication cannot be displayed as well as citation trends for more complex queries than single authors.
User Feedback The last part of the evaluation was a user feedback rating intuitiveness and overall implementation of certain aspects of FacetSearch. Table 1 gives an overview of the assessments.
Some participants of the evaluation were confused because facet elements are simultaneously used for input and output. Filter rules are generated from them but when a filter rule is added, elements dynamically change to match the query result. This phenomenon is complicated and a reason to classify FacetSearch as an expert system. One user fittingly noted that the more complex requirements for a system become, the more complex its operation becomes.
Other users remarked they trusted the results of FacetSearch far more than the results of Semantic Scholar because of its inconsistent data when for example searching for co-authors in author profiles. The stable structures in FacetSearch were complimented by many, overall consent was that with a suficient period of professional training, users would be much faster with our prototype. 5
FacetSearch is an extension of PaperLens [
Tiles are used in other research to display distribution and overlap of searched
terms in documents [
Dafodil [
Some works like FacetMap [
PaperQuest [
Further systems such as PivotPaths [
We presented FacetSearch, an information retrieval prototype designed for search and exploration of bibliographic data via facets as well as display and comparison of trends in result sets by usage of a trend visualization.
Our tool is able to keep up with text-based search engines such as Semantic Scholar according to our evaluation, although users thought of it as less intuitive (3.2). This observation is no surprise as there are a lot of features which need understanding and experience. Most persons coped well with usage of facets (5.2) and the trend visualization (5.1). Even compared to Semantic Scholar, our prototype was able to satisfy users (5.0). Nevertheless, FacetSearch is an expert system which needs a certain period of vocational adjustment.
Further research such as PaperQuest [
Another area for improvement would be the opportunity of incorporating
Doc2Vec [
Potential for future enhancements lies in the field of collections: Visualizing
the citation/reference linkage graph of collections could provide useful insights.
Venue ratings from Eigenfactor [