Introduction
The key objective of information retrieval systems is to satisfy the user's
information need in the provided context. In particular Interactive Information
Retrieval (IIR) is supposed to support the user beyond just typing in queries. In
this respect the principle of polyrepresentation is a highly recognized approach to
IIR [
1
]. It suggests to use various information/data representations to integrate
the context and interpretation of di erent actors into the information retrieval
process. The representations can come from the same actor but for di erent
purposes and are functionally di erent, or they can come from various actors and are
cognitively di erent. Polyrepresentation may refer to the information space, the
user's cognitive space or a combination thereof. In Bibliometrics, references
citing another source establish a cognitively di erent representation of a document.
Examples for di erent information need representations are work task
descriptions and the query. The principle of polyrepresentation in IR could be described
as follows: if multiple representations are pointing towards an information object
it is more likely to be relevant to the user's information need. This is depicted
in Figure 1. Let us assume R represents the relevance of a representation, hence
R1 denotes the documents relevant to representation 1, R2 to representation 2
and so on, so the documents in R1; R2 and R3 are only relevant to these
individual representations. The intersection of the two representations i.e, R12 R13
and R23 holds the documents relevant to the two respective representations, and
the intersection of all three representation, R123 is the total cognitive overlap.
According to the principle of polyrepresentation this set is supposed to hold the
most relevant documents as evaluated in [
2,3,4
].
The principle of polyrepresentation has been evaluated so far in ad hoc
retrieval. However, while di erent combinations of representations were evaluated
in a more static fashion, some open problems remain from a user perspective.
A system may be able to combine di erent representations, but the system
initially does not know whether and to what degree the user prefers some of them.
A weighting mechanism as proposed in [
5
] may mitigate the situation, but still
the system needs additional information. We argue that instead of presenting
users with a ranked list of results, we may present them with clusters they can
choose from. Clustering and polyrepresentation both create a partitioning of the
information objects under consideration. One approach to combine document
clustering and polyrepresentation is described by a \polyrepresentation cluster
hypothesis": documents relevant to the same representation should appear in
the same cluster [
6
].
The possible application of the above discussed approach could be presenting
the clusters to the user in a way depicted in Figure 1. In this case, we present the
user with a cluster containing a ranked list of documents representing the total
cognitive overlap rst. We assume the user browses the top k documents and
then moves on to the next cluster (one where only a subset of all representations
is relevant), depending on her representation preferences, where she examines
again some top k documents.
2
The science modeling and the bibliometrics provides the means to analyze and
quantify the structure and process of scholarly communication [
7
]. The methods
range from citation analysis [
8
], co-citation clustering [
9
] to bradfordizing [
10
].
The developments in e-publishing and availability of the full text and meta-data
regarding the scienti c information objects increased the scope and applicability
of the bibliometrics, hence more re ned measures are needed [
7
]. The
connections between IR, bibliometrics and relevance theory are discussed in [
11
]. The
citation networks and clustering methods suitable for block modelling in similar
context are discussed in [
12
]. The suitability of the bibliometirc measures for
enhancement of retrieval in scholarly systems is presented and evaluated in [
13,10
].
The authors have evaluated bibliometrics approaches like bradfordizing for
reranking and co-word model for query expansion in search term recommender and
report performance improvement. The principle of polyrepresentation in similar
context has been discussed in [
14
] using the references and citation information.
We look at the principle of polyrepresentation as a method go along with the
bibliometric approaches, because it allows the use of various representations, hence
the bibliographic information i.e. authors, references and the citation context
could be exploited as representations.
The Optimum Clustering Framework (OCF) [
15
] provides a theoretical
justi cation for clustering and is based on the notion of using so-called query sets.
We propose to apply the OCF for inferring the suitable candidate clusters
representing the various polyrepresentation sets R. The OCF operates on the
probability of relevance Pr(Rjd; qi) of the document d with respect to a query qi in
the query set; since in polyrepresentation we are dealing with measuring the
degree or probability of relevance for each representation, OCF is a suitable
framework for our ideas. From this, each document is represented by the vector
T (d) = (Pr(Rjd; q1); : : : ; Pr(Rjd; qn)) with n as the number of queries in the
query set. The vectors are now used to cluster documents with the choice of
clustering function depending on the overall setup. If each term in the
collection is regarded as a query qi the OCF simply models classical clustering using
term-based similarity.
To set up the OCF for polyrepresentation we need to distinguish between
polyrepresentation of information needs on the one hand and polyrepresentation
of documents on the other hand. In order to apply clustering to information need
polyrepresentation let REPin be the representations of an information need in.
Pr(Rjd; ri) is computed for each document d and ri 2 REPin. From this we
create a vector T (d) = (Pr(Rjd; r1); : : : ; Pr(Rjd; rn)) with n = jREPinj. When
applying polyrepresentation of documents, REPd consists of the di erent
representations rdi of a document d. In our case we assume that the information need
is represented by the query q alone1. We therefore need to compute Pr(Rjrdi; q)
and we get T (d) = (Pr(Rjrd1; q); : : : ; Pr(Rjrdn; q)) with n = jREPdj.
1 The combination of document and information need polyrepresentation is subject
to future work.
Evaluation
Collection and clustering approach In order to evaluate the proposed
approach, the PF part of the iSearch2 [
16
] collection is used. This sub collection
contains full text articles related to the physics domain. The collection comes
with 65 search tasks, each search task comprises upon ve information need
representations i.e. Search Terms, Work Task, Current Information Need, Ideal
Answer and Background Knowledge. These ve representation make REPin for
the R set for information need based polyrepresentation. In a rst step to
estimate Pr(Rjd; ri) BM25 based document weights were computed for every
information need representation ri using the Terrier IR platform [
17
]. The weights
for each representation were combined using CombSum [
18
] to get a retrieval
weight for each document based on the single representations. OCF-based
clustering was performed with k-means. k (in k-means) was set to 2jREPinj to create
exactly as many clusters as there are combinations of relevant representation in
polyrepresentation.
The second exploration has been about document based polyrepresentation,
for this the representations i.e title, abstract, body, context, and references were
extracted from the documents. The context c of a document d has been
extracted from all the articles cited in d, initially only the title and abstract of the
cited document were combined and used as a context. Hence, the context for a
document d becomes the concatenation of all abstracts and titles of the cited
documents in it, as depicted in Figure 2. The derived representations in this
part make the REPd part for the R set. The representation here has also been
indexed and the BM25 weights were computed as discussed above. The Search
Term part of the all 65 topics in the iSearch collection has been used as a query.
Baseline and Polyrepresentative Clustering The goal of our evaluation is
to verify the e ect of clustering on polyrepresentation. Therefore our baseline is
to combine the BM25 weights mentioned above from each representation using
CombSum, which creates a ranked list that utilises di erent representations but
no clustering.
2 http://itlb.dbit.dk/~isearch/
To compare our polyrepresentative clustering approach against the baseline
we create a new cluster-based ranking. As mentioned before we assume a system
that presents the user with the total cognitive overlap cluster rst, i.e. the user
is presented with a ranked list of documents in this cluster. We further assume
that the user examines the top k documents and then moves on to a di erent
cluster. To simulate this behaviour for evaluation purposes our strategy is as
follows. We rst create a clustering C as described above. From this we produce
an arti cial ranking as described in Algorithm 1. The ranking simulates the
documents that a user would see along the path when examining the top k
documents of each cluster. We rank the clusters in C so that the top ranked
cluster is an approximation of the total cognitive overlap (please see [
6
] for a
further discussion). We now process the cluster ranking in descending order.
We rank the documents in each cluster and append the topk documents to the
ranking.
Require: Clustering C, k
r () fThe ranking, initially an empty listg
C ranked list of clusters in C (using eF or SD)
for all cluster c 2 C do
l ranked list of documents in c fprocess C in descending weight orderg
for i = 1 to k do
r r + l[i] fappend document at rank i to rg
end for
end for
return r
Algorithm 1: Cluster-based ranking
To create the ranking of the clusters, we have used two measures. One is
the OCF based expected F-measure (eF ) as described in [
15
]. The second is the
sparsity density of the REP weight matrix constituting the cluster C, with the
idea to identify the clusters where many or all representations have contributed
and the corresponding cluster point matrix is less or not sparse. The sparsity
density (SD) of the cluster matrix is computed by counting the non zero elements
in the cluster matrix divided by the size of the matrix. In this approach the
matrix size is equal to the number of documents in the cluster (rows) multiplied
with the number of REP s (columns). The range of the density of the cluster is
between 0 and 1 where values closer to 1 show higher density. We have used eF
and SD to rank the clusters. The top k documents then were extracted form
each cluster and merged to create the rank for the trec eval, as described in the
Algorithm 1. For comparing the baseline and proposed approach we used k = 5
and k = 10. We extracted binary relevance judgements from the grades iSearch
ones with a value > 1 meaning relevance.
Results The iSearch collection consists of queries with a high and a low number
of relevant documents. It turns out the number of relevant documents for a query
has an e ect on the results. To document this, we provide gures for queries with
a high and a low number of relevant documents as well as for all queries. The
evaluation results for information need and document-based polyrepresentation
are shown in Table 1 for the queries where positive relevance judgments for 20 or
more documents are available (we call this High ). The Table 2 holds the results
for the queries where positive relevance assessment for less than 20 documents
were available, we refer to this as Low. The results for all the queries combined
are presented in Table 3, we refer to that as All. In the left half the tables the
results for information need based polrepresentation are given and the right half
holds the results for document and context-based polyrepresentation. In Table1
for information need based polyrepresentation, the proposed method performs
better than the BM25 baseline (IN BM25) at prec@n, prec@10, prec@20 and
so on, for both eF and SD, for k = 5 and k = 10. For the document-based
polyrepresentation part the performance of eF and SD at prec@5, prec@10,
prec@20 for both k = 5 and k = 10 is poorer than the baseline (DOC BM25). In
Table 2 for the Low queries for k = 5 and k = 10 the values for all prec@n are
very poor compared the baseline for information need polyrepresentation. In the
document-based polyrepresentation part the eF and SD for both k = 5 and k =
10 show a performance improvement over the baseline at all prec@n. In Table 3
for the information need part there is no improvement, for the document-based
Discussion The results show that our approach is able to improve the e
ectiveness, but there are interesting di erences when it comes to queries with a
low and a high number of relevant documents. For queries with high numbers
of relevant documents, polyrepresentative clustering based on documents and
bibliographic context does not perform well, whereas information need
polyrepresentation reacts well on clustering. It seems that with polyrepresentation of
information needs relevant documents are not necessarily found in the total
cognitive overlap and are unearthed by means of clustering. An explanation may
be that some combinations of information needs representations are not bene
cial [
19
] and clustering is a means to mitigate this. We see a di erent picture for
queries with only few relevant documents. Here, document-based
polyrepresentative clustering outperforms the polyrepresentation baseline, but information
need based clustering does not. For this reason the approach delivers a rather
mixed result when we consider all queries. We also observe that clustering has
a negative e ect on mean average precision (map) and recall, which is due to
the fact that we only select the top k documents from each cluster and drop the
rest.
In this study we have evaluated the suitability of combining principle of
polyrepresentation with document clustering. The idea is that instead of a ranked list
we present the user with clustering re ecting the relevance of documents w.r.t.
di erent representations. We report some insights on the performance of adding
bibliographic information as the representation as well as looking at
information need polyrepresentation. Apparently the aspect of presenting the results to
the user when various representations of information need as well as
information object come into play is a complex task. The initial exploration show that
presenting information to the user in illustrated way has its potentials and
drawbacks. We intend to test the approach for variable size k and the comparison of
the cluster ranking strategies, i.e. arithmetic mean, geometric mean etc, in the
future.