In this paper, we describe our query expansion approach submitted for the Semantic Enrichment task in Cultural Heritage in CLEF (CHiC) 2012. Our approach makes use of an external knowledge base such as Wikipedia and DBpedia. It consists of two major steps, concept candidates generation from knowledge bases and the selection of K-best related concepts. For selecting the K-best concepts, we ranked them according to their semantic relatedness with the query. We used Wikipedia-based Explicit Semantic Analysis to calculate the semantic relatedness scores. We evaluate our approach on 25 queries from the CHiC Semantic Enrichment dataset.
With the enormous amount of information emerging on the Web, the gap
between vocabularies used in indexed documents and user queries has been
increased. To ll this gap, many query expansion methods such as dictionary-based
query expansion [
One possible semantic enrichment of a query can be achieved by using the
Wikipedia or DBpedia. Wikipedia is a freely available large knowledge resource
built by a collaborative e ort of voluntary contributors. DBpedia [
In this work, we present an approach for semantic enrichment of queries using Wikipedia, DBpedia, and ESA based ranking. The rest of this paper is organized as follows: Section 2 describes our approach in detail; Section 3 explains our four di erent submitted runs for the semantic enrichment task; Section 4 shows the results; and nally we conclude in Section 5. 2
Our approach consists of two major steps; concept candidates generation from Wikipedia and DBpedia, and selection of K-best concepts. 2.1
The concept candidates are the titles of Wikipedia articles, which are relevant to the given query. To retrieve these concept candidates, we search all of the Wikipedia articles with the given query, and sort them according to their TFIDF scores. Among the retrieved articles the N best articles are selected as concept candidates. Then, we nd all the directly connected Wikipedia articles to the top ranked article from the N selected candidates, in the DBpedia graph. For example, for a given query \Hiroshima", we retrieve 260 di erent Wikipedia articles, such as \Atomic bombings of Hiroshima and Nagasaki", \Mazda Stadium", and \Nagasaki". With the intuition that the concepts containing similar strings may not provide the additional relevant documents, we exclude those concept candidates, which have a low edit distance to the query. 2.2
ESA attempts to represent the semantics of the given term in the high
distributional semantic space. These semantics are obtained by use of a high dimensional
vector, where each dimension re ects a unique Wikipedia concept. This high
dimensional vector is created by taking the TF-IDF weight of a given term in the
corresponding Wikipedia articles. Semantic relatedness of two given terms can
be obtained by calculating the correlation between two high dimensional vectors
generated by ESA. We used the ESA implementation described in [
We submitted four di erent runs in the semantic enrichment task at CHiC. All of these runs are based on the approach described in Section 2. They use di erent threshold of edit distance to eliminate the concept candidates. The values of N and K are taken as 20 and 10 respectively. Run1 excludes those concept candidates, that contain the query string as a substring. For example, for a given query \Hiroshima", we eliminate all the concept candidates such as \Hiroshima Prefecture", \Hiroshima Station", and \Hiroshima University", as they contain \Hiroshima" as a substring. Run2 and run3 exclude those concept candidates, that have token distance greater than 0.5, and 0.0 respectively. Run4 is the baseline, and does not perform the elimination step.
Run Type DERI SE1 CLEF-se(Run1) DERI SE2 CLEF-se(Run2) DERI SE3 CLEF-se(Run3) DERI SE4 CLEF-se(Run4) All of these runs are evaluated in two di erent phases: manually, and by using a query expansion experiment with a standard IR system. All of the suggested concepts are assessed manually for use in an interactive query expansion environment to check if these suggestions make sense with respect to the original query. These manual relevance assessment measures are on a three point scale: de nitely relevant, maybe relevant, and not relevant. Table 1 shows the scores of weak precision and strong precision. Strong precision is the average precision over 25 queries of the \de nitely relevant" suggestions, and weak precision is the average precision of the \de nitely relevant", and \maybe relevant", over all
In order to evaluate the approach in a query expansion environment, all of the suggestions are used as additional terms to the query. With these enriched queries, the results are assessed according to the ad-hoc retrieval standards. Then, the average precision and recall are calculated. In Table 2, we report the Mean Average Precision (MAP) of all the runs. Run1 performs the best, suggesting that concepts may not improve the results over the original query if they contain the query as a substring. For instance, for a given query \Hiroshima", the suggestion \Hiroshima University" may not help to nd the relevant documents, that cannot be found by the query \Hiroshima". Figure 2 shows the mean interpolated precision scores against di erent recall values, and Figure 1 shows the retrieved documents vs. mean precision, for all of the submitted runs. 5
We presented our approach for query expansion, which includes concept candidates generation from Wikipedia and DBpedia, and the selection of K-best concepts according to the ESA scores. The approach reached high precision according to the manual relevance assessment evaluation, meaning that most of the suggestions make sense in query expansion. Also, it raises the future direction of query expansion by using Wikipedia and DBpedia. Therefore, we are planning to investigate this approach with di erent ranking methods, and by taking more Wikipedia features, such as Wikipedia link and category structure, into account.
This work is supported in part by the European Union under Grant No. 248458 for the Monnet project and by the Science Foundation Ireland under Grant No. SFI/08/CE/I1380 (Lion-2).