This paper describes the participation of DAEDALUS at the LogCLEF lab in CLEF 2011. This year, the objectives of our participation are twofold. The first topic is to analyze if there is any measurable effect on the success of the search queries if the native language and the interface language chosen by the user are different. The idea is to determine if this difference may condition the way in which the user interacts with the search application. The second topic is to analyze the user context and his/her interaction with the system in the case of successful queries, to discover out any relation among the user native language, the language of the resource involved and the interaction strategy adopted by the user to find out such resource. Only 6.89% of queries are successful out of the 628,607 queries in the 320,001 sessions with at least one search query in the log. The main conclusion that can be drawn is that, in general for all languages, whether the native language matches the interface language or not does not seem to affect the success rate of the search queries. On the other hand, the analysis of the strategy adopted by users when looking for a particular resource shows that people tend to use the simple search tool, frequently first running short queries build up of just one specific term and then browsing through the results to locate the expected resource.
This paper describes the participation of DAEDALUS team at the LogCLEF lab [
Specifically, three involved languages are considered in this research: language in which the user has set up the search tool interface, language of the collections of information on which the user makes his/her queries and/or navigates through the results, and the inherent language of the user (his/her native language), inferred based on the browser IP.
After our participation in the previous edition of LogCLEF [
In the following sections we will fully describe our analysis and the results and conclusions that can be drawn from this work. 2
As our analysis involves the identification and analysis of a sequence of actions carried out by the same user, only those entries in the log files for which it was possible to extract a session identifier have been considered, so as to be able to associate them to a set of related actions.
Based on the analysis of the data existing both in the log files and the action file
provided with The European Library data [
This model is similar to the one that we defined for our previous participation in
LogCLEF [
In order to deal with the first of our objectives, each query is modelled by a series of properties: • Action that triggered the query: we have considered that a query is triggered when the user makes any of the following actions: search_sim, search_adv, search_res, search_url, and also when the text of the query is modified. • Primary language: language selected in the user interface at the beginning of the session. • Secondary languages: list of languages, different to the primary language, which the user has selected in the interface, without any modification of the query. • Query language: inherent language of the query, inferred from the user IP address. • Number of filtering actions: a filtering action (search_res_rec_any, search_res_rec_all) is one that allows the user to refine the results associated to the query. • Number of browsing actions: a browsing action (view_brief, jump_to_page, page_brief) represents an interaction by the user on the search results, which is not any successful action. • Number of collections: number of different collections on which the user has carried out any action. • Number of different collections in which the language matches the language in which the user interface is configured. • Number of different collections in which the language matches the user language inferred from his/her IP address (native language). • Number of times that the user has carried out a view detail action (view_full). This action is very important as it leads to actions identified as successful actions. • Number of unsuccessful queries after the last successful query in the same session. • Successful query: a query is successful if it involves at least one of these actions: available_at, see_online, option_save_session_favorite, option_send_email. • Number of times that each successful action has been run.
Moreover, for each session in which a previous selection of the search collections has been made (by means of the col_set_theme_country action), the relationships existing among the language inferred by the IP address, the language in which the user interface is configured and the language associated to the selected collections, has been considered in the data model.
In addition to this information, to deal with our second objective, the following information has been extracted for each of the resources that have been requested by means of the available_at or see_online actions within a query and a session: • Successful action: one of the following actions: available_at, see_online. • Successful language: language of the interface when the action was run. • Resource: URL of the requested resource. • Number of filtering actions after the last successful operation, or, for the first successful action, the number of filtering actions from the first run of the query. • Number of jumping actions: a jumping action (jump_to_page) represents a navigation (or browsing) operation over the result listing. 3
Once the information in the log files has been filtered and organized according to the previously described model, 367,348 sessions are kept (i.e., those including significant information for our analysis) out of the 320,001 total sessions. This means that 12.88% of the started sessions do not involve any search operation. In those selected actions, a total of 628,607 queries have been made, 6.89% of which are successful, corresponding to a 11.11% of successful sessions.
The following Table 1 shows the average value of the main features in a session, considering whether the interface language matches the language inferred from the IP (Lang=1) or not (Lang=0). Parameters in rows include Sessions (number of sessions), Queries (average number of queries per session), Jumps (average number of navigation operation over the result listing), Filters (average number of filtering actions), Detail (average number of view_full actions), NotSuccess (average number of queries between two successful queries), and ActionSuccess (average number of successful actions).
It can be easily observed that the most frequent language for the interface is English, although it only matches the user language in 26.1% of queries.
Table 3 shows a detailed analysis of some selected parameters similar to Table 1 for the 5 main languages.
273,936 8,206 5,339 4,935 4,726 4,530 3,636 3,152 2,499 2,385 520,337 15,929 11,630 10,311 9,496 8,046 7,181 7,071 5,419 4,340
Parameter
After a correlation analysis of these figures, we could affirm that, in general for all languages (as shown in Table 1), the fact that the native language of the user matches or not the interface language, does not have apparently any impact on the success rate of the search queries.
However, there are noticeable differences in the detailed analysis for each language (Table 3), especially for German (20.5% increment in success when languages match). These differences have yet to be explained.
Another conclusion that can be drawn from Table 1 is that the filtering option in the interface does not receive a high interest from the users.
If we analyze the way the users carry out different types of queries, it can be noticed that there is no direct relation between the involved languages and the query type. Only 14.27% of queries make use of the advanced search form in the web page, and only 4.73% are successful as compared to the 6.89% of the rest of queries.
So as to explore the way in which users interact with the system when they are looking for a given resource, we have carried out a set of studies that focus on the resources that have been accessed by a given user after a search process and the queries that such user has run to locate them.
Regretfully, as dynamic parameters in the URL that identify the resources are not currently stored in the logs, the information provided was useful only for resources whose URL is static. Thus, this analysis is only possible for .jpg, .pdf, .txt and .doc resources.
For this analysis, we only have considered queries that allowed to access any of those resource types by means of an available_at or see_online action.
Assuming those criteria, 2,391 different queries have been identified, 6,002 requested resources and 6,884 different query-resource combinations.
Table 4 shows some statistics associated to the most frequent queries. Columns include Query (the user query), Queries (number of times that the query has been run), Sessions (number of different sessions), Resources (number of different requested resources), LangU (number of different user languages involved), LangI (list of different interface languages involved), and the number of matches between the interface and user language.
Again, it can be observed that the relation between the interface language and the user language does not have a strong effect on the success of the query. We believe that the main reason for such lack of correlation is due to the fact that most queries are composed up of just one search term, which typically are very specific queries containing a given proper name (such as the examples shown in Table 4). Thus, in this scenario, only 43 of the 304 queries that are formulated in more than one session (14%) contain more than one search term, and 29 of them (68%) correspond to a multiword proper noun (such as “da vinci”) . 4
The aim of our research was to study if there is any measurable effect on the success of the search queries if the native language and the interface language chosen by the user are different. Based on the results achieved, the main conclusion that can be drawn is that, in the general case, the fact that the native language is used or not as the interface language does not apparently affect the success rate of the search queries. In other words, whether this difference in languages conditions or not the way in which users interact with the search application does not have any significant impact on the success rate.
On the other hand, we have analyzed the strategy adopted by users when they are looking for a particular resource. People tend to use the simple search tool, frequently first running short queries build up of just one specific term and then browsing through the results to locate the expected resource.
For future participations in the task, we are still interested in researching on the actual semantic content of the query and its relation (if there is any) with any of the involved languages or the success of the query. Unfortunately we had to abandon this idea due to lack of time and resources, but we may be able to carry it out in future years.
This work has been partially supported by several Spanish research projects: MA2VICMR: Improving the access, analysis and visibility of the multilingual and multimedia information in web for the Region of Madrid (S2009/TIC-1542), MULTIMEDICA: Multilingual Information Extraction in Health domain and application to scientific and informative documents (TIN2010-20644-C03-01) and BUSCAMEDIA: Towards a semantic adaptation of multi-network-multiterminal digital media (CEN-20091026). Authors would like to thank all partners for their knowledge and support.