When search behaviour is studied in information retrieval it is nearly always studied with respect to work tasks. Recent research, however, has indicated that search tasks people perform in leisure situations can be quite di erent. In leisure contexts needs tend to be more hedonistic in nature and often don't require speci c information to be found. Instead, information is sought that can lead to a speci c emotional or physical response from the user, such as feelings of being stimulated or entertained. In this paper we investigate how people behave to meet such needs in one particular leisure context. We analyse search log data collected from a largescale (n=391), naturalistic study of behavior with a mobile search tool designed to help people nd events of interest to them at the Long Night of Museums, Munich. We examine the queries submitted, establish performance metrics and investigate how spoken queries di er from those typed via the keyboard on a mobile device. The ndings provide insight into how users behave in one speci c casual-leisure context and lead to several open questions for future research.
Search behaviour has traditionally been studied in the
context of people completing work tasks. Despite its name, a
work task need not be work-related. It is simply a sequence
of activities a person has to perform in order to accomplish a
goal [
Based on two investigative studies, one examining
information needs in the context of television viewing and the
other analysing broader information behaviour reported on
twitter, Elsweiler and colleagues [
Beyond these two studies, very little literature explicitly
focuses on information seeking behaviour in casual-leisure
situations. Exceptions include studies of nding ction [
In this paper we describe a study designed to answer these kinds of questions. We report analyses of interaction logs for a search system supporting one speci c leisure situation - the Long Night of Munich Museums, 2011. While we do not claim that the logs are representative of all casual-leisure search behaviour, they do provide an insight into how users behave in one speci c casual-leisure context and a situation where the user has a high-level, hedonistic goal. Our ndings represent a good starting point from which to investigate search behaviour more generally in casual-leisure situations. 2.
A distributed event is a collection of single events occurring at approximately the same time and conforming to the same general theme. One such event is the Long Night of Munich Museums (Lange Nacht der Munchner Museen), an annual cultural event organised in the city of Munich, Germany1. In addition to a diverse range of small and large museums, other cultural venues, such as the Hofbrauhaus and the botanical garden open their doors during one evening in October. Many venues organise special activities and exhibitions not otherwise available.
Visitors to the Long Night include both locals and tourists and represent a broad range of age groups and social backgrounds. In 2011 an estimated 20,000 people visited a total of 176 events at 91 distinct locations, including exhibitions, galleries and interactive events. Events take place all over the city, mostly in the city centre, but some, such as the Mu1The event is organised by Munchner Kultur GmbH (http://www.muenchner.de/museumsnacht/) seum of the MTU Aero Engines and the Potato Museum, are located in suburbs. Special bus tours are set up to transport visitors between events.
From interviews (n=25) we conducted with people attending the evening we know that on average each visitor attends 4 events meaning that approximately 80,000 visits took place in 2011. The standard way to discover events on o er is to use the booklet that is distributed for free by the organisers and contains descriptions of all events in the order they lie along the bus tours. This booklet is necessarily large (110 A6 pages) and can be di cult to navigate.
Only a few of our interviewees reported having speci c
events they would like to visit. Instead, most described
having the same kinds of high-level, hedonistic needs as reported
in the literature [
Here we want to establish how visitors to the Long Night of Museums query a search system to address these kinds of needs. We also want to know how successful they are, and identify noteworthy behaviours, problems and any potential solutions. The long-term goals of our work are to learn about behaviour in order to understand how to build better search tools and to augment existing theoretical models of casualleisure search. We present the results of initial analyses that lead to more detailed future research questions.
An Android app was developed to help visitors of the Long Night nd events of interest to them personally. Once they have found and indicated the events they would most like to visit, the system can create a time plan for the evening, taking into account constraints such as start and end times of events, time to travel between events and public transport routes and schedules. If the user chooses more events than would t into the available time2, then the system tries to maximise the number of scheduled events by leaving out those that require long travel time. It is also possible for the user to manually customise the plans by adding, removing and re-ordering events to be visited. Based on the created plan, the application can lead the user between chosen events using a map display and textual instructions. Figure 1 provides some screenshots of the app3.
The user has four ways to nd events he would like to visit, namely he can: Browse events by bus route; browse events by event type (e.g. exhibitions, guided tours, interactive event, etc.); submit free-text queries, which search over the names and descriptions of the events; receive recommendations based on a pre-de ned pro le and collaborative ltering algorithm built into the app.
In this paper, in line with the research aims as outlined
above, we focus on the way the search features were used.
The search functionality was implemented in Lucene4 and
documents were represented by titles and descriptions from
the Long Night booklet. Based on interviews conducted,
we expected visitors to search for topics or for other high
level needs not accessible for a full text search. Therefore
2most events are open between 7pm and 2am
3a video demo of the application can be found on YouTube
(http://www.youtube.com/watch?v=woVjpivxtMc)
4Lucene version 3.1. (http://lucene.apache.org)
we extended Lucene to perform a search based on topics. In
a rst step the event descriptions and titles were tokenised
and stemmed. To match topically similar words we then
map every token to one or more topic groups (these groups
are taken from [
We examined user search behaviour by recording user interactions with our app at the 2011 Long Night. The app was available for download from the Android Market and advertised on the o cial Long Night of Museums web page. In total the application was downloaded approximately 500 times and 391 users allowed us to record their interaction data. We recorded all interactions with the application including submitted queries, result click-throughs, all interactions with browsing and recommendation interfaces, tours generated, modi cations to tours, as well as all ratings submitted for events. Users interacted on average for 45.26 minutes5 with the system (median 19.31). 80.1% of users interacted for more than 5; 38.4% for more than 30.
A short questionnaire provided us with demographic
information. 51% of the app users were rst-time visitors to
the Long Night of Museums, 22% were second-time visitors
and 27% had attended more than twice previously. 4% of
users were 17 years of age or younger, 39% were between
18 and 29, 30% 30-39, 18% 40-49, 8% 50-59 and 1% above
60 years old. These demographics are very similar to those
reported by event organisers for previous Long Nights [
Since queries were submitted after every typed character, it was necessary to pre-process the recorded queries to establish those that the users actually intended to submit. For example, if the user wanted to search for \food", the system logged \f", \fo", \foo", as well as \food". Furthermore, should the user wish to submit a new query, then he must rst remove the old search terms from the search box, resulting again in all pre xes but this time in decreasing length.
Automatically extracting the intended query proved difcult due to spelling errors and automatic correction. We therefore manually judged queries to be intended or not. 3 assessors separately annotated all of the approx. 10,000 queries logged as being either intended or not-intended. A high inter-assessor agreement was found (Fleiss' kappa = 0.872, 86.2% of queries which were labeled by at least 1 assessor were also labelled by at least one other assessor). This process resulted in a nal list of 801 search queries, which is used in the following analyses.
Overall the search queries were short, having a mean length
of 1:21 terms ( = 0:52) and 8:9 characters ( = 5:31).
These values are much shorter than those reported for
similar mobile-like devices for web search. [
It was very apparent while analysing the queries that many represented searches for named entities, in particular the names of speci c museums. Again 3 human assessors were asked to assign queries into categories: speci c event name, not a speci c event name or indeterminate. The third category was necessary as some queries were short and it was not possible to de nitively claim that the term referred to a speci c event. For example \deutsches" is likely to be a reference to the \deutsches Museum" but it is not possible to say for certain. For 87.3% of all queries at least two of the assessors were able to agree on one of the three categories (Fleiss Kappa of 0.43).
59.4% of the agreed on queries were marked as clearly named entities and 34.6% that might be named entities. Only 6.0% were labeled as non named entity searches. These remaining searches were often queries for non-museum locations, e.g. 18.2% of these are names of bus stops.
Notably absent from the logs were queries describing topical content of events e.g. \art history", \engineering", \modern art", etc. There were also no queries referring to properties of events e.g. \interactive", \talks", \discussions" and no evidence of high-level, hedonistic qualities an event might bring about e.g. \fun", \exciting", \entertainment", etc.
In line with previous query analysis papers, we analysed the diversity of submitted queries. The cleaned query set contained 417 unique queries. As expected the distribution looks rather Zipf-like with the top 2 queries being \deutsches" and \deutsches Museum". The top 50 unique queries amount to 43.1% of all queries, the top 10 amount to 16.6% and the most common search term was used in 2.5% of all searches. The entropy of the unique search terms is 2.44 bits. The queries submitted were therefore far less diverse than web search queries on desktop or mobile devices. This can be partially explained by the fact that our collection is much smaller and much more speci c than the web. Another explanation for the more homogenous queries is the fact that most queries are event names which are usually only one or two words long. This reduces the possibilities for searching for these names when compared with the possibilities to express interest, constraints or needs in general.
In summary, our main observation is that the queries
submitted to the search system did not re ect the information
needs described in the pre-study interviews. It seems as
if the users did not use the search engine to discover new
events, but rather used the feature to lter to events they
already knew existed. Re ecting this, our queries have
similar properties to those reported for known-item searches in
web, email and desktop search, which have also been shown
to be very short and contain a high percentage of
namedentities [
We wanted to understand how successful queries were. With this in mind we de ned three success metrics based on the user's interaction with search results. The rst refers to whether the user selected a returned result to read a detailed description of the event. This metric is our equivalent to click-through data. 58.4% of all searches resulted in a click-through with an average of 0.73 clicks per query ( = 0:93) and 5.95 results on average ( = 9:10). We didn't consider good abandonment since the result list contains no information beyond name and nearest bus stop.
Two further, more explicit, de nitions of success were if the user marked a returned event as a candidate for tour inclusion (38.0% of all searches) or the user added the event to an preexisting tour (15.6% of all searches). These searches were performed at di erent stages of application use. Reecting this we derived a general success metric: in 59.7% of all searches at least one of these three actions was performed. Of the remaining 40.3% unsuccessful queries 59.8% were using a search term which resulted in an empty result list, in most cases a miss-spelled or only partial written named entity. The huge number of spelling errors underlines the need for fuzzy search methods in this application context.
As the queries that were submitted were very short, we wanted to investigate if the length of the query had any impact on the success of the search. Searches de ned as successful were on average longer with a mean of 1:26 terms ( = 0:57) compared to unsuccessful searches with a mean of 1:13 terms ( = 0:42); a highly signi cant di erence (p 0:01). Likewise the number of characters per query was signi cantly (p 0:01) longer with the successful searches having on average 9:90 characters ( = 5:42) and the unsuccessful searches having just 7:47 characters ( = 4:80). We implemented a search-as-you-type system which searches for whole words, however the evidence suggests that users used the system as a means to lter to events they already knew about. Therefore while entering the search term the result list is empty till you entered the complete word. This might have led users to the conclusion that their queries will be unsuccessful and abandon the search early. This would be one explanation for the shorter query length in unsuccessful searches.
An additional feature our app o ers is the possibility to submit spoken queries. Rather than typing search terms in using the keyboard, the user speaks the query into the phone. The system uses Google Speech Recognition to identify the query terms and the user selects the queries based on a list. This is familiar to android users as it is a standard feature for web search on Android phones. We wanted to establish how this feature was used, if queries submitted in this way di ered from typed queries and whether there was a notable di erence in performance between spoken and typed queries.
In total 22 app users submitted 68 spoken queries, which equates to 8.5% of all search queries. Of these 6 users used it more than three times. When comparing the length of the search queries we discovered that voice searches tend to be considerably longer than typed searches: 1.8 ( = 0:65) vs. 1.2 ( = 0:46) terms and 14.9 ( = 8:1) vs. 8.4 ( = 4:6) characters. Both comparisons6 are signi cant (p 0:01). It seems it is easier to create long queries with the voice interface than typing. The success rate is also signi cantly higher: 75% success for speech queries compared to 58.3% (p-value7: 0.01) success for typed queries.
It could be that the complicated input method when typing combined with the expectation of a ltering system might have tempted people to give up early, whereas spoken queries are always full words. This would explain the ratio of empty result list where 11.8% of the voice searches have an empty result list compared to 25.2% of non-voice searches; a difference which is signi cant (p-value7: 0.013). In summary, there is evidence to suggest that voice search can be an effective tool for entering search queries on a mobile device in leisure situations. There are, however, issues such as background noise and user self-consciousness that may explain why only a limited set of users used this functionality.
In this paper we analysed the query behaviour of users in a speci c casual-leisure situation: a mobile application to assist users at a distributed event. It was apparent when analysing the queries that there was a mismatch between the queries people submitted to the search system and what we anticipated based on the needs reported in the interviews. The overwhelming majority of queries were partial or complete event names, where the user was trying to lter to a speci c event. There were very few queries relating to topics that the user may be interested in e.g. \art", \history", etc. Furthermore there were no references to descriptors of events that people noted they wanted in interviews e.g. \interactive", \talks", \discussions". Likewise there was no evidence of the high-level, hedonistic qualities an event might bring about e.g. \fun", \entertainment", etc.
This poses the question: why are people using the search system in this way? Are people conditioned to do so, i.e. do they have a preconceived notion about how search engines work and only use the system in ways that re ects this? Or is it because the app has other features, such as browsing by tour or genre that might be better suited for tasks other than known-item search? To answer these questions we are currently analysing the log data for the other features of the system. A comparison with other casual-leisure search would also complement our understanding of this issue. Are there similar trends for search on YouTube, Wikipedia or the web? 6Wilcoxon sign rank test 7Two-Tailed Test of Population Proportion
Our analysis of query performance showed that a high number of spelling mistakes were made. We wonder if this is caused by environmental factors, e.g. typing on a bumpy bus or if it is caused by a high number of named entities, the spelling of which people are not familiar? Further research would be needed to di erentiate between the two, however a fuzzy search feature would certainly help people who struggle with the query input. A grep-style search would further reduce this problem since users would only need to enter a few characters as opposed to whole terms. In the comparison of spoken vs. typed queries we have seen that although not used much it provides a more successful way of querying the system.
We also believe that voice-queries deserve further research. The reason behind the decision for typing or speaking a query is di cult to analyse based on the logged data. Perhaps users are shy of speaking to their smartphone in the public. Further studies would be necessary to gain a proper insight into this behaviour. The information obtained from this early study points to a number of potential avenues for further research. One plan we have is to look at di erent usage patterns with the system and see how they correlate with the outcomes of the evening e.g. number of events visited, the ratings of visit events, the geographical coverage of the user etc. This would provide insight into how the features of our system support casual-leisure needs. Acknowledgments This work was supported by the Embedded Systems Initiative (http://www.esi-anwendungszentrum.de).