People searching for books make use of various sources and systems. While some users prefer online systems, others tend to visit bookstores in their neighborhood. This study compares book search sessions from the iSBS track with observed bookstore search sessions with the aim to investigate similarities and di erences in user behavior and preferences. In particular, the focus lies on the observation and comparison of searching, browsing and recommending strategies. The results indicate signi cant di erences in session duration as well as a strong preference for browsing strategies in bookstore sessions. No such strong preference for one strategy could be observed within the online sessions, even though browsing was still preferred. Recommendations are the least preferred interaction type in either environment.
How do we search for books? What makes us select a particular book? Both
o ine [
Users discover books in various ways that are not always covered by simple search functionalities. Social book search applications like Goodreads1 have found that recommendations from online and o ine friends play an important role discovering new books. Often, we only search for a particular book 1 https://www.goodreads.com/ based on personal recommendations2. Recommender systems try to adopt these relationship-based access strategies by suggesting relevant objects to a similar user group. However, information systems still often fail in supporting the users in discovering new or unknown content, especially in casual leisure situations. In order to improve these systems, we need to better understand user strategies and preferences and translate them into purposeful features. A common analysis approach is to compare strategies and interactions in the digital environment with those that occur in a similar physical environment. Since the physical environment (in this particular case bookstores) usually precedes the development of digital environments, processes and strategies for interactions in the physical environment have already stabilized and experiences can be translated into patterns for digital information system development. It is a matter of course that digital environments should o er di erent (usually more e cient or innovative) user experiences than the physical ones. But starting with a familiar user experience and expanding from that with the additional features that the digital environment allows, is a frequent user experience design strategy. This study compares a physical (i.e. neighborhood) and a digital (i.e. hypertext) book discovery environment in order to determine where each environment can learn from the other.
In particular, the study analyzes di erent types of interactions that were
already found and discussed in casual search sessions: searching (the user issues
a particular request), browsing (the user overviews a variety of resources), and
following recommendations (the user follows a previously issued reference)[
Within this research context, this analysis aims at addressing two main questions: RQ1 Are searching, browsing and recommendation strategies and preferences (equally) observable in online and o ine book search sessions? RQ2 Are there di erences between online (digital book discovery environment) and o ine (neighborhood bookstore) sessions? In section 2, the Interactive Social Book Search track is described as well as the participants and data gathering approach. Section 3 reports on the bookstore observations. The comparison of online and o ine sessions is given in Section 4. The paper concludes with a discussion on the results and the approach. 2
Experimental Setup. The Interactive Social Book Search track investigates book
search sessions in order to nd out how users interact with traditional metadata
as well as user-generated content in book search applications [
In order to provide a realistic book discovery environment, the underlying
book collection should be large and comparable to other book discovery systems
such as online book sellers or library resource discovery systems. A monolingual
English subset of the INEX Social Book Search's Amazon/LibraryThing book
collection is used, consisting of approximately 1.5 million books. Each book
contains general metadata (title, author(s), publisher, publication year, etc.),
subject metadata (classi cation codes, subject headings), user-generated content
(Amazon user reviews, LibraryThing user tags), and a thumbnail image[
In contrast to previous years, this year's observation environment only
contained the multistage interface of the ISBS book discovery information system[
For an observation experiment, participants are rst given a training task that introduces them to the interface. Following, users were asked to complete either an open or closed task, after which they could also work on a second optional task if they so chose. For this analysis, only sessions including the following mandatory open task were considered:
Imagine you are waiting to meet a friend in a co ee shop or pub or the airport or your o ce. While waiting, you come across this website and explore it looking for any book that you nd interesting, or engaging or relevant. Explore anything you wish until you are completely and utterly bored. When you nd something interesting, add it to the bookbag. Please add a note (in the book-bag) explaining why you selected each of the books.
The open tasks sessions are most likely comparable to individual bookstore sessions and have therefore been chosen for further analysis.
Participants and Data Gathering. Usage data was gathered through log les as well as user background information and responses through questionnaires. A total of 111 users participated in this year's experiment with an almost balanced representation of female (46%) and male (54%) users. The majority of sessions (67%) occurred in a lab and most participants decided to view and conduct an additional task (80%). In total, 55 sessions contained an open task and were extracted for further analysis. Only the rst task was considered. 3
Natural Setting. For the bookstore observations, a small bookstore in Berlin, Germany was chosen, o ering roughly 4.000 items. The small store is housed in a single room with books grouped by di erent categories and one central information desk. The observer was placed near the information desk at a position where it is possible to view the customers in the store as well as listening to the customer service dialogues. The observations took place during two days (one weekday and one day in the weekend) in order to capture di erent user groups and situations. Based on the proposed research question, an observation protocol was designed focusing on duration, strategies and results for each session. The session duration represents the period between entering and leaving the store for each observed person. Interactions were assigned to the search, browsing or recommendation categories and if necessary session details listed. A session outcome could be a purchase or ordering of unavailable books. Additionally, gender information and, if possible, information needs were documented. Participants and Data Gathering. In total, 39 customers in 37 sessions were observed. The vast majority of them were females (74%). Comparing weekday and weekend observations, it is apparent that people tend to visit bookstores alone during the week and with their family and especially with their children during the weekend. However, the session duration did not increase signi cantly. A large amount of information needs were related to other people the customers where shopping for: either family members or friends.
Besides the obvious di erences in environments (online and o ine) and the number of observed sessions (55 in the online and 37 o ine environment), the two set-ups also di ered in their collections, their "interfaces"3 and the types of available interactions.
It is very common that customers approach the information desk to inquire about books that are not available in the stock of the bookstore, but can be ordered to be delivered within a 24 hour time-span. In one interpretation, the physical bookstore added a digital dimension here, which is not reciprocated by the ISBS interface.
While the ISBS discovery interface does not immediately reveal the size of its collection, a bookstore customer can quickly get an overview over the number and di erent categories of books available due to the physical representation.
Additionally, the ISBS online interface utilized an automatic algorithm based on the book records in the collection and the book reviews for recommendation purposes, while the bookstore used a personalized, human recommender (the book sellers) who based their recommendations not only on the stock, but also on other experiences. The subsequent analysis will show whether these di erences actually lead to di erent interactions. 4
In this section, the 55 iSBS sessions are compared to the 37 bookstore sessions with respect to their session duration and interaction type preferences. 3 In a bookstore, one would probably speak of layout and representation of the stock. Session Duration. Table 1 shows the average duration of sessions for each data set. It shows that online sessions last almost double as long as bookstore sessions. The longest online task completion took 57 minutes, while the longest bookstore session was 34 minutes long.
One reason for the longer online sessions might be the experimental environment as well as the unknown interface and system, which led to longer task completion times because of unfamiliar complexity. Another juxtaposed reason might be that participants in the ISBS lab study spent more time with the system because they were more attuned to the fact that they were being observed, which led to longer task completion times because of increased carefulness and attention to the task. A third reason might be that task completion in bookstores is faster because of the reduced selection variety, which could be interpreted as both a positive or negative impact factor.
Interaction Type Preferences. For this analysis, characteristics indicating search, browsing or recommendation sessions were chosen: Search sessions are either de ned by the amount of queries issued as well as the usefulness rating of the ISBS search box by a study participant (i.e. did they use the search box and nd it useful or not) or through a speci c request by a bookstore customer.
Browse sessions are either de ned by browsing interactions through topics as well as the ratings of the hierarchical topic explorer o ered for browsing by ISBS or through book(shelf) browsing by a bookstore customer. Recommendation sessions are either de ned by the usage and rating of similar books provided at the exploratory bookbag stage or through a recommendation o ered by the book seller.
Table 2 compares search strategies from both environments belonging to either search, browsing or recommendation interactions. With 419 single interactions iSBS sessions contain on average 7,6 topic re nements. Following, 166 queries were issued during the open task, averages at least 3 queries per session. Only 32 times in 12 sessions the similar book feature was used. Although almost all sessions included at least one book saving it remains unclear why the majority of users did not use the recommendation possibility at the bookbag stage. Those that made use of this function reported di ering evaluations for the presented results (see table 3). Similar, browsing activities are predominately observed in the bookstore. At the second place customers asked a book seller for a speci c item, i.e. they performed a search. Only 7 sessions contained an interaction related to recommendations. Mostly, these recommendations were given by the book seller, only in one case a man called his wife and asked for advice while searching for a gift.
Table 3 provides an overview of participants responses for how useful the above mentioned interface elements were on a scale between 1 (Not at all) and 5 ("Extremely").
Consistently to the usage data, only 5 participants indicated that they did not use the hierarchical topic explorer at the browsing stage, which indicates that browsing was used by more participants than the other interaction types. In comparison, the search box and similar books were not used by 24% and 42% of the participants. Although a quarter of the participants did not use the search interaction, when used, most participants found the possibility to query for a book extremely useful. 16 participants responded the same for the topic explorer and only 9 for the similar books, indicating that recommendations were not as helpful as search or browsing.
Browsing seems to be the preferred interaction type in both environments, however, it is not considered as useful as searching in the online environment. This might be due to the sub-optimal browsing possibilities in the online environment. The instant and - at the same time - complete overview that a bookstore a ords, cannot be easily replicated here. Recommendations are the least preferred interaction type in either environment, a surprising nding, considering how much the leading bookseller Amazon focuses on exactly this interaction type. However, this needs to be interpreted carefully. The browsing capabilities of both environments already provide useful indicators for their users, so an additional recommendation might simply not be necessary. Conversely, the recommendations in both study environments could also be not as good as might be expected from a book discovery environment (i.e. as in Amazon) and users learned to not use this as much.
These questions once again demonstrate that even carefully designed interactive studies as the ISBS one only lead to more research questions in need to be explored.
Success Rates of Sessions. An interesting follow-up analysis of interaction type preferences would be to compare preferences with their impact on the success of an interaction. The vast majority of bookstore sessions were successful with 78 % of all sessions ending with a purchase or ordering of a book. Success in the online sessions could be de ned through the saving of a book in the bookbag. In total, 201 books were saved in 55 sessions during the open task. While participants saved 3.7 books on average, individual session results ranged from one to 18 saved books. However, since the experimental tasks required at least one book saving, this is no reliable measure. Also, there is a decided di erence between storing books in a digital environment, which has no consequences and the completion of a book buying transaction, which demonstrates a much higher commitment of the buyer. Therefore success rate between online and o ine sessions were not taken into account. 5
This small study of online and o ine book search sessions could point out some similarities in the environments, but also some di erences. The results indicate signi cant di erences in session duration as well as a strong preference for browsing strategies in bookstore sessions. No such strong preference for one strategy could be observed within the online sessions, even though browsing was still preferred.
At the same time, both methods used for data gathering come with limitations. While interactive retrieval experiments provide a more controlled environment and details about the participating users, it remains an experiment that is somehow limited to a certain user group and situation. Unobtrusive observations in the wild (i.e. a bookstore) provide a more realistic picture, but lack context information both about the user and environmental factors. For example, users that look at bookshelves can either want to locate a speci c book which would be a classic search or browse through the available content.
It remains unclear to which extent individual and environmental di erences in uences the observed behavior, a research question that remains open for user studies. It makes sense to combine di erent methods and sources to support the translation of casual leisure situations into information system architectures as the ISBS experiments attempt. For example, the o ine study produced a set of information needs that could be used for future experiments. Also, the observation has shown, that bookstore sessions are rather short in duration. However, we do not know how much time people spend in advance to inform themselves about or locate interesting books. It seems that book search is rather a multiple-stage than a single session a air. Especially for personalization features and session time outs this could be an important aspect to consider.
Acknowledgments We would like to thank Manuela Wiggert for providing access to her bookstore Albertinen Buchhandlung as well as valuable insight into a physical bookstore environment. European Workshop on Human-Computer Interaction and Information Retrieval, Dublin, Ireland, August 1, 2013 (2013), pp. 19{22. 11. Zhang, Y., and Kudva, S. E-books versus print books: Readers' choices and preferences across contexts. JASIST 65, 8 (2014), 1695{1706.