Evaluating the Cognitive Impact of Search User Interface Design Decisions Max L. Wilson Future Interaction Technology Labs Department of Computer Science, College of Science Swansea University, UK m.l.wilson@swansea.ac.uk ABSTRACT highlighted options in unused filters that were related to The design of search user interfaces has developed guide searchers [10]. Frequently, however, we informally dramatically over the years, from simple keyword search noted that searchers spent increasing periods of time on systems to complex combinations of faceted filters and visually comprehending the interface before making their sorting mechanisms. These complicated interactions can first move. In follow up studies, we saw minimal provide the searcher with a lot of power and control, but at interaction with facets during the first visit, but recorded a what cost? Our own work has seen users experience a sharp significant increase in the use of faceted features during learning curve with faceted browsers, even before they subsequent return visits. It is the hypothesis of our begin interacting. This paper describes a forthcoming forthcoming work that this non-use of such powerful period of work that intends to investigate the cognitive features is caused by an increased cognitive load created by impact of incrementally adding features to search user the associated increased complexity of the SUI. It is this interfaces. We intend to produce search user interface cognitive impact that we believe can be measured and design recommendations to help designers maximize attributed to specific design decisions. support for searchers while minimizing cognitive impact. mSpace is one specific faceted browser, but the principle of Author Keywords faceted browsing can be implemented in many different Search, Exploratory Search, User Interface Design, ways [2]. We also hypothesize that not only the presence, Cognitive Load Theory but also the subsequent design of SUI features can also have an impact. The following sections cover some related ACM Classification Keywords work before describing our plans to evaluate the cognitive H5.2. Information interfaces and presentation (User impact that adding features to SUIs can have. Interfaces): evaluation/methodology, screen design. H3.3. Information search and retrieval: Search process. RELATED WORK SUI design is affected by many factors. Interaction INTRODUCTION designers can decide how best to support searchers, but User Interface (UI) Designers are always concerned with designs may be limited by the metadata that is available supporting users effectively and intuitively, but a common about the possible results. Both the underlying data and the recent focus for Search User Interface (SUI) designers has graphical design may also have an impact, then, on how the been to increase the interactive power and control that chosen interaction will look and feel. As perhaps the most searchers have over results. As a community, we want to recognized SUI for many users around the world, Google support users in exploring, discovering, comparing, and has always maintained a very clean and clear white design1, choosing results that meet their needs. SUI designers, and make very incremental careful design changes that stay therefore, are concerned with maximizing the use of within that design. Competitor search engines have notably powerful interface features while maintaining a clear and changed over the years, with many now being very similar intuitive design. to Google in terms of interaction design, while trying to keep their own visual design consistent. In our prior work, we developed mSpace [7] as a faceted browser that lets searchers use combinations of orthogonal For more exploratory websites that sell a wide range of metadata filters to narrow their search. We developed products, or provide large collections of information or advanced interactions for faceted browsers that took documents, there are now many different features that advantage of visual location within the SUI, and support people, from tabular or dropdown-based sorting Copyright © 2011 for the individual papers by the papers' authors. 1 Copying permitted only for private and academic purposes. This volume is http://searchengineland.com/qa-with-marissa-mayer- published and copyrighted by the editors of euroHCIR2011. google-vp-search-products-user-experience-10370 mechanisms, to categories, clusters, filters, and facets. where two systems provide the same support, one may be Some websites that provide these features are frustrating harder or easier to use because of its simple visual design. and difficult to use, while others are simple, intuitive, and Our conclusion is that to understand the success of a SUI, successful. In these systems it is often the way that the ideal we must analyse both the support in terms of functionality, support has been developed that has affected their success. and the cognitive impact is creates. Being able to In a study of the success of different faceted browser understand and predict these two things would help us to implementations, Capra et al [1] directly compared two design and build better SUIs faceted browsers to a government website, all over the same hierarchical government dataset, and discovered that the EVALUATING THE SUPPORT PROVIDED BY SUIS customized hierarchical design of the original website Beyond the common practice of performing task-oriented supported searchers far better than the functionally more user studies, my own doctoral work focused on the design powerful faceted browsers. of an analytical evaluation metric for SUIs, called the Search Interface Inspector2 (Sii). Sii calculates the support Both the choice of content and the visual design have both for different types of users based upon the set of features in been shown to have an impact on usability. White et al the interface, and how many interactions they take to use showed that the text that includes the search terms is best, [9]. To analyse a SUI, the evaluator catalogues the features and that highlighting these terms also improves search [12]. of the design and calculates how many interactions are Similarly, Lin et al. have shown that simply highlighting required to perform a set of known search tactics. The the domain name in the URL bar significantly reduces the method then interpolates the likely support for different chances that users will be caught be fishing attacks [4]. types of searchers (explorers or searchers that know what Zheng et al [13] have also shown that users can make often- they are looking for, for example), based upon the types of accurate snap judgments about the credibility of websites tactics they are likely to perform. Sii can be used to within half a second. Further, Wilson et al [10] noted that compare several designs and produces a series of 3 the success of adding guiding highlights to their faceted interactive graphs that allow evaluators perform an browser was affected by the choice of highlight-colour and investigative analysis of the results. its implied meaning. Sii is based on detailed established information seeking The choice of SUI features within a single implementation theory and rewards the design of search functionality that has also been shown to have an impact on search success. has simple interaction. Consequently, however, Sii rewards Diriye et al compared a keyword search interface with a the addition of new simple functionality, without being able revised version that also included query suggestions [3]. to estimate the increasing complexity of the SUI as new Their results showed that such features slowed down features are added. To remedy this problem, a chapter of the searchers who were performing simple lookup tasks, but thesis investigated Cognitive Load Theory and initially supported those who were performing more complicated specified a similar metric that calculated the cognitive load exploratory tasks. Similarly, Wilson and Wilson have also of a UI. This second measure of intrinsic cognitive load was found early results indicating that the simple presence, proposed for inclusion in Sii, estimated the intrinsic without interaction, of a keyword cloud provides additional cognitive load of a SUI. Similar to how the original metric support, where subsequent interaction provides very little was correlated with study results, one aim of the work gain [11] during exploratory tasks. Wilson and Wilson’s described below is to further refine and validate this results suggest that searchers can learn more about the analytical measure of the cognitive impact of SUIs. result set from seeing the terms in the keyword cloud, than actually using them to filter the results. Cognitive Load Theory highlights that capacity for learning is affected by three aspects: intrinsic, extrinsic, and The location of features within a SUI has also been shown germane cognitive load. Intrinsic cognitive load is created to have an impact. Morgan and Wilson studied the visual by the materials providing the learning experience, or in our layout of search thumbnails, predicting that having a rack of case the SUI. Extrinsic cognitive load is created by the thumbnails at the top of the user interface would allow complexity in the task at hand. Germane cognitive load is searchers to make faster judgments when trying to re-find then required to process what is learned and commit it to pages [5]. Their results showed that a rack of thumbnails long-term memory. If intrinsic load and extrinsic load are was significantly more disruptive to searchers when the too high, then there may not be enough space load left for target page was not in the results, than the support it germane cognitive load. Although, it is commonly accepted provided when it was. that effort can increase overall capacity, the aim should still The studies above indicate that the success of SUIs can be be to reduce intrinsic cognitive load by improving the attributed to the appropriateness of the functionality design of learning materials or SUIs [6]. Reducing intrinsic provided, where unnecessary functionality can slow users load creates space for users to perform increasingly down. Further, the studies indicate that the success of SUIs can be determined by simple visual or spatial changes that do not necessarily impact functionality. Consequently, 2 http://mspace.fm/sii complex tasks, or opens-up germane cognitive load so that turn help us make hypotheses about design issues. This what is being learned can be retained. phase will help us identify the cost of adding a feature, where task success would allow us to measure their benefit. EVALUATING THE COGNITIVE IMPACT OF SUIS The general structure of the studies we are planning is to Phase 2 – capturing impact in the context of tasks use brain scanners to record the cognitive impact that Where the first phase above allows us to learn to recognize different SUIs have on a user. The initial phases will focus the signs from EEG signals, we intend to try and detect on identifying and measuring such responses to significant cognitive load in situ, and in the context of a task. We will and obvious differences, before trying to capture changes to be setting participants specific simple and exploratory tasks, more subtle designs and, hopefully, in-situ. Initially, we whilst controlling the type of user interface features they will be using EPOC Emotiv headsets3, as shown in Figure see, to capture the cognitive impact as they start. This phase 1, to take readings. These headsets are commercialized will help us identify whether the impact of a search user versions of EEG scanners, but are designed for use in more interface is affected by task context. natural contexts. EEG scanners, as with many other brain scanning systems, are typically affected by simple body Phase 3 – the impact of different implementations movements and so are often restricted to confined While adding features creates an obvious change in the user conditions. Such scanners, therefore, are often not suitable interface, different features can be put in different places in for task-based evaluations, which require action and the SUI and also be implemented differently. Google, for movement. In psychology, EEG scanners are typically used example, puts suggested refinements at the bottom of the in constrained environments where users are only allowed page, while Bing has them on the side. Bing also chooses to to move their thumbs to answer yes or no. Consequently, provide a mix of refinements and alternative directions. In this work requires scanners that can be used in more natural Phase 2 we intend to analyse both of these kinds of contexts while performing everyday searching tasks. In the variables to see if they have significant impacts on future, funding permitting, we also intend to buy an fNIR cognitive load. This phase will help us identify whether the scanner, which has been shown to be suitable for task-based cost of adding SUI features can be minimized by refining evaluation conditions [8]. We intend to use these their design. measurements to understand the impact of design decisions, in order to make clear recommendations to SUI designers. Discussion There are many challenges remaining in this planned work. So far, we have planned very controlled comparisons of SUI changes, but in real life these systems are used in the context of complex tasks and for extended periods of time. Controlled situations will help identify cause and effect, but other similar objective measurements, like eye trackers, still require interpretation. We hope to expand on these methods, and the findings of existing brain scanning HCI research [8], by addressing this issue over time. Finally, although this research is primarily interested in the development of SUI interfaces and how they affect people learning to use powerful search features, there are many other things that can be distracting in general UI design. These methods will likely expand to help address other Figure 1: EPOC Emotiv Headset design questions; we, however, are particularly aiming to answer questions about encouraging exploratory search and Phase 1 – the impact of additional features learning, by increasing the power of SUIs, while reducing Beginning this summer, with two summer interns, we will their impact on searchers. be performing our first studies, which will simply display SUIs of incremental complexity to participants. We will CONCLUSIONS begin with a simple keyword search design, and add This work has yet to begin formally, but we intend to learn features such as recommendations and filters. The order more about the impact that very simple design decisions that interfaces are shown to participants will be randomized can have on searchers. From previous experience of to avoid learning and familiarity bias. The aim of this phase searcher success in evaluations, both industry and academia is to prove that the learning curves experienced by users know that such changes can seriously impact the success of exist and the cognitive load can be measured objectively. a search user interface. This work will use objective We hope that the results will show initial insight into the measurements of brain response to help us identify the amount of impact that different features have, which may in factors that make search user interfaces hard to comprehend. We hope that such measurements will a) help 3 http://www.emotiv.com/ us analyse the cost-benefit trade-off of adding additional support to search user interfaces, and b) help us develop multimodal exploratory search. Commun. ACM 49, 4 design recommendations for implementing search user (April 2006), 47-49. interface features so that they have minimal impact. 8. Erin Treacy Solovey, Audrey Girouard, Krysta Chauncey, Leanne M. Hirshfield, Angelo Sassaroli, REFERENCES Feng Zheng, Sergio Fantini, and Robert J.K. Jacob. 1. Robert Capra, Gary Marchionini, Jung Sun Oh, Fred 2009. Using fNIRS brain sensing in realistic HCI Stutzman, and Yan Zhang. 2007. Effects of structure settings: experiments and guidelines. In Proc. UIST and interaction style on distinct search tasks. In Proc. '09. ACM, New York, NY, USA, 157-166. JCDL '07. ACM, New York, NY, USA, 442-451. 9. Max L. Wilson, M. C. schraefel, and Ryen W. White. 2. Edward C. Clarkson, Shamkant B. Navathe, and 2009. Evaluating advanced search interfaces using James D. Foley. 2009. Generalized formal models for established information-seeking models. J. Am. Soc. faceted user interfaces. In Proc. JCDL '09. ACM, New Inf. Sci. Technol. 60, 7 (July 2009), 1407-1422. York, NY, USA, 125-134. 10. Max L. Wilson, Paul André, and mc schraefel. 2008. 3. Abdigani Diriye, Ann Blandford, and Anastasios Backward highlighting: enhancing faceted search. Tombros. 2010. Exploring the impact of search In Proc UIST '08. ACM, New York, NY, USA, 235- interface features on search tasks. In Proc. ECDL'10. 238 4. Eric Lin, Saul Greenberg, Eileah Trotter, David Ma, 11. Wilson, M. J. and Wilson, M. L. Tag Clouds and John Aycock. Does Domain Highlighting Help People Keyword Clouds: evaluating zero-interaction benefits. Identify Phishing Sites. In Proc. CHI2011 (in press). In Ext. Abstract CHI’11. 5. Rhys Morgan and Max L. Wilson. 2010. The Revisit 12. Ryen W. White, Ian Ruthven, and Joemon M. Jose. Rack: grouping web search thumbnails for optimal 2002. Finding relevant documents using top ranking visual recognition. In Proc. ASIS&T '10. sentences: an evaluation of two alternative schemes. 6. Sharon Oviatt. 2006. Human-centered design meets In Proc. SIGIR '02. ACM, New York, NY, USA, 57- cognitive load theory: designing interfaces that help 64. people think. In Proc. MULTIMEDIA'06. ACM, New 13. Xianjun Sam Zheng, Ishani Chakraborty, James Jeng- York, NY, USA, 871-880. Weei Lin, and Robert Rauschenberger. 2009. 7. m.c. schraefel, Max Wilson, Alistair Russell, and Correlating low-level image statistics with users - Daniel A. Smith. 2006. mSpace: improving rapid aesthetic and affective judgments of web pages. information access to multimedia domains with In Proc. CHI '09. ACM, New York, NY, USA, 1-10.