SearchPanel: A browser extension for managing search activity Simon Tretter Gene Golovchinsky Pernilla Qvarfordt University of Amsterdam FX Palo Alto Laboratory, Inc. FX Palo Alto Laboratory, Inc. Amsterdam, The Netherlands 3174 Porter Drive 3174 Porter Drive Palo Alto, CA Palo Alto, CA s.tretter@gmail.com gene@fxpal.com pernilla@fxpal.com ABSTRACT documents in relation to the searcher’s activity: how many People often use more than one query when searching for times was a document retrieved, whether it was viewed be- information; they also revisit search results to re-find infor- fore, etc. This kind of information can help searchers to mation. These tasks are not well-supported by search inter- remember, understand and plan their search processes. faces and web browsers. We designed and built a Chrome The browser plugin enhances the searcher’s ability to use browser extension that helps people manage their ongoing process metadata to understand their search results and to information seeking. The extension combines document and plan subsequent activity by displaying surrogates for the process metadata into an interactive representation of the current set of retrieved documents. We represent prior re- retrieved documents that can be used for sense-making, for trieval state, whether a document was opened, and whether navigation, and for re-finding documents. it was bookmarked in an integrated overview that appears at the side of the browser window. We also make it pos- sible for searchers to examine multiple documents without 1. INTRODUCTION returning to the search results or using multiple tabs. Broder et al. [3] proposed a taxonomy of web search that The remainder of this paper is organized as follows: we included transactional and navigational searches in addition review the relevant related work, describe the browser ex- to the more traditional (from an IR perspective) informa- tension, and conclude with a discussion of the design space. tional searches. To this taxonomy we might add re-finding [17] [5], the task of locating a previously-found document. From a theoretical perspective, it is not clear whether refind- 2. RELATED WORK ing is a different kind of search activity or an orthogonal di- There are two broad categories of related work: the man- mensions. Regardless, while major web search engines offer agement of search history and the representation of search simple and efficient interfaces for navigational and transac- results. Refinding has received increasing attention recently. tional searches, relatively little support is available for more While the browser implements some history mechanisms, complex informational search or re-finding. these are typically not well-suited to users’ needs [15]. El- These seemingly neglected activities are not unimportant, sweiler and Ruthven [5] described different patterns of re- however: Teevan et al. [17] reported that 39% of queries are finding; Teevan [16] proposed a mechanism for merging pre- re-finding queries; furthermore, 20-30% of searches represent viously-found and newly-retrieved documents. More explicit open-ended informational needs [13]. Related, Qvarfordt et management of search history has also been investigated in al. [11] found query overlap rates of 50-60% in exploratory the literature; see [7] for a succinct summary. search, and suggested that awareness of this overlap may be Information overload due to large numbers of results is useful in supporting more efficient searching behavior. Thus a common problem in information seeking [2]. This prob- we decided to explore ways in which searchers’ interactions lem can be addressed in a variety of ways. MetaSpider with search engines could be enhanced to support these more [4] uses a 2D map to display and classify retrieved doc- complex information-seeking tasks. uments. Grokker [8] uses nested circular and rectangular We created a web browser extension that enriches com- shapes to present results and also shows them in a hier- mon web search engine interfaces and addresses important arachical grouped way. Sparkler [12] uses a star plot for the deficits with respect to open-ended (exploratory) search and result presentation, where every star represents a document. re-finding. Our extension visualizes search results to help One potential issue with the systems above is that the users find the right document or documents by visualizing overall organization of the interface itself may induce us- metadata of the retrieved pages. ability problems. Complex interfaces allow more individual Following Golovchinsky et al. [7] we distinguish docu- settings to be specified by a user, but simple interfaces allow ment metadata from process metadata. Document metadata a broader spectrum of users to use them. This tradeoff is – dates of publication, titles, hosting web sites, etc. – are not trivial to handle, and as we see nowadays, most Web basic characteristics of documents that are independent of search interfaces tend to be quite simple. the means by which these documents were retrieved. Pro- Supporting the searcher’s decision making process can be cess metadata, on the other hand, characterize aspects of crucial for effective search performance for complex infor- mation needs. This support can take the form of enhanced Presented at EuroHCIR2013. Copyright c 2013 for the individual papers by the papers’ authors. Copying permitted only for private and academic surrogates for documents. One type of information often purposes. This volume is published and copyrighted by its editors. used for this purpose is document metadata (author, date, images of the document, etc.). Even et al. [6] has shown Table 1: Design space: Activities and supporting features that the decision making process can be highly improved by related to document and process metadata. ”Doc” refers to adding process metadata (in our case information that is re- document metadata and ”Proc” to process metadata. lated to the search process) to the user interface. Research has shown that presenting simple tasks in a slightly differ- Activity Feature Doc Proc ent way may help the user to understand how the search Search perform search yes no is performing and what can be done to gain better results switch engine no yes [18]. One common example of incorporating process meta- results list yes no data in web browsers is the practice of changing the color of visit status no yes a traversed link anchor. visualize no. of visits no yes Spoerri [14] showed that users can benefit from different Navigation access results - - or additional visualizations of web search results. However, mark current result path no yes none of the techniques above have been integrated by major identify results: preview yes no search engines into their main interfaces. In some cases, ex- snippet tension developers have enhanced the user experience of web identify results: favicon yes no search. Examples include: SearchPreview[9] that fetches Organization bookmarking no yes screen shots of the result pages and shows them directly organize bookmarks no yes next to the each search result. Bettersearch[1] is a Firefox extension that performs a similar task, but also enriches the When searchers find useful web pages, they may wish to result page with more features and links. For example, this save those documents for future access. More specialized extention allows users to open a result in a new tab, or adds search engines sometimes support this capability directly, links to a search result to quickly show the web page on the but it is most often supported only by the browser’s book- ”Wayback Machine”1 . WebSearch Pro [10] is also a Firefox marking capability. extension that adds the ability to look up a text by high- We can consider these search and sense-making activities lighting it on a page. Another feature is drag&drop zones in light of the kinds of information required to satisfy them. to search for things directly from any website. In particular, Table 1 shows when document and process metadata might be pertinent for the different categories of search activities. A representation of the number of visits 3. BROWSER EXTENSION to a retrieved result (process metadata) could be used by a To compensate for the deficiencies of SERPs we created a searcher to decide how to interact with that result. In a re- browser extension called SearchPanel. This extension com- finding sub task, for example, searchers might want to ignore bines document and process metadata in a visual represen- newly-found documents or pages that were not opened. tation of search results to help people manage their infor- The purpose of the search panel is to complement the mation seeking. We chose the browser extension approach SERP and to be available when exploring search results; we rather than creating a proxy for several reasons. While both wanted the design to be simple and unobtrusive but still offer the potential of parsing and augmenting SERP and convey useful information. Some features (e.g., organizat- document pages, a browser extension has some advantages. ing bookmarks) listed in Table 1 are too complex to be in- It scales better with respect to storing user history data. It tegrated into the extension. Others, such as favicons, while ensures a higher level of data privacy, since data that might seemingly trivial, may still provide useful information for potentially reveal user interests (e.g., query keywords, se- navigating search results. lected URLs, etc.) can be logged as hashed values. Finally, it has access to bookmarks and local browsing history. 3.2 Implementation 3.1 Design space SearchPanel displays automatically on the right side of the browser window when it is enabled (Figure 1). The right side When performing search tasks, searchers may need differ- of the content page has been chosen because this location is ent kinds of information to support their information seek- frequently free of document content. In cases of overlap, its ing. We represent the design space as consisting of three vertical position can be adjusted manually to accommodate categories of activities: search activity, navigation activity, page content that may be occluded. and organization activity. SearchPanel displays immediately after a search has been Historically, web UI support for the search process, or performed on a supported web search engine (currently, they search activity, has been focused on query formulation and are Google, Google Scholar, Yahoo, Bing and Microsoft Aca- understanding the current query. Web browsers offer lim- demic Search). SearchPanel remains visible even if the sear- ited support for comparing current results set with earlier cher follows links from retrieved documents. In addition, activity by marking the visited status of documents. searchers can return directly to the original query, or re-run When engaged with a search task, users need to shift their it on a different search engine. attention between the SERP and the retrieved pages. In A short tutorial page is displayed at installation, and can some cases, the searcher does not find the desired informa- also be reached through the option menu. This page also tion in a retrieved document, but rather in links to other allows logging (see 3.2.4) to be disabled, and can be used to documents containing relevant information. This naviga- delete the recorded history. tion activity can be an important part of the information seeking process. 3.2.1 Document metadata 1 SearchPanel displays several kinds of document metadata. The Wayback Machine is a service that provides access to archived and historical versions of web sites. Documents are represented by bars arranged in order corre- Figure 2: Highlighting of snippet on the SERP when mous- ing over SearchPanel. Figure 3: Snippets of other pages are shown on a document Figure 1: SearchPanel control annotated to show impor- page when mousing over SearchPanel. tant aspects. 1 search engine selector; 2 bar representing a newly-found page; 3 favicon representing the site from the star to bookmark the corresponding page. Second, pre- viously bookmarked documents in the SERP will show a which the page was retrieved; 4 bar representing page that yellow star next to them. This allows to re-find a web page has been visited; 5 highlighted bar based on cursor posi- quicker, as the user does not need to navigate to a document tion; 6 bookmark indicator; 7 currently-selected page. to know if they have previously bookmarked it. sponding to the retrieved list; clicking on a bar is equivalent 3.2.3 Navigational support to clicking on a link on the SERP. Almost all websites have The selection indicator (see item 7 in Figure 1) indicates icons (favicons) to help re-identify the web page quickly; the currently-selected result page. If a link on a result page these icons are shown to the right of the bar (see Figure 1, is clicked, the page indicator will stay on the last retrieved item 3 ). A tooltip with the title of the document is added document page to indicate that navigation started with it. to each bar as well. We considered identifying other meta- Hovering over the result highlights the associated bar (item data such as document MIME type, but that would incur 5 ), and also highlights the corresponding snippet in the the overhead of a separate HTTP request for each document. SERP (Figure 2); the SERP is scrolled as necessary to bring At least initially, we chose not to pursue this strategy. highlighted snippet into view. Conversely, when the mouse is over a snippet on the SERP, the related bar jiggles left- 3.2.2 Process metadata right to reinforce the connection between the two. Process metadata is also incorporated into SearchPanel. When the user navigates off the SERP to a search re- First, the icon of the search engine that ran the search is sult, SearchPanel remains active. Clicking on bars navigates highlighted in the top bar (item 1 ). Other icons repre- among the retrieved documents, bypassing the intermediate sent available comparable search engines. Clicking on one step of reloading the search results. When the mouse is over of these icons re-runs the query with the selected search en- a bar in SearchPanel, the SERP snippet of that result will gine. Search engines are grouped into two categories (web be shown. This can be seen in Figure 3, where a preview search and academic research) and only the relevant ones are of the Wolfram Alpha snippet is shown. If the snippet is shown. The current selection (highlighted with a black bor- not available, a tooltip with the document title is shown der) links back to the search result page if the user navigates instead. Both of these features should make it easier and to one of the retrieved documents. more efficient to navigate the search results without neces- Each bar can have one of three different colors, depending sarily creating a large number of tabs in the process. on the link history. If a link has never been retrieved before, the state of the link is ”new” and the color will be teal. Re- 3.2.4 Logging sults that have been retrieved by prior queries but have not The extension was created to study people’s information been clicked on are colored blue. Visited links are colored seeking behaviors. The goal of the project is to understand violet. The local browser history is examined to retrieve the how people use the web when looking for information to link status. This allows us to incorporate page views that improve their search experience. Therefore logging of user occurred before SearchPanel was installed. activity was necessary. To encapsulate it from the basic Each bar’s length reflects the frequency of retrieval of the functionality it was designed as plugin that could be con- corresponding page. The more frequently a page has been nected or disconnected from SearchPanel. It collects infor- retrieved, the shorter the bar gets (item 3 ). The retrieval mation related to the use of SearchPanel for the purposes of history is stored locally in the browser for privacy reasons statistical analysis of patterns of behavior. and can be deleted through SearchPanel’s option page. To maximize searchers’ privacy, no personally-identifying In SearchPanel, the bookmarking function serves two pur- information is saved. Queries and found URLs are recorded poses (item 6 in Figure 1). First, searchers can click on as MD5-hashed values only. This allows us to identify re- curring queries and documents, without being able to read [6] Even, A., Shankaranarayanan, G., and Watts, the content of the query or to observe which pages people S. Enhancing decision making with process metadata: view. Specifically, the following information is recorded: Theoretical framework, research tool, and exploratory examination. In System Sciences, 2006. HICSS’06. • The IP address and the time the event was logged Proceedings of the 39th Annual Hawaii International • When a search result was clicked and where this hap- Conference on (2006), vol. 8, IEEE, pp. 209a–209a. pened (SearchPanel or SERP) [7] Golovchinsky, G., Diriye, A., and Dunnigan, T. • Hash strings that represent the queries and found web The future is in the past: designing for exploratory pages. search. In Proceedings of the 4th Information • Time spent with the mouse on different interface parts Interaction in Context Symposium (New York, NY, (SearchPanel vs SERP) USA, 2012), IIIX ’12, ACM, pp. 52–61. [8] Hong-li, Q. A novel visual search engines: Grokker. • Various actions related to the extension (adding book- Journal of Library and Information Sciences in marks by clicking the start, moving it, etc.). Agriculture 8 (2008), 047. [9] KG, P. U. . C. Searchpreview, the browser extension 4. NEXT STEPS previously known as googlepreview. After an in-house pilot deployment, SearchPanel has been http://searchpreview.de/, 2013. [Online; accessed made available through the Google Chrome store. The goal 06/06/2013]. of the deployment is to understand whether the extension [10] Martijn. Web seach pro, search the web the way you helps people with their search tasks, and to assess the rela- like... http://websearchpro.captaincaveman.nl, tive utility of document vs. process metadata. We also ex- 2012. [Online; accessed 06/06/2013]. pect to collect a dataset that characterizes people’s browsing [11] Qvarfordt, P., Golovchinsky, G., Dunnigan, T., and searching behaviors in terms of patterns of retrieval and and Agapie, E. Looking ahead: Query preview in re-retrieval, search result navigation, etc. exploratory search. In Proceedings of the 36th international ACM SIGIR conference on Research and 5. CONCLUSIONS development in Information Retrieval (New York, NY, Web search engines are used for many different kinds of USA, 2013), SIGIR ’13, ACM. search tasks. While navigational and transactional uses of [12] Roberts, J., Boukhelifa, N., and Rodgers, P. search engines are well-supported by current interfaces and Multiform glyph based web search result visualization. algorithms, searchers are left to their own devices for more In Information Visualisation, 2002. Proceedings. Sixth open-ended information seeking and re-finding. We created International Conference on (2002), IEEE, a Google Chrome browser extension to help people manage pp. 549–554. their search activity. We explored the design space of doc- [13] Rose, D. E., and Levinson, D. Understanding user ument and process metadata related to the wide range of goals in web search. In Proceedings of the 13th activities searchers may engage in during information seek- international conference on World Wide Web (2004), ing. The extension keeps track of retrieval, page visits, and ACM, pp. 13–19. bookmarking, and integrates traces of these activities with [14] Spoerri, A. How visual query tools can support users document metadata to give people a more complete impres- searching the internet. In Information Visualisation, sion of their search activity. An upcoming deployment will 2004. IV 2004. Proceedings. Eighth International explore the effect that this extension has on how people in- Conference on (2004), IEEE, pp. 329–334. teract with search results. [15] Tauscher, L., and Greenberg, S. How people revisit web pages: empirical findings and implications 6. REFERENCES for the design of history systems. Int. J. [1] ABAKUS. Bettersearch a firefox addon for enhancing Hum.-Comput. Stud. 47, 1 (July 1997), 97–137. search engines. http://mybettersearch.com/, 2010. [16] Teevan, J. The re:search engine: simultaneous [Online; accessed 06/06/2013]. support for finding and re-finding. In Proceedings of [2] Baeza-Yates, R., Ribeiro-Neto, B., et al. the 20th annual ACM symposium on User interface Modern information retrieval, vol. 463. ACM press software and technology (New York, NY, USA, 2007), New York, 1999. UIST ’07, ACM, pp. 23–32. [3] Broder, A. A taxonomy of web search. SIGIR Forum [17] Teevan, J., Adar, E., Jones, R., and Potts, M. 36, 2 (Sept. 2002), 3–10. A. S. Information re-retrieval: repeat queries in [4] Chen, H., Fan, H., Chau, M., and Zeng, D. yahoo’s logs. In Proceedings of the 30th annual Metaspider: Meta-searching and categorization on the international ACM SIGIR conference on Research and web. Journal of the American Society for Information development in information retrieval (New York, NY, Science and Technology 52, 13 (2001), 1134–1147. USA, 2007), SIGIR ’07, ACM, pp. 151–158. [5] Elsweiler, D., and Ruthven, I. Towards [18] Wang, T. D., Deshpande, A., and Shneiderman, task-based personal information management B. A temporal pattern search algorithm for personal evaluations. In Proceedings of the 30th annual history event visualization. Knowledge and Data international ACM SIGIR conference on Research and Engineering, IEEE Transactions on 24, 5 (2012), development in information retrieval (New York, NY, 799–812. USA, 2007), SIGIR ’07, ACM, pp. 23–30.