Vague Query Formulation by Design Marcus Nitsche Andreas Nürnberger Faculty of Computer Science, Faculty of Computer Science, Otto-von-Guericke-University, Otto-von-Guericke-University, Germany Germany marcus.nitsche@ovgu.de andreas.nuernberger@ovgu.de ABSTRACT know how to formulate their information need. Often this prob- When users search for information in domains they are not famil- lem coexists with an unfamiliarity with the domain they search in iar with, they usually struggle to formulate an adequate (textual) [17]. In this work we like to tackle this problem of formulating ap- query. Often users end up with repeating re-formulations and query propriate queries by offering dynamic user interface (UI) elements refinements without necessarily achieving their actual goals. In this that users can manipulate directly by touch gestures to give them paper we propose a user interface that is capable to offer users flexi- a feeling for a certain query configuration that matches a certain ble and ergonomic interaction elements to formulate even complex result set. Thereby learning and exploring aspects will be covered queries in a simple and direct way. We call this principle vague as well [17, 11]. This concept of interactive visual filtering of rele- query formulation by design. By this formulation we like to point vant information in a more natural way enables data processing in out its design-driven origin. The proposed radial user interface sup- cases, where standard algorithms can not be applied since these al- ports phrasing and interactive visual refinement of vague queries to gorithms might filter out relevant data. We introduced the concept search and explore large document sets. The main idea is to pro- of this paper back in 2011 [15], where we described the basic idea vide an integrated view of queries and related results, where both and did some pre-studies with a digital mock-up prototype. In this queries and results can be interactively manipulated and influence paper, we first introduce a running implementation and a more de- each other. Changes will be immediately visualized. The concept tailed user study towards this concept. Therefore we present some was implemented on a tablet computer and the usability was step- related work aspects in Section 2, followed by a presentation of the wise evaluated during a formative and a summative evaluation pro- UI concept in 3 and the description of the implementation, evalua- cess. The results reveal high usability ratings, even if the concept tion concept and results of the final user study in Section 4. Finally, was completely unknown to our test users. we conclude and discuss possible future work in 5. Keywords 2. STATE-OF-THE-ART User-specific context aware data filtering is not a new challenge. Search User Interface, Query Reformulation, Query Refinement, In the following we show two tools, that can also be used for this Exploratory Search User Interface, Information Retrieval. application. The VIBE-system [10, 16] supports users in finding relevant information using magnets to attract relevant documents Categories and Subject Descriptors to specific screen points (Fig. 1). H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval.; H.5.2 [Information Interfaces and Presentation]: User Interfaces. General Terms Design, Human Factors, Management. 1. INTRODUCTION When users try to handle complex information needs they often end up in conducting exploratory searches [11]. One of the main characteristics of exploratory searches is that users often do not Figure 1: webVIBE, a variant of [10, 16]. This system follows the principle of dust-and-magnet [18]. Our proposed concept uses this principle also - as one aspect of the in- Presented at EuroHCIR2012. Copyright c 2012 for the individual papers by the papers’ authors. Copying permitted only for private and academic teraction concept. In contrast to VIBE we offer users of our system purposes. This volume is published and copyrighted by its editors. an interactive visualization without any classical WIMP-interface elements (Windows, Icons, Menus, Pointer). By this, no virtual mapping of functions is necessary and users might be able to use the interface in a more firm and reliable way. Cousins et al. [5] de- veloped a system that follows a direct manipulation approach like done here. But in contrast to our proposed solution it is divided into different UI elements and different views. It is less integrated in a single view. Therefore user’s work load might be higher since he needs to face various mode switches. Commercial systems, like the Vis4you concept1 , are more focused on visualization than on interaction via direct manipulation. Furthermore, their system is Figure 3: Desktop: more relevant documents are centred. designed to be used on desktop computers with a mouse (single point and click-principle), no multi-touch-support. In the next sec- tion we like to present our concept in more detail. weights can be linked to specific query terms (Fig. 2). Data points represent the data space. Query objects (widgets) can be entered via a virtual keyboard and can also be dragged by the user to formulate 3. CONCEPT & DESIGN more complex or vague queries. Selecting a specific data point Due to the increasing amount of data and complexity, it is neces- supports the user with additional information on this data point and sary to apply and improve the concepts of visual information filter- highlights all further related data points. ing and retrieval. This goes along with the underlying methods and The distance of a certain term is directly connected to its impor- tools. Considering clustering algorithms (e.g., k-means [3]), we tance for the user. In other words, if a user thinks a specific term thought about the concept of vague query formulation: Since users is more relevant to its actual filter/search task, he positions the cor- sometimes do not know what they are searching for, we like to sup- responding UI element nearer to the center, which influences the port them by the opportunity to formulate vague queries. Here the weight of this term when computing its Term Frequency / Inverted user is asked to narrow the search results by dragging user interface Document Frequency (TF/IDF)-value [2], which in fact is a calcu- (UI) elements, so called widgets, with query terms, see also Fig. 2. lated weight to influence the ranking of the data space and this in return effects the visualization (Fig. 6). Thereby, users do not need to specify a concrete position of UI elements on the screen, we support this by a non-determined precision. The widget-induced relevance of a query term is calculated according to the formula in Fig. 4. Figure 4: Widget-induced relevance of a query term. Result elements are placed near to corresponding query elements. The formula for calculating the relevance of a SearchResult object (result dot) is shown in Fig. 5. Figure 2: Radial design of the implemented UI. The concept follows the idea that more relevant data are cen- tred. Note, this is equivalent to filtering an overcrowded desktop, cf. Fig. 3 (left picture)2 , where the more centralized documents are possibly more important (highlighted in the right picture). The system was designed to be a multi-user system. Therefore a number of multiple users need to be supported at the same time, Figure 5: Relevance of a SearchResult object. also considering security aspects [14]. To offer each user the same possibility to interact with the system we use a radial form for the The calculated relevance determines the distance to the center, interface layout. Furthermore, an underlying multi-touch device considering further result objects. is a hardware requirement, that enhances the combination of tool To address various types of end devices such as multi-touch desk- and application domain significantly. Another appealing advantage tops or mobile interfaces with large displays, we use direct manip- is, that multi-touch also supports users in a more natural way of ulation as a central interaction paradigm. Only the relative distance interaction [9]. Other radial user interfaces for selecting or filtering of an UI element to the center is relevant for the system. Thus, often offers fixed places for items. In contrast to this our system we provide users with a direct linking to the data they like to filter. is supposed to be more flexible since users are allowed to position By this interaction concept, we propose to achieve more precise their query widgets where they like. results. Additionally, we support users with the concept of What- We offer users a dimension merging according specified weights, if -queries, which supports a fault-tolerant interaction system, using similar to the result listing of search engines, where also different a ghosting technique: Dragging an element and holding it on a spe- cific position triggers the system to show the user how many items 1 http://www.vis4you.com/vis4you/ (accessed on 04.07.2012) are in the center point of interest (POI) after releasing the element. 2 http://lawprofessors.typepad.com/ (accessed on 04.07.2012) Thereby, users are able to explore the impact of possible next steps. Figure 6: Concept of relevance mapping. Changes of the query configuration also effect the data points to provide the user with a direct link to the data (interactive visu- Figure 9: Prototypical search result popover as a website pre- alization). By the underlying metaphor of magnets, we offer an view feature, here a result for ’Labrador Retriever’. integrated feedback, comparable to Dust-and-Magnet [18]: When users drag a specific UI element to a certain point, relevant data points follow this UI element. Data points that have the same TF- fore the application was written in ObjectiveC using the xCode en- IDF value (equal relevance to a query configuration) are drafted vironment3 . The backend architecture is the CARSA system [1], with a minimal distance to each other to minimize the possibility an information retrieval framework for research purposes. For a of occlusions. detailed overview about the system’s architecture see Fig. 10. 3.1 Features The UI supports direct feedback since the relevance value is si- multaneously shown while users interact with the widget (Fig. 7). Figure 7: Direct feedback: relevance value next to the widget. Results, corresponding to a specific query object are visually highlighted and grouped to each other (Fig. 8). Figure 8: Corresponding results are visually highlighted to group them (e.g. highlighted results for the search term ’cat’). Detailed information on particular result objects, like a website preview, is provided after clicking on the result dot (Fig. 9). Figure 10: System architecture & UI interaction, cf. [1]. 4. IMPLEMENTATION, EVALUATION & The evaluation concept followed a formative evaluation concept RESULTS where several usability testings were conducted. Also in parallel to Since this contribution is basically driven by fields of human the development process: To identify at least 85% of all usability factors and user interface design, we are using common methods issues this mock-up was evaluated according to Nielsen and Lan- from these research areas. Such as user centred design (UCD) pro- dauer [13] with only a small number of test users since most usabil- cesses [7], formative evaluation methods [12], questionnaires [6], ity issues will be mentioned repeatedly by users. The sixth tested think-aloud-protocols [8], and cognitive walkthroughs [4]. user would report new usability issues in only 15% of all cases. To proof the concept of the proposed user interface, a prototype 3 was implemented. This was done by using an Apple iPad. There- developer.apple.com/xcode/ (accessed on 04.07.2012) Therefore we decided to ask only eight users. The results of this 7. REFERENCES first user test seem to be promising that this concept works as de- [1] Bade, K., De Luca, E. W., Nürnberger, A., Stober, S.: sired. Users were introduced in the main features and were asked CARSA - an architecture for the development of context afterwards to formulate a filter query consisting of three terms to adaptive retrieval systems. In: Proceedings of Adaptive find all relevant documents while visualizing most important re- Multimedia Retrieval: User, Context, and Feedback, Volume lations to other potential interesting data. After going through a 3877/2006, Lecture notes in computer science, pp. 91-101, cognitive walk-through of a movie filtering task our eight test users Springer Berlin / Heidelberg (2006). (six male, two female, average age: 23.4) answered seven usability [2] Baeza-Yates, R. and Ribeiro-Neto, B.: Modern Information questions by filling out a 7-step Likert scale from 1 (very bad) to 7 Retrieval, pp. 29-30. Addison Wesley / ACM Press, NY (very good). Next to cognitive walk-throughs, we used think-aloud- (1999). protocols and questionnaires. The usefulness of the prototype was [3] Bradski, G. and Kaehler, A.: Learning OpenCV Computer rated high, the functionality was praised by test users, perform- Vision with the OpenCV Library. O’Reilly, p. 479 (2001). ing tasks were rated as very easy and test users were satisfied with [4] Busemeyer, J. R.: Choice behavior in a sequential this prototype. Terminology, attractiveness, and consistency were decision-making task. In: Organizational Behavior and rated lower. Our final evaluation revealed the results you can see in Human Performance 29 (2), pp. 175-207(1982). Fig. 11. Even if there is room for improvement the results reveal overall a good usability, several test users mentioned that it was fun [5] Cousins, S. B., Paepcke, A., Winograd, T., Bier, E. A., Pier, to use it, which might is reflected by a high rating of joy of use K.: The digital library integrated task environment (DLITE). measurings. In: Proceedings of the second ACM international conference on Digital libraries (DL ’97). ACM, New York, NY, USA, pp. 142-151 (1997). [6] Czaja, R. and Blair, J.: Designing Surveys. Pine Forge Press. A useful resource for factual-style surveys, including material on interviews as well as mail surveys (1996). [7] Eason, K. D.: User centred design for information technology systems. In: Physics in Technology 14 (5), p. 219 (1983). [8] Ericsson, K. A. and Simon, H. A.: Verbal reports as data. In: Psychological Review 87 (3), pp. 215-241 (1980). [9] Han, J. Y.: Multi-touch interaction wall. In: Proceedings of ACM SIGGRAPH 2006 Emerging technologies (2006). [10] Koshman, S. L.: VIBE User Study Technical Report LS062/IS97001, University of Pittsburgh (1997). [11] Marchionini, G.: Exploratory search: from finding to understanding. In: Communications of the ACM 49 (4), pp. 41-46 (2006). [12] Moxley Jr., R. A.: Formative and non-formative evaluation. In Instructional Science 3(3), pp. 243-283 (1974). [13] Nielsen, J. and Landauer, T. K.: A mathematical model of Figure 11: Results of final usability testing. the finding of usability problems. In: Proceedings of ACM INTERCHI’93 Conference, pp. 206-213, Amsterdam, The Netherlands (1993). 5. DISCUSSION & OUTLOOK [14] Nitsche, M., Dittmann, J., Nürnberger, A., Vielhauer, C., We described a newly designed user interface concept for fil- Buchholz, R.: Security-relevant Challenges of selected tering, exploring and managing data via direct manipulation, sup- Systems for Multi-User Interaction. In Proceedings of the porting multiple reference systems and context sensitive interaction 7th International Workshop on Adaptive Multimedia techniques. We proposed a prototype for visual filtering, that is Retrieval (2011). [15] Nitsche, M. and Nürnberger, A.: Supporting vague query • flexible: parameters can be adapted or enhanced by users formulation by using visual filtering. In Proceedings of Lernen, Wissen, Adaption (2011). • context-sensitive: initial parameters are extracted from the current use case [16] Olsen, K. A., Korfhage, R. R., Sochats, K. M., Spring, M. B., and Williams, J. G.: Visualization of a Document • easy to learn: through work environment metaphor and direct Collection: the VIBE System. In: Information Processing & manipulation Management, 29(1), pp. 69-81 (1993). [17] White, R. W. and Roth, R. A.: Exploratory search: Beyond In the near future, a more detailed and larger user study will be the Query-Response paradigm. In: Synthesis Lectures on conducted to identify further room for improvements of our tool Information Concepts, Retrieval, and Services, Ed.: G. and the overall concept. We also plan to re-design it slightly. Marchionini, Morgan & Claypool Publishers (2009). [18] Yi, L. S., Melton, R., Stasko, J. and Jacko, L.: Dust & 6. ACKNOWLEDGEMENT Magnet: multivariate information visualization using a Part of the work is funded by the German Ministry of Educa- magnet metaphor. In: Information Visualization, pp. tion and Science (BMBF) within the ViERforES II project (no. 239–256 (2005). 01IM10002B). We also thank Martin Schemmer for the implemen- tation of the presented concept during his diploma thesis.