In complex search scenarios such as planning a vacation or nding a suitable gift for a friend, at the beginning the user usually does not know exactly what he is looking for. However, this is the question that most search interfaces present as rst step. This research aims to analyze approaches for supporting the user in expressing a search query based on vague motives and ideas and in evaluating the search results in order to nd a suitable search result. Various visualization techniques and prototypes are developed to support di erent stages of the search process and lead to a construction kit for visual search interfaces.
Complex search tasks such as nding a suitable car, planning a vacation, or
identifying the perfect investment opportunity can last days or weeks and usually
the user does not know exactly what he is looking for at the beginning. Search
engines o er access to large data volumes and various possibilities to interpret the
users query like providing corrections and suggestions. Besides these technical
advantages, the search paradigm itself did not change a lot during the last years.
Most of the conventional web search interfaces use the well-known search box
or search forms to express the search query and require the user to transform
a possibly vague information need into a speci c search query [
Furthermore, the introduced scenarios correspond to more exploratory forms
of search, which require much more diverse strategies, rather than simply
submitting a query and seeing a list of matching results [
The aim of this research is to analyze the search process from the users perspective focusing on the use case of a product search. The use cases focus on complex search tasks with a vague information need such as planning a holiday or nding a suitable investment opportunity. Furthermore, di erent visualization techniques will be evaluated with the aim to enhance the quick interpretation and analysis of the products that are structured as multidimensional data sets, and the quick evaluation and comparison of the results.
The previous considerations lead to the following objectives of my PhD work: 1. the examination of approaches to support the user to express the search query 2. the investigation of di erent search behaviors and search models to support di erent search strategies and stages during the search process 3. an in-depth analysis of methods to present the result set to the user that supports a quick interpretation and comparison
The paper is organized as follows: Section 2 addresses background information and state of the art concerning Information Search and Information Visualization. Section 3 outlines the research methodology and section 4 describes a proposed approach to support a product search with vague information need. Section 5 presents brie y di erent prototypes and evaluation results, which are a basis for a construction kit for visual search interfaces that is introduced in section 6. Finally, section 7 concludes the paper and outlines future work. 2
This research spans several areas, such as Information Retrieval, Information
Seeking, Human-Computer Interaction and Information Visualization. Whereas
Information Retrieval focuses on the technologies that support the nding and
presentation of information, Information Seeking is primarily concerned with the
seeking of information and focuses on the users and their search activities [
Information Seeking
Search activities can be distinguished in Lookup and Exploratory Search [
Information Domain Search Design
Need Expertise Strategy Paradigm Concrete Vague
High Low
A Direct
A Search
B
A + B Navigational
A + B Search
A – Analytical
B – Browsing
process that usually starts with a vague information need and therefore requires
multiple iterations of learning, investigation, and reformulation of the search
query. The use case of a product search corresponds to exploratory search, with
the di erence that product search is not an open-ended search but aims to
conclude with nding a suitable product. Exploratory search scenarios often
start with a vague information need and usually blend two search strategies:
analytical and browsing [
Furthermore, two search paradigms are well established in the web search
world: Direct Search and Navigational Search [
I analyzed di erent design patterns in the context of product search and
assigned to the introduced search strategies and paradigms (see Fig. 1, left).
Design Patterns have emerged as recurring solutions to common problems and
can be adapted to the current context [
Information Visualization
Furthermore, I analyzed di erent e-commerce websites to identify patterns to
present the result sets. Products were either presented as an image, when the
visual appearance is important e.g. in case of travel destinations or clothes, or
they were presented in an abstract way, mostly through textual descriptions or
using simple diagrams like bar charts or line graphs. Thus, not many attributes
of a product can be presented and one-dimensional lists and galleries are still the
mostly used patterns (see Fig. 1, right). In contrast, the research area of
Information Visualization o ers various techniques for visualizing multidimensional
data, which are needed in the context of product search. Keim distinguishes
between geometric techniques (e.g. scatterplots, parallel coordinates), icon-based
techniques (e.g. star plots, cherno faces), and pixel-oriented techniques, where
each data value is mapped to a colored pixel [
The research methodology is divided into 5 main steps: 1. Analysis of the current state of the art in Information Seeking to understand the search process and its strategies and to identify limitations and desiderata 2. Analysis of the current state of art in Information Visualization with respect to the visualization of products with multidimensional attributes 3. Proposal of a model to support the product search with a vague information need and the aim to nd a suitable product 4. Design, prototyping, and evaluation of di erent search approaches targeting di erent data sets, user experiences, and information needs 5. Decomposition of the prototypes and assignment to di erent contexts in product search with the aim to develop a construction kit for visual search interfaces that is providing patterns, which suit to di erent data structures and targets and is supporting the designer in giving inspiration for the development of new interfaces 4
To get a better understanding of how people seek information and to describe the
process of information search from the users perspective, Kuhlthau performed a
series of studies and identi ed distinct phases and emotions unique to each phase
[
The goal of this research is to investigate strategies and methods to support the rst stages of the process of information search. Therefore, it is concentrated on complex search tasks, such as planning a vacation, in which the user is unsure of what he is looking for at the beginning. Motive-based Search re nes Explorative Search by specifying the motive and the aim of the process to nd a suitable product. A motive can be de ned as the reason for a search as well as particular conditions such as how much a product should cost and is usually in uenced by emotions and interests of the user. Because this motive is the starting point of the search task, this type of task is called Motive-based Search in this thesis.
vague
concrete
entirety individual
Based on a workshop with potential users and the research of Marchioni
and Russel-Rose, who consider search as a holistic process, integrating
ndability with analysis and sensemaking [
In the beginning of the search, the user decides to book a vacation without a concrete idea where he wants to go. Based on his motive for the vacation, some basic conditions and constraints have to be met. In this example, he is looking for a cheap short trip in the near future and all travels by air can be quickly discarded because of his fear of ying. In the phase of Exploration, the user is unsure at the beginning of his search and needs inspiration and guidance to start the search process. To support the user in creating new ideas, the interface has to provide functions that broaden his scope of information (see Inspire in Fig. 2). During the search, the search idea is getting more concrete and leads to Formulation and Reformulation tasks.
When nding some interesting topic, e.g. a trip to Paris, Rome or Madrid, the users task is to read thematically relevant information, and to relate this information with previous knowledge in order to extend the personal understanding of the topic. The interface can support the user by o ering functions to construct and organize his knowledge space (see Investigation in Fig. 2). The tasks that should be supported in this phase are described as follows: Trace (shows the user, where he has been already and helps to orientate in the information space), Collect (supports the user with collecting and organizing his ndings), Examine (gives more information about an unknown nding and supports the learning process).
Finally, the user has to grasp the possibilities of combining bits of information and di erent alternatives. The task of the interface in the last phase is to help the user to narrow the scope and to create a focus (see Evaluation in Fig. 2). The Evaluation is necessary to judge the value of an item or item collection with respect to the search goal. It is supported by tasks that are leading from Analyzing the entire result set and Comparing two ore more items to identify similarities and di erences; to a Veri cation, that is used to con rm if one item meets all speci c criteria. 5
Ten prototypes were developed, which focus on di erent search scenarios and stages of the motive-based search process. In this section, four of these prototypes are presented based on di erent data sets and visualization techniques1. Each concept supports a particular search strategy to nd the desired product in a very large database with structured or semi-structured data. Further on, di erent search domains such as image search, travel search, and nancial products are considered. Since the domains entail di erent user requirements and expectations, the resulting interface concepts support emotional or analytical decisions. 5.1
Relation-based Concept
The rst concept is based on a folksonomy as data structure organized in a
multidimensional classi cation scheme. The developed DelViz (Deep exploration and
lookup of Visualizations) concept supports di erent search tasks such as nding
suitable visualizations for a given context, and analysis of the underlying
structured data set to discover relationships between the search results [
Facet-based Concept
The second concept focuses on data sets that are structured with a faceted
classication. It is based on two visualization techniques that allow the representation
of multidimensional data across a set of parallel axes: parallel coordinates [
Recommendation-based Concept
We designed a recommendation-based concept to support and trigger
emotionally driven decisions and uses an ontology of concepts as data set that describe a
holiday, such as warm, beach, party and culture. Fig. 4 (top) shows the interfaces
getInspired for a travel search with a vague information need. Instead of a direct
query on the attribute of the result set, the user communicates his preferences
to the system through a selection of concepts represented by expressive images.
Based on the previous decision of the user, the system decides which concepts are
presented in the next re nement step. A user study with 29 participants and 12
di erent search tasks was conducted to compare the recommendation-based
approach to a strict navigational concept selection. The study measured time and
clicks to solve a given search task and indicated that the recommendation-based
approach was faster and more e cient than the navigational concept.
Furthermore, the results show an obvious trend of increased user experience as well as
inspiration while the search result quality has not dropped [
Example-based Concept user starts the search with a given example of one nancial product on the lefthand side and gets more information about its properties, such as risk, maturity, and other special features for this single product. Then the user can decide for each attribute if this criterion is important and can select and deselect them, which in uences the similarity algorithm. On the right-hand side, each circular glyph represents a single product and depicts its most important features, such as performance over time and investment term.
An early prototypical implementation suggests that this kind of interface is
well suited for the paradigm of search by example and the browsing of a large
structured or semi-structured dataset. Whilst the overall design of the interface
may remain the same for any kind of structured dataset, the iconic representation
should be adapted to the use case [
Based on the previous introduced prototypes a construction kit was developed,
which aims to support the search process with the aid of visualization. The
construction kit contains di erent building blocks, introduced in Figure 5. These
building blocks can be combined to a pattern, that is subdivided into three parts:
What: describes the data input
Why: describes the task that has to be solved
How: describes how the pattern is designed
The elements of the parts "What" and "How" are in uenced by the visualization
taxonomy used in the multidimensional classi cation scheme for information
visualizations introduced in [
These patterns can be composed to construction plans, that enables to describe complex search interfaces with the help of three di erent connectors: Successive: the patterns are shown successively Juxtaposed: the patterns are shown next to each other Superimposed: the patterns are combined with each other in one view Furthermore, it is possible to combine individual building blocks with reference blocks to indicate the in uence on another pattern (see Figure 6, reference blocks "B" and "C").
In order to explain this composition, Figure 6 shows a construction plan that has been reverse engineered from a search interface using parallel sets (see section 5.2). The underlying data set is a faceted classi cation. The Search Task is to formulate search queries and to analyze how many results are left in each node. Pattern A is visualized by using the building blocks area, a rectangular grid and bars to compare the sizes of data sets. Highlighting is used to refer to connected streams, and the bars function as a lter to reduce the result set. The last interaction method in uences pattern B and C, hence, it is combined with two reference blocks (shown as small rectangles attached to the bottom right of a pattern. Pattern B deals with the underlying faceted classi cation as well. Facets can be nominal (location), ordinal (ranking) or quantitative (price). The Search Task is to analyze the correlations between these facets and to support the reformulation of search queries. It uses a parallel plot as Layout Structure with rectangular Grid but with ows instead of lines to represent a set of results. Single ows can be highlighted. It is combined with pattern A by a superimposed connector, hence they overlap in one visualization. The bars serve as axes for Network 1-Dimensional
2-Dimensional Set Temporal Computer Model 3-Dimensional
Multidimensional
Ordered Ordinal
Quantitative
Item A1 A2 . . An
IIIttteeemmm..12n Spatial Table
Form DATA STRUCTURE Hierarchy Faceted Classification Ontology Folksonomy ATTRIBUTE TYPE
Categorical Nominal HOW CONTENT
Text
Image ABC SEARCH TASKS vague the parallel plot layout and also as lter in pattern B, which is indicated by the reference block attached to pattern A, referencing pattern B. Also in pattern B, the interaction method reduce combined with the reference block indicates the in uence of the lter on pattern C. The underlying Data Structure of pattern C is an ordered (ordinal ) result set with the Search Task to verify individual items. All items are ordered in a list of a title (text ) in a rectangular grid. More details are shown for each item by using the inspect interaction method. The pattern is combined with the juxtaposed connector to pattern A and B and is in uenced by their ltering. The interaction select in pattern C also in uences pattern B, by selecting single items in the list that are highlighted in the other visualization. 7
This paper presents the current stage of the PhD thesis. The presented construction kit, presents the last phase of the PhD and serves the purpose to support the designer in quickly creating new visual search interfaces by giving him or her a set of elements, which can be easily combined with each other. Resulting patterns can be used for reuse and adaptation in di erent contexts. The patterns are conceptually simple and provide a solid foundation for reuse and redesign. Furthermore, they can be networked in a very exible way to create complex search interfaces. However, the construction kit itself provides only syntactic support for this combination process by o ering the elements and the rules to combine the building blocks. The semantic level is concerned with the content and its meaning, which will be addressed by providing example patterns based on existing solutions. This will be addressed in the last phase of my PhD work. Acknowledgments. I would like to thank my supervisor, Prof. Rainer Groh for the support and guidance provided in this research. Parts of this research has been supported by the European Union and the Free State Saxony through the European Regional Development Fund (ERDF).