=Paper=
{{Paper
|id=Vol-1338/paper1
|storemode=property
|title=First Workshop on Supporting Complex Search Tasks
|pdfUrl=https://ceur-ws.org/Vol-1338/paper_1.pdf
|volume=Vol-1338
}}
==First Workshop on Supporting Complex Search Tasks==
https://ceur-ws.org/Vol-1338/paper_1.pdf
First Workshop on Supporting Complex Search Tasks
Maria Gäde Mark Hall Hugo Huurdeman
Humboldt University Berlin Edge Hill University University of Amsterdam
Jaap Kamps Marijn Koolen Mette Skov
University of Amsterdam University of Amsterdam Aalborg University
Elaine Toms David Walsh
University of Sheffield Edge Hill University
ABSTRACT uments. Yet information seeking research has shown that
There is broad consensus in the field of IR that search is users go through different phases in their search sessions,
complex in many use cases and applications, both on the from exploring and identifying vague information needs, to
Web and in domain specific collections, and both profes- focusing and refining their needs and search strategies, to
sionally and in our daily life. Yet our understanding of com- finalizing their search. To be able to support exploring and
plex search tasks, in comparison to simple look up tasks, discovering strategies we need to understand the characteris-
is fragmented at best. The workshop addressed the many tics of different tasks including open-ended, leisure-focused
open research questions: What are the obvious use cases sessions. This is a highly complex problem that touches
and applications of complex search? What are essential fea- upon and bridges areas of information seeking, interactive
tures of work tasks and search tasks to take into account? information retrieval, system-centered (ranking, evaluation),
And how do these evolve over time? With a multitude of user interface design.
information, varying from introductory to specialized, and The background for this workshop is derived from the
from authoritative to speculative or opinionated, when to CLEF/INEX Interactive Social Book Search Track (2014–
show what sources of information? How does the infor- ) [8], which investigates scenarios with complex book search
mation seeking process evolve and what are relevant dif- tasks and develops systems and interfaces that support the
ferences between different stages? With complex task and user through the different stages of their search process.
search process management, blending searching, browsing, Book search provides an excellent scenario to investigate
and recommendations, and supporting exploratory search to these issues. Information needs in book search are highly
sensemaking and analytics, UI and UX design pose an over- complex, combining aspects of topical relevance (genre, sub-
constrained challenge. How do we know that our approach ject), user relevance (background knowledge, reading level,
is any good? Supporting complex search task requires new preferences and interests) and social relevance (recommen-
collaborations across the whole field of IR, and the proposed dations and opinions of friends and other trusted sources).
workshop will bring together a diverse group of researchers Moreover, book search needs develop from vague notions of
to work together on one of the greatest challenges of our interest (books similar to X) to more specific criteria (likable
field. characters, academic treatment of topic, etc.) This change
in the users needs, and the development of the tasks asso-
ciated with those needs, demonstrates that current search
1. INTRODUCTION systems provide little active support for such scenarios. Ex-
One of the current challenges in information access is sup- amples from the ISBS collection, findings based on the user
porting complex search tasks. A user’s understanding of the studies, and prototypes of information seeking stage sensi-
information need and the overall task develop as they inter- tive search systems are available, and will be used to focus
act with the system. Supporting the various stages of the the discussion in the breakout groups.
task involves many aspects of the system, e.g. interface fea-
tures, presentation of information, retrieving and ranking.
Many search systems treat the search process as a series of
2. GOALS AND OBJECTIVES
identical steps of submitting a query and consulting doc- The overall goal of the workshop is to create and foster an
interdisciplinary forum where researchers can exchange and
contribute to the development of alternative experiments
and prototypes.
The main aim is to better understand how to support com-
plex search tasks, addressing many open research questions
to be explored, including:
Copyright c 2015 for the individual papers by the papers’ authors. Copy-
ing permitted for private and academic purposes. This volume is published Context What are the obvious use cases and applications
and copyrighted by its editors.
ECIR Supporting Complex Search Task Workshop ’15 Vienna, Austria of complex search? In what sense are these “complex”?
Published on CEUR-WS: http://ceur-ws.org/Vol-1338/. What generic characteristic do they share? How can
search become an integral part of its context, and the reported to the audience and a panel of experts, with con-
context integral part of search? tinued discussion on what we learned, concrete plans for the
next year, and a road-map for the longer term.
Tasks What are essential features of work tasks and search The workshop brought together a varied group of researchers
tasks to take into account? And how do these evolve with experience covering both user and system centered ap-
over time? How do can complex tasks be decomposed proaches, to work together on the problem and potential
into manageable sub-tasks, and partial results com- solutions, and identify the barriers to success and work on
posed into comprehensive answers? How can we mon- ways of addressing them.
itor and support task progress?
Heterogeneous sources With a multitude of information, 4. ACCEPTED PAPERS
varying from introductory to specialized, and from au- We invited submissions of 3 page papers to be presented
thoritative to speculative or opinionated, when to show as posters. All submissions were peer reviewed by at least
what sources of information? When to show more or three reviewers. In this section we briefly describe each of
other types of information than directly requested by the seven accepted papers.
the searcher? Do we know when the user has gotten Dori-Hacohen et al. [6] present searching on controversial
enough? topics as a complex search task. Challenges include how
one searches for such views on such topics and in what way a
Search process How does the information seeking process system should engage users, by e.g. showing all the differents
evolve and what are relevant differences between differ- views or censoring certain views.
ent stages? What search tactics and search strategies Gunadi et al. [7] highlight the problems of distributed
are effective? How can we promote the use of effec- IR, such as dealing with heterogeneous document collections
tive search strategies? How does the information need and information systems, which presents a challenge for sys-
evolve and what are relevant success criteria for the tems to support users requiring information from multiple
end result and intermediate steps? How can we cast resources.
these as effective complex queries, and how to (inter- Balog [2] introduces the notion of task completion engines
actively) construct such queries? and argues for re-thinking the search experience. The author
presents use cases such as travel planning and shopping and
UI and UX Does the need of complex task and search pro- sketches an interface that integrates functionality to support
cess management, blending searching, browsing, and users in the various steps of their complex tasks.
recommendations, and supporting exploratory search Walsh and Hall [14] discuss the scenario of users with no
to sense-making and analytics, make UI and UX de- particular goal or information need, who engage the system
sign an overconstrained challenge? What affordances with the aim to explore collections in the domain of cultural
are required and in what stage of the search process? heritage, where the system helps them in the initial phase
How can we make the search process transparent to the to discover areas or topics of interest.
user? How and when does the initiative shift between Kumpulainen and Huurdeman [11] describe the relation
system and user? between searcher’s confidence in their search skill and un-
derstanding of the search topic and/or the system, focusing
Evaluation How do we know that our approach is any on those users with limited knowledge but high confidence.
good? Can we carve out one or a range of generic For such users, the system could shake up the search expe-
aspects testable on a suitable benchmarks? Is there rience to help them formulate their search goals better and
enough empirical evidence to ground simulated inter- change their search behaviour.
active search? What kind of novel retrieval models are Toms [13] argues that search tasks in and of themselves are
needed to combine topical, contextual and preferential often not complex, but that the complexity arises from the
aspects? related work task for which the searches are conducted. The
focus of the system should be on supporting search within
the larger work task. The author also describes different
3. FORMAT kinds of support that a system could or should give.
SCST 2015 was a half day workshop on supporting com- Dean-Hall et al. [5] present an evaluation framework for
plex search tasks—a work shop proper where discussion was Point-Of-Interest recommender systems and scenarios where
central, and all attendees were active participants. users want personalised suggestions that takes many aspects
The workshop started with a keynote by Diane Kelly (Uni- of their context into account, such as geographical location,
versity of North Carolina, Chapel Hill) to set the stage and time of day, season as well as their interests and previous
ensure all attendees were on the same page. A small number experiences.
of the short/position papers were selected for short oral pre-
sentation (10-15 minutes), all other papers had a 2 minute
boaster, and all papers were presented as posters in an inter- 5. DISCUSSION AND CONCLUSIONS
active poster session. The second half of the workshop con- This workshop is closely related to the INEX Interactive
sisted of 3-4 breakout groups, seeded from the open research Social Book Search Track (ISBS) at CLEF 2014 [8] and
questions (see §2) and the contributed papers, each group CLEF 2015. The ISBS track is focused on the domain of
thoroughly prepared by a chair who guided the discussion, book search, whereas the proposed workshop addressed is-
with examples from relevant IR evaluation campaigns such sues around the search process and system interaction from
as the TREC Session and Tasks Tracks and the SBS Inter- a broader perspective. The ISBS track of CLEF’15 ran in a
active and Suggestion Tracks. Finally, the breakout groups number of cycles, with the last and main cycle starting just
after the workshop at ECIR’15. [7] E. Gunadi, T. Plumbaum, and S. Albayrak. Ap-
Some of the organizers were involved in the SIGIR 2011 plied Distributed Information Retrieval in Enterprise
Workshop on ”entertain me” Supporting Complex Search Tasks Search. In Proceedings of the first international work-
[3]; in related discussion within the SWIRL’12: Strategic shop on Supporting Complex Search Tasks, volume 1338
Workshop on Information Retrieval in Lorne [1]; and the of CEUR Workshop Proceedings. CEUR-WS.org, 2015.
NSF Task-Based Information Search Systems Workshop [9].
There is a broad research agenda emerging that attracts in- [8] M. M. Hall, H. C. Huurdeman, M. Koolen, M. Skov,
terest from research in all area’s of information retrieval. and D. Walsh. Overview of the INEX 2014 interac-
The workshop built on the results of the earlier discus- tive social book search track. In Cappellato et al. [4],
sion, and through the CLEF/INEX SBS track [10] has al- pages 480–493. URL http://ceur-ws.org/Vol-1180/
ready been pushing this line of research with a range of user CLEF2014wn-Inex-HallEt2014.pdf.
studies, novel user interfaces, and analysis of large scale so- [9] D. Kelly, J. Arguello, and R. Capra. Nsf workshop on
cial data. The workshop was held to have a more focused task-based information search systems. SIGIR Forum,
discussion based on the results so far, and in time to inform 47(2):116–127, Jan. 2013. ISSN 0163-5840. doi: 10.
new experiments running within the CLEF’15 Social Book 1145/2568388.2568407. URL http://doi.acm.org/10.
Search Track [12]. 1145/2568388.2568407.
The workshop provided a comprehensive overview of cur-
rent work on supporting complex tasks in a variety of set- [10] M. Koolen, T. Bogers, J. Kamps, G. Kazai, and
tings, and fostered new collaboration within our field on one M. Preminger. Overview of the INEX 2014 so-
of the most important topics in the coming years. cial book search track. In Cappellato et al. [4],
pages 462–479. URL http://ceur-ws.org/Vol-1180/
6. ACKNOWLEDGMENTS CLEF2014wn-Inex-KoolenEt2014.pdf.
We thank the ECIR workshop chairs Guido Zuccon and [11] S. Kumpulainen and H. Huurdeman. Shaken, not
András Benczur and the local organizers for their support. steered – the value of shaking up the search process. In
Proceedings of the first international workshop on Sup-
References porting Complex Search Tasks, volume 1338 of CEUR
Workshop Proceedings. CEUR-WS.org, 2015.
[1] J. Allan, B. Croft, A. Moffat, and M. Sanderson. Fron-
tiers, challenges, and opportunities for information re- [12] SBS. CLEF’15 Social Book Search track, 2015. http:
trieval: Report from SWIRL 2012 the second strate- //social-book-search.humanities.uva.nl/.
gic workshop on information retrieval in Lorne. SIGIR
Forum, 46(1):2–32, May 2012. ISSN 0163-5840. doi: [13] E. Toms. Complex Tools for Complex Tasks. In Pro-
10.1145/2215676.2215678. URL http://doi.acm.org/ ceedings of the first international workshop on Sup-
10.1145/2215676.2215678. porting Complex Search Tasks, volume 1338 of CEUR
Workshop Proceedings. CEUR-WS.org, 2015.
[2] K. Balog. Task-completion Engines: A Vision with a
[14] D. Walsh and M. M. Hall. Just looking around: Sup-
Plan. In Proceedings of the first international workshop
porting casual users initial encounters with Digital Cul-
on Supporting Complex Search Tasks, volume 1338 of
tural Heritage. In Proceedings of the first international
CEUR Workshop Proceedings. CEUR-WS.org, 2015.
workshop on Supporting Complex Search Tasks, volume
[3] N. J. Belkin, C. L. Clarke, N. Gao, J. Kamps, and 1338 of CEUR Workshop Proceedings. CEUR-WS.org,
J. Karlgren. Report on the sigir workshop on ”enter- 2015.
tain me”: Supporting complex search tasks. SIGIR
Forum, 45(2):51–59, Jan. 2012. ISSN 0163-5840. doi:
10.1145/2093346.2093354. URL http://doi.acm.org/
10.1145/2093346.2093354.
[4] L. Cappellato, N. Ferro, M. Halvey, and W. Kraaij,
editors. Working Notes for CLEF 2014 Conference,
Sheffield, UK, September 15-18, 2014, volume 1180
of CEUR Workshop Proceedings, 2014. CEUR-WS.org.
URL http://ceur-ws.org/Vol-1180.
[5] A. Dean-Hall, C. Clarke, J. Kamps, and J. Kiseleva.
Online evaluation of point-of-interest recommendation
systems. In Proceedings of the first international work-
shop on Supporting Complex Search Tasks, volume 1338
of CEUR Workshop Proceedings. CEUR-WS.org, 2015.
[6] S. Dori-Hacohen, E. Yom-Tov, and J. Allan. Navigat-
ing Controversy as a Complex Search Task. In Proceed-
ings of the first international workshop on Supporting
Complex Search Tasks, volume 1338 of CEUR Work-
shop Proceedings. CEUR-WS.org, 2015.