When Effort exceeds Expectations:
                              A Theory of Search Task Difficulty

                                                                   Diane Kelly
                                                School of Information and Library Science
                                                University of North Carolina at Chapel Hill
                                                       Chapel Hill, NC 27599 USA
                                                          dianek@email.unc.edu

Abstract                                                                    Biography
In a recent review of over one hundred interactive informa-                 Diane Kelly is a Professor at the School of Information
tion retrieval studies, Wildemuth et al. [1] observed that                  and Library Science at the University of North Carolina at
despite widespread usage of the concepts of task complex-                   Chapel Hill. Her research and teaching interests are in inter-
ity and task difficulty, “clear and consistent definitions of               active information search and retrieval, information search
these attributes are lacking and there is no consensus on                   behavior, and research methods. She is the recipient of
how to distinguish levels of complexity or difficulty within                the 2014 ASIST Research Award, the 2013 British Com-
a set of search tasks” (p. 1119). Importantly, the authors                  puter Society’s IRSG Karen Spärck Jones Award, the 2009
called for the creation of richer conceptual models and more                ASIST/Thomson Reuters Outstanding Information Science
precise and valid measurement practices. In this talk, I will               Teacher Award and the 2007 SILS Outstanding Teacher of
present a theory of search task difficulty that arose from                  the Year Award. She is the current ACM SIGIR treasurer
results of a recent empirical study that evaluated a set of                 and served as conference program committee co-chair in
search tasks that were constructed using a cognitive com-                   2013. She serves on the editorial boards of several jour-
plexity framework from educational theory. While results of                 nals including ACM Transaction on Information Systems,
this study showed participants engaged in significantly more                Information Processing & Management, and Information
search activity when completing more cognitively complex                    Retrieval Journal. Kelly received a Ph.D., M.L.S. and a
tasks, they did not evaluate more cognitively complex tasks                 graduate certificate in cognitive science from Rutgers Uni-
as more difficult and were equally satisfied with their per-                versity and a B.A. from the University of Alabama.
formances across tasks. These findings have led to a new
theory of search task difficulty that is rooted in expectancy
and appraisal theories, and posits that when searchers first
encounter a search task description, they first appraise the
task and its requirements in the context of their abilities, de-
sires and other aspects of the search situation (e.g., system
familiarity, time limits) and then make an estimate of how
much effort will be needed to complete the task to achieve
their ideal level of performance. A search task, consequently,
becomes “difficult” when expended effort exceeds expected
effort. This theory differs from many current notions of
search task difficulty that assume a linear relationship be-
tween search task difficulty and search behavior. It also has
implications for how search task difficulty is measured.

References
[1] B. W. Wildemuth, L. Freund, and E. G. Toms. Untan-
    gling search task complexity and difficulty in the context
    of interactive information retrieval studies. Journal of
    Documentation, 70(6):1118–1140, 2014.




Copyright c 2015 for the individual papers by the papers’ authors. Copy-
ing permitted for private and academic purposes. This volume is published
and copyrighted by its editors.
ECIR Supporting Complex Search Task Workshop ’15 Vienna, Austria
Published on CEUR-WS: http://ceur-ws.org/Vol-1338/.