=Paper=
{{Paper
|id=Vol-1798/keynote1
|storemode=property
|title=None
|pdfUrl=https://ceur-ws.org/Vol-1798/keynote1.pdf
|volume=Vol-1798
}}
==None==
<pdf width="1500px">https://ceur-ws.org/Vol-1798/keynote1.pdf</pdf>
<pre>
        Where does it end? Complex Search Tasks and Evaluation
                                                                             Mark Hall
                                                                      Edge hill University
                                                                              UK
                                                                    Hallmark@edgehill.ac.uk

ABSTRACT
The task is one of the core concepts around which Information
Retrieval (IR) and particularly its evaluation are structured. The
traditional search task is a set of instructions that motivate a query
and which enable the assessment of the query results’ relevance.
With the introduction of the user into the evaluation process the
task evolved into a set of instructions that the user is provided with
when they start the evaluation process. As these tasks become ever
more complex the boundary between the IR task and the larger
work task that motivates the IR task start to blur. This transition
poses a major question for evaluation, as it also blurs the question
of whether the evaluation is judging the quality of the IR system,
the quality of the data for completing the work task, or the user’s
ability to identify with and complete the larger work task. In my
keynote I will explore some of these issues.




CHIIR 2017 Workshop on Supporting Complex Search Tasks, Oslo, Norway.
Copyright for the individual papers remains with the authors. Copying permitted
for private and academic purposes. This volume is published and copyrighted by its
editors. Published on CEUR-WS, Volume 1798, http://ceur-ws.org/Vol-1798/.

</pre>