The Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021) continued the explorations from the first two editions of this workshop, into issues relating to evaluation in information retrieval where the search system includes personalisation. The workshop included invited keynote presentations, peer reviewed papers and participant working groups. The Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021) [1] at CHIIR 2021 brought together researchers interested in developing methods for the evaluation of personalisation in information retrieval. WEPIR 2021 was a follow on to the first WEPIR held at at CHIIR 2018 [2] and the second WEPIR held at CHIIR 2019 [3]. WEPIR 2021 included invited keynotes, peer reviewed paper presentations and online breakout working groups focused on the exploration of two specific use cases in the evaluation of personalisation in information retrieval. Breakout groups worked on the development of methods to address evaluation of personalisation in their use case. Each group sought to identify specific relevant factors relating to their use case in terms of factors including user activities, data to be collected, ethical issues, and evaluation metrics. The overall goal of the workshop was to work towards developing a general set of principles and guidelines for addressing the evaluation of specific instances of the use of personalisation in information retrieval tasks.
WEPIR 2021 featured two invited keynote presentations, three peer reviewed papers and two online breakout discussion groups.
The following keynote presentations were made at WEPIR 2021, further details of the talks can be found on the workshop website1 • Keynote 1: Making A Diference in User Search Experience Hideo Joho, Faculty of
Library, Information and Media Science, University of Tsukuba, Japan • Keynote 2: From Sessions to Conversations and Back Again Evangelos Kanoulas,
Information Retrieval Lab, University of Amsterdam, The Netherlands
The following peer reviewed papers were presented at WEPIR 2021. These paper are included in these proceedings.
• Yoshiyuki Shoji, Kenro Aihara, Martin J. Dürst, Noriko Kando, Takuya Nakaya, Hiroaki Ohshima, Takehiro Yamamoto, Yusuke Yamamoto: Location-based Reminder for Memorizing What Visitors Learn at a Museum • Yasin Ghafourian, Petr Knoth, Allan Hanbury: Information Retrieval Evaluation in Knowledge Acquisition Tasks • Masaki Ueta, Tomoya Hashiguchi, Huu-Long Pham, Yoshiyuki Shoji, Noriko Kando, Yusuke Yamamoto, Takehiro Yamamoto, Hiroaki Ohshima: Quiz Generation on the Electronic Guide Application for Improving Learning Experience in the Museum
Two breakout groups examined topics relating to personalisation in the following search application areas: • Personalization of Museum Visitors’ Experiences • Personalised Medical Search [2] G. J. F. Jones, N. J. Belkin, S. Lawless, G. Pasi, Report on the CHIIR 2019 Second Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2019), SIGIR Forum 53 (2019) 29–37. [3] G. J. F. Jones, N. J. Belkin, S. Lawless, G. Pasi, Report on the CHIIR 2018 Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2018), SIGIR Forum 52 (2018) 129–134.