Preface to the Third Workshop on Evaluation of
Personalisation in Information Retrieval
(WEPIR 2021)
Nicholas J. Belkin1 , Gareth J. F. Jones2 , Noriko Kando3 and Gabriella Pasi4
1
  School of Communication & Information, Rutgers University, New Brunswick, NJ, USA
2
  ADAPT Centre, School of Computing, Dublin City University, Dublin 9, Ireland
3
  National Institute of Informatics (NII), Tokyo, Japan
4
  Department of Informatics, Systems & Communication, University of Milano-Bicocca, Milan, Italy


                                         Abstract
                                         The Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021) contin-
                                         ued the explorations from the first two editions of this workshop, into issues relating to evaluation in
                                         information retrieval where the search system includes personalisation. The workshop included invited
                                         keynote presentations, peer reviewed papers and participant working groups.




1. Introduction
The Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021) [1]
at CHIIR 2021 brought together researchers interested in developing methods for the evaluation
of personalisation in information retrieval. WEPIR 2021 was a follow on to the first WEPIR held
at at CHIIR 2018 [2] and the second WEPIR held at CHIIR 2019 [3].
   WEPIR 2021 included invited keynotes, peer reviewed paper presentations and online break-
out working groups focused on the exploration of two specific use cases in the evaluation
of personalisation in information retrieval. Breakout groups worked on the development of
methods to address evaluation of personalisation in their use case. Each group sought to identify
specific relevant factors relating to their use case in terms of factors including user activities,
data to be collected, ethical issues, and evaluation metrics.
   The overall goal of the workshop was to work towards developing a general set of principles
and guidelines for addressing the evaluation of specific instances of the use of personalisation
in information retrieval tasks.


WEPIR 2021: Third Workshop on Evaluation of Personalisation in Information Retrieval, March, 19th 2021
" belkin@rutgers.edu (N. J. Belkin); Gareth.Jones@dcu.ie (G. J. F. Jones); kando@nii.ac.jp (N. Kando);
pasi@disco.unimib.it (G. Pasi)
~ https://comminfo.rutgers.edu/belkin-nicholas (N. J. Belkin); https://www.computing.dcu.ie/~gjones/
(G. J. F. Jones); https://researchmap.jp/kando/ (N. Kando); http://www.ir.disco.unimib.it/people/pasi-gabriella/
(G. Pasi)
 0000-0002-0621-0688 (N. J. Belkin); 0000-0003-2923-8365 (G. J. F. Jones); 0000-0002-2133-0215 (N. Kando);
0000-0002-6080-8170 (G. Pasi)
                                       © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
    CEUR
    Workshop
    Proceedings
                  http://ceur-ws.org
                  ISSN 1613-0073       CEUR Workshop Proceedings (CEUR-WS.org)



                                                                                                          76
2. Keynote Presentations, Accepted Papers and Breakout
   Groups
WEPIR 2021 featured two invited keynote presentations, three peer reviewed papers and two
online breakout discussion groups.

Keynote Presentations
The following keynote presentations were made at WEPIR 2021, further details of the talks can
be found on the workshop website1

    • Keynote 1: Making A Difference in User Search Experience Hideo Joho, Faculty of
      Library, Information and Media Science, University of Tsukuba, Japan
    • Keynote 2: From Sessions to Conversations and Back Again Evangelos Kanoulas,
      Information Retrieval Lab, University of Amsterdam, The Netherlands

Accepted Papers
The following peer reviewed papers were presented at WEPIR 2021. These paper are included
in these proceedings.

    • Yoshiyuki Shoji, Kenro Aihara, Martin J. Dürst, Noriko Kando, Takuya Nakaya, Hiroaki
      Ohshima, Takehiro Yamamoto, Yusuke Yamamoto: Location-based Reminder for Memo-
      rizing What Visitors Learn at a Museum
    • Yasin Ghafourian, Petr Knoth, Allan Hanbury: Information Retrieval Evaluation in Knowl-
      edge Acquisition Tasks
    • Masaki Ueta, Tomoya Hashiguchi, Huu-Long Pham, Yoshiyuki Shoji, Noriko Kando,
      Yusuke Yamamoto, Takehiro Yamamoto, Hiroaki Ohshima: Quiz Generation on the Elec-
      tronic Guide Application for Improving Learning Experience in the Museum

Breakout Working Groups
Two breakout groups examined topics relating to personalisation in the following search
application areas:

    • Personalization of Museum Visitors’ Experiences
    • Personalised Medical Search


References
[1] G. J. F. Jones, N. J. Belkin, N. Kando, G. Pasi, Third Workshop on Evaluation of Person-
    alisation in Information Retrieval (WEPIR 2020), in: Proceedings of the 2020 Conference
    on Human Information Interaction and Retrieval, CHIIR ’20, Association for Computing
    Machinery, New York, NY, USA, 2020, p. 488–491.
   1
       https://wepir2021.disco.unimib.it



                                             77
[2] G. J. F. Jones, N. J. Belkin, S. Lawless, G. Pasi, Report on the CHIIR 2019 Second Workshop
    on Evaluation of Personalisation in Information Retrieval (WEPIR 2019), SIGIR Forum 53
    (2019) 29–37.
[3] G. J. F. Jones, N. J. Belkin, S. Lawless, G. Pasi, Report on the CHIIR 2018 Workshop on
    Evaluation of Personalisation in Information Retrieval (WEPIR 2018), SIGIR Forum 52 (2018)
    129–134.




                                              78