OSIRRC 2019
The Open-Source IR Replicability Challenge

Table of Contents

• Preface i-ii
  Ryan Clancy , Nicola Ferro , Claudia Hauff , Jimmy Lin , Tetsuya Sakai , Ze Zhong Wu
• Overview of the 2019 Open-Source IR Replicability Challenge (OSIRRC 2019) 1-7
  Ryan Clancy , Nicola Ferro , Claudia Hauff , Jimmy Lin , Tetsuya Sakai , Ze Zhong Wu

Position Papers

• STELLA: Towards a Framework for the Reproducibility of Online Search Experiments 8-11
  Timo Breuer , Philipp Schaer , Narges Tavalkolpoursaleh , Johann Schaible , Benjamin Wolff , Bernd Müller
• Let's measure run time! Extending the IR replicability infrastructure to include performance aspects 12-16
  Sebastian Hofstätter , Allan Hanbury
• Reproducible IR needs an (IR) (Graph) Query Language 17-20
  Chris Kamphuis , Arjen P. de Vries

Docker Papers

• Entity Retrieval Docker Image for OSIRRC at SIGIR 2019 21-25
  Negar Arabzadeh
• Dockerising Terrier for The Open-Source IR Replicability Challenge (OSIRRC 2019) 26-30
  Arthur Barbosa Câmara , Craig Macdonald
• Dockerizing Automatic Routing Runs for The Open-Source IR Replicability Challenge (OSIRRC 2019) 31-35
  Timo Breuer , Philipp Schaer
• University of Waterloo Docker Images for OSIRRC at SIGIR 2019 36-36
  Ryan Clancy , Zeynep Akkalyoncu Yilmaz , Ze Zhong Wu , Jimmy Lin
• A Docker-Based Replicability Study of a Neural Information Retrieval Model 37-43
  Nicola Ferro , Stefano Marchesin , Alberto Purpura , Gianmaria Silvello
• Dockerizing Indri for OSIRRC 2019 44-46
  Claudia Hauff
• The OldDog Docker Image for OSIRRC at SIGIR 2019 47-49
  Chris Kamphuis , Arjen P. de Vries
• PISA: Performant Indexes and Search for Academia 50-56
  Antonio Mallia , Michał Siedlaczek , Joel Mackenzie , Torsten Suel
• ielab at the Open-Source IR Replicability Challenge 2019 57-61
  Harrisen Scells , Guido Zuccon
• BM25 Pseudo Relevance Feedback Using Anserini at Waseda University 62-63
  Zhaohao Zeng , Tetsuya Sakai