EBeM 2022
AI Evaluation Beyond Metrics 2022

Table of Contents

• Preface
  Summary: There were 16 papers submitted for peer-review to this workshop. Out of these, 10 papers were accepted for this volume, and 3 extra were accepted but elected not to be published.

• Evaluating Sports Analytics Models: Challenges, Approaches, and Lessons Learned
  Jesse Davis , Lotte Bransen , Laurens Devos , Wannes Meert , Pieter Robberechts , Jan Van Haaren , Maaike Van Roy
• Evaluating Object Permanence in Embodied Agents using the Animal-AI Environment
  Konstantinos Voudouris , Niall Donnelly , Danaja Rutar , Ryan Burnell , John Burden , José Hernández-Orallo , Lucy Cheke
• A Framework for Categorising AI Evaluation Instruments
  Anthony G Cohn , José Hernández-Orallo , Julius Sechang Mboli , Yael Moros-Daval , Zhiliang Xiang , Lexin Zhou
• Reject Before You Run: Small Assessors Anticipate Big Language Models (short paper)
  Lexin Zhou , Fernando Martínez-Plumed , José Hernández-Orallo , Cèsar Ferri , Wout Schellaert
• The Relevance of Non-Human Errors in Machine Learning (short paper)
  Ricardo Baeza-Yates , Marina Estévez-Almenzar
• Robustness Testing of Machine Learning Families using Instance-Level IRT-Difficulty
  Raül Fabra-Boluda , Cèsar Ferri , Fernando Martínez-Plumed , Maria Jose Ramirez-Quintana
• Item Response Theory to Evaluate Speech Synthesis: Beyond Synthetic Speech Difficulty (short paper)
  Chaina Oliveira , Ricardo Prudêncio
• Evaluating Understanding on Conceptual Abstraction Benchmarks
  Victor Vikram Odouard , Melanie Mitchell
• On Young Children’s Exploration, Aha! Moments and Explanations in Model Building for Self-Regulated Problem-Solving
  Vicky Charisi , Natalia Díaz-Rodríguez , Barbara Mawhin , Luis Merino
• FERM: A FEature-space Representation Measure for Improved Model Evaluation
  Yeu-Shin Fu , Wenbo Ge , Jo Plested