SafeAI 2019 Table of Contents Table of Contents Invited Talk Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Peter Eckersley Session 1: Safe Planning and Operation of Autonomous Systems Minimizing the Negative Side Effects of Planning with Reduced Models . . . . . . . . . . . . . . . . . . 9 Sandhya Saisubramanian and Shlomo Zilberstein Robust Motion Planning and Safety Benchmarking in Human Workspaces . . . . . . . . . . . . . . . . 16 Shih-Yun Lo, Shani Alkoby and Peter Stone Enter the Matrix: Safely Interruptible Autonomous Systems via Virtualization . . . . . . . . . . . 25 Mark Riedl and Brent Harrison Session 2: New Paradigms in AI and AGI Safety Towards Robust End-to-End Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Lê Nguyên Hoang Integrative Biological Simulation, Neuropsychology, and AI Safety . . . . . . . . . . . . . . . . . . . . . . . . 40 Gopal Sarma, Adam Safron and Nick Hay Session 3: Safety in Automated Driving How Many Operational Design Domains, Objects, and Events? . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Philip Koopman and Frank Fratrik Monitoring Safety of Autonomous Vehicles with Crash Prediction Networks . . . . . . . . . . . . . . 49 Saasha Nair, Sina Shafaei, Stefan Kugele, Mohd Hafeez Osman and Alois Knoll Session 4: Safety-Related AI Requirements and Characteristics Requirements Assurance in Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Alec Banks and Rob Ashmore Surveying Safety-relevant AI Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Jose Hernandez-Orallo, Fernando Martı́nez-Plumed, Shahar Avin and Sean O Heigeartaigh Session 5: Adversarial Machine Learning Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering . . . . . . . . . . 66 Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy and Biplav Srivastava DPATCH: An Adversarial Patch Attack on Object Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Xin Liu, Huanrui Yang, Ziwei Liu, Linghao Song, Yiran Chen and Hai Li 1 SafeAI 2019 Table of Contents Attacks on Machine Learning: Lurking Danger for Accountability . . . . . . . . . . . . . . . . . . . . . . . . 82 Katja Auernhammer, Ramin Tavakoli Kolagari and Markus Zoppelt Short Poster Papers Towards international standards for evaluating machine learning . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Frank Rudzicz, P Alison Paprica and Marta Janczarski Counterfactual Explanations of Machine Learning Predictions: Opportunities and Challenges for AI Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Kacper Sokol and Peter Flach Safe Temporal Planning for Urban Driving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Bence Cserna, William Doyle, Tianyi Gu and Wheeler Ruml Linking Artificial Intelligence Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Yi Zeng, Enmeng Lu and Cunqing Huangfu Emergence of Addictive Behaviors in Reinforcement Learning Agents . . . . . . . . . . . . . . . . . . . . . 107 Vahid Behzadan, Roman V. Yampolskiy and Arslan Munir Temporally Extended Metrics for Markov Decision Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Philip Amortila, Marc G. Bellemare, Prakash Panangaden and Doina Precup AutoMPC: Efficient Multi-Party Computation for Secure and Privacy-Preserving Cooperative Control of Connected Autonomous Vehicles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Tao Li, Lei Lin and Siyuan Gong Security-preserving Support Vector Machine with Fully Homomorphic Encryption . . . . . . . . 120 Saerom Park, Jaewook Lee, Jung Hee Cheon, Joohee Lee, Jaeyun Kim and Junyoung Byun Bamboo: Ball-Shape Data Augmentation Against Adversarial Attacks from All Directions 122 Huanrui Yang, Jingchi Zhang, Hsin-Pai Cheng, Wenhan Wang, Yiran Chen and Hai Li 2