Preface This volume contains the papers presented at ORE 2015: The 4th OWL Rea- soner Evaluation Workshop held on June 6, 2015 in Athens (co-located with the 28th International Workshop on Description Logics). The committee decided to accept 11 papers (6 Evaluation and Benchmarks papers and 5 System papers). Each submission was reviewed by at least 3 program committee members. OWL is a logic-based ontology language standard designed to promote inter- operability, particularly in the context of the (Semantic) Web. The standard has encouraged the development of numerous OWL reasoning systems, and such systems are already key components of many applications. The goal of ORE, the OWL Reasoner Evaluation workshop, is to bring together both the developers and users of reasoners for (subsets of) OWL 2, including systems focusing on both intensional (ontology) and extensional (data) query answering. ORE 2015, as previous years, also included a competition in which OWL rea- soners were faced with different reasoning tasks. The competition included six disciplines in which reasoners could compete: ontology classification, consistency checking, and realisation each for the OWL 2 EL and OWL 2 DL profiles. The tasks were performed on several large corpora of real-life OWL ontologies ob- tained from the web, as well as user-submitted ontologies which were found to be challenging for reasoners. i Acknowledgements We thank all members of the program committee, competition organisers, addi- tional reviewers, authors of the submitted papers, developers of the submitted reasoners and ontologies, and local organizers for their invaluable effort. We also thank Andreas Steigmiller (Ulm University) for providing us with the competition framework and Konstantin Korovin (University of Manchester) who kindly provided us with the PC cluster for the competition, supported by the Royal Society grant RG080491. We also gratefully acknowledge the support of our sponsors: B2i Healthcare (https://www.b2international.com/) and DBOnto (http://dbonto.cs.ox. ac.uk/). We would also like to acknowledge that the work of the ORE organisers was greatly simplified by using the EasyChair conference management system (http://www.easychair.org) and the CEUR Workshop Proceedings publica- tion service (http://ceur-ws.org/). Last but not least, we also thank Yevgeny Kazakov who brought the cer- tificates and betting slips, Despoina Trivela who dealt with the payment of the workshop T-shirts, and Bernardo Cuenca Grau and Yujiao Zhou who will most probably bring us a query answering track in ORE 2016 (this year they were almost there). ii Organisers, PC Chairs Michel Dumontier Stanford University Birte Glimm University of Ulm Rafael Gonçalves Stanford University Matthew Horridge Stanford University Ernesto Jimenez-Ruiz University of Oxford Nicolas Matentzoglu University of Manchester Bijan Parsia University of Manchester Local Organisers Giorgos Stamou National Technical University of Athens Giorgos Stoilos National Technical University of Athens Program Committee Ana Armas Romero University of Oxford Franz Baader TU Dresden Fernando Bobillo Universidad de Zaragoza Claudia D’Amato University of Bari Christine Golbreich University of Montpellier Yevgeny Kazakov The University of Ulm Evgeny Kharlamov University of Oxford Pavel Klinov University of Ulm Ilianna Kollia National Technical University of Athens Markus Krötzsch TU Dresden Despoina Magka Yahoo! Francisco Martin-Recuerda Universidad Politecnica de Madrid Christian Meilicke University of Mannheim Julian Alfredo Mendez TU Dresden Catia Pesquita University of Lisbon, Faculty of Sciences Maria Del Mar Roldan-Garcia Universidad de Malaga Joerg Schoenfisch University of Mannheim Floriano Scioscia Politecnico di Bari Alessandro Solimando Università di Genova Weihong Song University of New Brunswick Kavitha Srinivas IBM Research Andreas Steigmiller University of Ulm Valentina Tamma University of Liverpool Dmitry Tsarkov University of Manchester Zhe Wang Griffith University Dmitriy Zheleznyakov University of Oxford iii ORE 2015 Competition In addition to workshop paper submissions, ORE 2015 also included a compe- tition in which OWL reasoners were faced with different reasoning tasks. The competition included six disciplines in which reasoners could compete: ontology classification, consistency checking, and realisation each for OWL EL and OWL DL profiles. The tasks were performed on several large corpora of real-life OWL ontologies obtained from the web, as well as user-submitted ontologies which were found to be challenging for reasoners. The ORE competition results are available here: http://dl.kr.org/ore2015/ Participating Systems TrOWL: http://trowl.org/ Konclude: http://www.derivo.de/en/produkte/konclude/ ELepHant: https://code.google.com/p/elephant-reasoner/ HermiT: http://www.hermit-reasoner.com/ MORe: http://code.google.com/p/more-reasoner/ ELK: http://code.google.com/p/elk-reasoner/ jcel: http://jcel.sourceforge.net/ FaCT++: http://code.google.com/p/factplusplus/ Jfact: http://sourceforge.net/projects/jfact/ Chainsaw: http://sourceforge.net/projects/chainsaw/ PAGOdA: http://www.cs.ox.ac.uk/isg/tools/PAGOdA/ Pellet: https://github.com/ignazio1977/pellet Racer: http://racer-systems.com/ Framework We used the same competition framework as in 2014. It is available from GitHub https://github.com/andreas-steigmiller/ore-2014-competition-framework/. Datasets The ORE 2015 data set contains more than 16,500 unique ontologies. The set comprises: – the MOWLCorp (Manchester OWL Corpus), which was obtained through a Web Crawl, Google Custom Search API and user submissions (http:// mowlrepo.cs.manchester.ac.uk/datasets/mowlcorp/), – the Oxford Ontology Library (http://www.cs.ox.ac.uk/isg/ontologies/), – a BioPortal (https://bioportal.bioontology.org/) Snapshot (June 2014), – and user submitted ontologies such as BioKB, DMOP, FHKB, USDA, DPC, genomic-CDS, City-Bench, Cell ontology, DINTO, Virtual Fly Brain, GO, Drosophile Plenotype ontology. iv The ontologies in the data set are binned by profiles. For the competition, the EL profile bin (8,805 ontologies) and the pure DL bin (7,704 DL ontologies that do not fall into one of the profiles) were used. Two further bins are ob- tained from these two bins by considering only the ontologies with an ABox (DL 2,439, EL 1,941 ontologies). The latter two are used for the realisation discipline, whereas the former ones are used for the classification and consistency checking disciplines. Within these bins, the ontologies are further categorised by size (very small, small, medium, large, very large). A file list is then created by iterating over these categories (skipping categories that are already fully covered). From these file lists, the first X are used for the competition, where X is chosen such that most reasoners are able to finish within a time limit (7 hours for classification and realisation, 3 hours for consistency checking). For classification X is 250 (OWL DL) and 300 (OWL EL), for consistency checking and realisation X is 200 (OWL DL) and 250 (OWL EL). The whole data set is available for download at http://zenodo.org/record/ 10791 and more details about the corpus can be found at http://mowlrepo. cs.manchester.ac.uk/datasets/ore-2014/. Execution The competition was executed live on June 9th with a PC cluster at the Univer- sity of Manchester provided by Konstantin Korovin. The machines of the cluster were equipped with an Intel Xeon QuadCore CPU running at 2.33GHz and 12GB RAM, where 10GB could be used by the reasoners. The reasoners were executed on the machines (one reasoner per machine) by running them natively on the used Fedora 12 operating system (64bit) or within a Java Runtime Envi- ronment (Java version 1.6). A three minute time limit was given every reasoner for each ontology, where 2.5 min was allowed for reasoning, i.e., 0.5 min could additionally/separately be used for parsing of the ontology and serialization of the result. Expected results were determined by a majority vote between the hash codes of the normalised results of those reasoners that terminated within the time limits. In case of a draw, one hash code was randomly chosen and declared as the expected hash code. Results Figures 1-3 shows the results of the ORE 2015 live competition. The competition queries are available from https://zenodo.org/record/11133/ v Fig. 1. Results of the consistency checking disciplines (OWL EL & DL) Fig. 2. Results of the classification disciplines (OWL EL & DL) Fig. 3. Results of the realisation disciplines (OWL EL & DL) vi