In addition to workshop paper submissions, ORE 2014 also included a competition in which OWL reasoners were faced with di erent reasoning tasks. The competition included six disciplines in which reasoners could compete: ontology classi cation, consistency checking, and realisation each for OWL EL and OWL DL reasoners. The tasks were performed on several large corpora of real-life OWL ontologies obtained from the web, as well as user-submitted ontologies which were found to be challenging for reasoners.
The ORE 2014 data set contains overall 16,555 unique ontologies. The set comprises:
The ontologies in the data set are binned by pro les. For the competition, the EL pro le bin (8,805 ontologies) and the pure DL bin (7,704 DL ontologies iv that do not fall into one of the pro les) were used. Two further bins are obtained from these two bins by considering only the ontologies with an ABox (DL 2,439, EL 1,941 ontologies). The latter two are used for the realisation discipline, whereas the former ones are used for the classi cation and consistency checking disciplines.
Within these bins, the ontologies are further categorised by size (very small, small, medium, large, very large). A le list is then created by iterating over these categories (skipping categories that are already fully covered). From these le lists, the rst X are used for the competition, where X is chosen such that most reasoners are able to nish within a time limit (7 hours for classi cation and realisation, 3 hours for consistency checking). For classi cation X is 250 (OWL DL) and 300 (OWL EL), for consistency checking and realisation X is 200 (OWL DL) and 250 (OWL EL).
The whole data set is available for download at http://zenodo.org/record/ 10791 and more details about the corpus can be found at http://mowlrepo. cs.manchester.ac.uk/datasets/ore-2014/.
The competition was executed live on July 18th with a PC cluster at the University of Manchester provided by Konstantin Korovin. The machines of the cluster were equipped with an Intel Xeon QuadCore CPU running at 2.33GHz and 12GB RAM, where 10GB could be used by the reasoners. The reasoners were executed on the machines (one reasoner per machine) by running them natively on the used Fedora 12 operating system (64bit) or within a Java Runtime Environment (Java version 1.6). A three minute time limit was given every reasoner for each ontology, where 2.5 min was allowed for reasoning, i.e., 0.5 min could additionally/separately be used for parsing of the ontology and serialization of the result. Expected results were determined by a majority vote between the hash codes of the normalised results of those reasoners that terminated within the time limits. In case of a draw, one hash code was randomly chosen and declared as the expected hash code.
The results of the ORE 2014 live competition are available from https:// zenodo.org/record/11142/. The competition queries are available from https: //zenodo.org/record/11133/
The rst three reasoners (ranked by number of expected results within the time limit of 3 min per ontology) were given prizes:
v
The competition was also part of the 1st FLoC Olympic Games 2014 (http: //vsl2014.at/olympics/) together with 13 other competitions. For the Olympic Games each competition could award three Kurt Godel Medals. For ORE 2014 the reasoners were ranked according to the number of expected results over the number of attempted tasks over all disciplines in which a reasoner participated. The medal winners were: 1. Prize: Konclude (95.5%) 2. Prize: ELK (86.4%) 3. Prize: MORe (85.7%)