-Ontologies are increasingly used for semantic integration across disparate curated biomedical resources, while gold-standard annotated corpora are needed for accurate training and evaluation of text-mining tools. Bringing together the respective power of these, we created the Colorado Richly Annotated Full-Text (CRAFT) Corpus, a collection of full-length, open-access biomedical journal articles that have been manually annotated both syntactically and semantically with select Open Biomedical Ontologies (OBOs), the first release of which includes ~100,000 annotations of concepts mentioned in the text of 67 articles and mapped to the classes of eight prominent OBOs. Here we present our continuing work on the corpus, including updated versions of these annotations with newer versions of the ontologies, new annotations made with two additional OBOs, annotations made with newly created extension classes defined in terms of existing classes of the ontologies, and new annotations of roots of prefixed and suffixed words.
With the ever-rising amount of biomedical literature, it is
increasingly difficult for scientists to keep up with the
published work in their fields of research, much less related
ones. The use of natural language processing (NLP) tools can
make the literature more accessible by aiding concept
recognition and information extraction. As NLP-based
approaches have been increasingly used for biocuration, so too
have biomedical ontologies, whose use enables semantic
integration across disparate curated resources, and millions of
biomedical entities have been annotated with them. Particularly
important are the Open Biomedical Ontologies (OBOs), a set
of open, orthogonal, interoperable ontologies formally
representing knowledge over a wide range of biology,
medicine, and related disciplines [
Manually annotated document corpora have become critical
gold-standard resources for the training and testing of
biomedical NLP systems. This was the motivation for the
creation of the Colorado Richly Annotated Full-Text (CRAFT)
Corpus, a collection of 97 full-length, open-access journal
articles from the biomedical literature [
All continuing work on the concept annotations of the
CRAFT Corpus was performed in Knowtator, a plugin to
Protégé-Frames [
Annotation of the corpus with the Molecular Process
Ontology (MOP) and Uberon was performed in one primary
round (by NV) followed by a review (by MB) using the
original concept annotation guidelines [
So as to remain current and relevant, the v1.0 concept annotations of the corpus are being reviewed and updated by addition, editing, and deletion of annotations as appropriate, relying on newer versions of the eight OBOs previously used. Updating with four of these has been completed.
The extension of annotation of specific affixed root words is largely for consistency: In the v1.0 corpus, any whitespace or punctuation character could serve as an annotation delimiter; thus, “chromatin” of “anti-chromatin” would be annotated with the Gene Ontology class for chromatin (GO:0000785), but it could not be annotated within “antichromatin”, as there is no delimiter. The rendering of such affixes is variable in that they can be nondelimited from their root words or delimited by whitespace or punctuation, so with this updating, the markup of such affixed words is now more consistent; furthermore, additional knowledge is captured. A specific list of such affixes to consider has been compiled and will be provided with the next release.
While creating the concept annotations for the v1.0 corpus, we encountered a variety of difficulties with annotating exclusively with explicitly represented OBO classes, including class ambiguity, lack of sufficiently generic classes, lack of classes for words consisting of combinations of multiple ontology classes, representation of the same concept in multiple ontologies and incompleteness of ontologies. To ameliorate these issues, we have been creating and using specific extension classes for concept annotations for the corpus update. All of these are formally defined in terms of explicitly represented OBO classes, and we intend to make these definitions available in OWL files in the next release. However, we also intend to release the annotations in sets both including and excluding these extension classes for users who respectively do and do not wish to make use of annotations with such classes in their work.
Finally, for the purpose of capturing additional types of
biomedically relevant concepts, annotations have been created
for the articles of the corpus using the classes of the MOP
ontology of chemical processes [
average # unique concepts per article
median # unique concepts per article
We have presented our continuing work on the goldstandard concept annotations of the CRAFT Corpus, including updated versions of the annotations with newer versions of ontologies, new annotations made with additional OBOs, annotations made with newly created ontology extension classes, and new annotations of roots of prefixed and suffixed words. We intend to soon release these updated annotations in future versions of the corpus, and we also have longer-term plans for further development of the corpus.
This work was supported by grant DARPA-BAA-14-14.