In the life sciences, the amount of available knowledge has increased drastically over the last decades. Reference ontologies are an essential tool for organising and making this knowledge accessible. However, since they are designed and maintained manually, extending them is costly and keeping up with scientific progress is almost impossible. In this PhD project, we develop methods that automatically extend the coverage of reference ontologies while remaining faithful to developers' intentions, using the example of the Chemical Entities of Biological Interest (ChEBI) ontology. In particular, we are interested in neural-symbolic integration methods that combine Machine Learning with axiomatic knowledge from the ontology. The research project focuses on three avenues: Firstly, we examine how to represent chemical structures in Machine Learning methods, in particular Graph Neural Networks. Secondly, axiomatising ontology classes in monadic second-order logic (MSOL) and first-order logic (FOL) and integrating them with OWL ontologies. And thirdly, we study the direct injection of ontology axioms into the training process of Machine Learning methods. The overarching goal of this work is to provide ontology developers and domain experts with a suite of tools that lighten the load of manual ontology development and broaden the scope of reference ontologies without lowering quality standards.
Ontologies in the biomedical domain, such as those of the OBO Foundry [
Therefore, this research project focuses on developing and improving methods for automated ontology extension, i.e., the addition of content to ChEBI, or the application of ChEBI to new data.
A major challenge for ontology extension is staying consistent with the manually curated ontology. Each ontology term is the result of a consensus between experts and refers to background knowledge that is not always made explicit. This requires neural-symbolic techniques that can integrate knowledge from symbolic sources, e.g., the OWL axioms of the ontology, as well as sub-symbolic sources, i.e., the considerable amount of chemicals that are already annotated by ChEBI.
Given the diversity of the chemical domain, we believe there is no one-size-fits-all solution for
ontology extension. Instead, our goal is to provide an ensemble of diferent approaches. In previous
work, various Machine Learning methods have been applied to this task, including LSTMs [
This author has also participated in preliminary work on a logic-based approach to ontology extension
using SMILES [
In this project, we aim to answer the following research questions: 1. Can a graph representation of molecules and feature augmentation improve the performance of
Machine Learning methods for ontology extension? 2. How can complex chemical definitions be formalised? And how can such formalisations be used for ontology extension? 3. Can ontology extension be improved by directly infusing the training process of Machine Learning methods with ontology axioms?
The first question that has to be addressed when using Machine Learning for the chemical domain is: How do we get molecular structures into a model? And which model provides the best representation for a molecule?
Fingerprints encode molecules as fixed-length vectors, taking the structure and physicochemical
properties into account. Depending on the application, e.g., toxicity prediction or virtual screening,
diferent types of fingerprints have been developed [
In previous work, both fingerprints for classical Machine Learning methods and SMILES strings
for Transformer models have been used [
We hypothesise that a graph representation in which each atom is translated to a node and each bond
becomes an edge would avoid such problems and facilitate the learning process. This representation
also requires a new model architecture, namely Graph Neural Networks (GNNs) [
In our work with ChEBI, we have identified several areas which are underspecified. For example, about 15 thousand compounds are classified as belonging to the peptide class. This class has a subhierarchy which allows for a more fine-grained classification, e.g., into oligo- and polypeptides (peptides with either “few” or “many” amino acids). These two subclasses, by their definition, form a partition of the peptide domain. This means that all peptide compounds could be classified into one of the two subclasses. However, there are 7,500 direct children of peptide which do not have an is a axiom to one of the subclasses. Such missing axioms make learning more dificult for Machine Learning models. For one, there is simply less data to train on, i.e., less positive samples for the classes oligopeptide and polypeptide. But there are also more negative samples, i.e., molecules that are chemically oligo- or polypeptides, but have a negative label in the Machine Learning dataset.
Therefore, in this project, peptides and 13 of its related classes or subclasses (cf. Figure 2) have been
selected to study a rule-based approach to ontology extension [
The classification methodology that is necessary for this process is based on an MSOL axiomatisation.
While [
This raises a new issue: How can one use second-order definitions in classification tasks? We address this issue with a methodology that translates the MSOL definitions into a FOL model checking problem in which some components are calculated algorithmically. Here, the central idea is that reasoning over the whole ontology is not necessary. Instead, each model checking problem has a single molecule as its domain. With this domain, which consists of the molecule’s atoms and the algorithmically supplied components, model checking becomes feasible on the scale of ChEBI.
As a third step, the MSOL and FOL axiomatisations have been used to verify the trustworthiness of a purely algorithmic classifier. With the algorithmic method, we were able to classify all 121 million compounds of PubChem, which goes significantly beyond the scope of ChEBI.
In future work, we will generalise our methodology and extend it towards other areas of ChEBI.
One of the drawbacks of our current Machine Learning pipeline is that it is agnostic about the OWL axiomatisation of ChEBI. The only axioms that are used are the subsumption relations which connect label classes to compound classes. From this, the dataset is constructed, taking each compound as a sample with a list of positive or negative labels. Importantly, transitive parents are used as positive labels as well. Thus, given a peptide for instance, the model not only has to predict the peptide class, but also the amide class, the organic class and so on.
This ignores the subsumption relations between label classes (e.g., we can infer from the ontology that each peptide is organic and an amide). It also ignores disjointness and other relations between classes. Therefore, a model might predict that a molecule is a peptide and not an amide, or that it is both organic and inorganic. While it will still receive a loss for such predictions during training, this loss teaches the ontology axioms only by example, not as a general rule.
In [
In addition, we performed experiments with an additional unsupervised learning task. There, the fact is used that fuzzy loss works without labels: Even if the correct classification is unknown, we can tell if a classification is consistent or not. Data from PubChem was used to augment our labelled dataset with additional unlabelled samples. The goal was to improve the out-of-distribution generalisation abilities of the model by drawing data from a wider distribution than the original dataset.
The main drawbacks of the fuzzy loss are that, for one, in our experiments, adding fuzzy loss terms
was detrimental to model performance. While a balancing technique reduces the performance gap
between the baseline and fuzzy loss models, a successful application of the fuzzy loss would require
further performance improvements on the classification task. Also, in [
Expanding the fuzzy loss method to other axiom types poses a new challenge: How does the axiom correspond to the loss function? Unlike for subsumption and disjointness axioms, where this is relatively straight-forward (if A and B are disjoint, a model should not predict A and B at the same time), other axiom types are more complex. For instance, ChEBI uses the object property has functional parent for relations between classes where one can be derived from the other by functional modification. This does not give us clear rules we can use for a loss function. Given the knowledge that, lets say, penicillin has the functional parent 6-aminopenicillanic acid, we can derive no statement about individual molecules that belong to either of the classes. One way to change that would be to train a separate model on the prediction of functional modifications, which has been relegated to future work.
For now, this project focuses on a promising axiom type that draws information from a diferent source: has part relations are used to relate classes to chemical parts which they contain. Since the chemical structures are provided by ChEBI, we can identify their parts and compare them to the axioms. Take for example the class carboxylic acid. It has an axiom carboxylic acid has part some carboxy group. If a model would predict a given molecule that has no carboxy group as a carboxylic acid, we would know that this prediction is wrong. We can identify the carboxy group based on the ChEBI class carboxy group, which, despite not being a molecule itself, is annotated with a SMILES string. This SMILES string describes the substructure we have to identify in the molecule.
While ChEBI already includes a subhierarchy for groups with more than 3,000 members, the parthood axioms linking them to molecule classes are relatively sparse. We aim to expand the coverage of ChEBI in this area, focussing specifically on the classes on which Machine Learning models are trained.
In a separate approach, we also plan to use parthood relations for feature augmentation. For individual samples, we can directly annotate molecules with the groups that are part of them. This does not require a class-level axiomatisation. Instead, the hypothesis is that the groups themselves are chemically relevant substructures. Since the groups in ChEBI have been selected by expert curators, giving this knowledge to Machine Learning models might induce them to learn concepts in a way similar to how experts usually see them, namely in terms of functional groups.
Finally, this project will investigate the use of Logical Neural Networks (LNNs, [
This work has been funded by the Deutsche Forschungsgesellschaft (DFG, German Research Foundation) under grant number 522907718.
The authors have not employed any Generative AI tools.