Ontology development is becoming increasingly mature and as a consequence the question and challenge arise on how to handle larger amounts of knowledge and data in automated ways. Current approaches on constructing knowledge graphs are based on machine learning, especially natural language processing. They are improving in quality and increasingly add quality assurance aspects to ensure better usability of the data and semantics for machines. However, none of those currently make use of consistency checking in order to ensure adherence to a consistent schema. In this work, we provide a new approach that allows for continuous and better quality assurance by integrating reasoner and human in the loop into the knowledge graph construction process. We show our approach, system and an evaluation of the results of our system engineering and electronics knowledge graph. The results show an improved knowledge graph construction according to the metrics of conciseness (succinctness) and consistency (coherency) of up to 67,4 % and our method decreases development time to train the neural network.
Ontologies are a central part of semantic web and knowledge management. They provide a
structured and formal representation of knowledge, which enables machines to reason about
this knowledge. In the context of semantic web, ontologies serve as a common vocabulary for
describing concepts, relationships, and properties of various domains. Constructing knowledge
graphs is a challenge because (1) we are dealing with large amounts of data which are hard
to oversee, supervise and understand. And because (2) the quality of those graphs is often
not good from the ontology engineers perspective. Usually graphs generated by for example
GraphGPT [
Reasoning in the Semantic Web usually has three purposes:
1. checking for concept consistency of definitions or in general of classes and their
relationships for example with complex class expressions
2. adding implicit knowledge via axioms with for example transitive object properties,
hasValue restrictions, inverse object properties and others
3. adding if-then conclusion via rules which can be qualitative, but also quantitative with
for example SWRLs Math Bult-In
In this paper we apply reasoning in the context of purpose (1). We construct a knowledge
graph based on a vocabulary called GBO (GENIAL! Basic Ontology) [
Basic conceptualization of the microelectronics domain in regard to automotive. Basic classes include hardware, software, systems, quantities, functions, dependencies, roadmap and context knowledge. GBO is based on BFO and the ISO26262 standard in electronic safety. Figure 1 is a simplified and shortened version of the relevant classes and axioms to understand the approach in this paper. As we can see, both hardware and software are structured into hierarchical levels. Each level is the direct next hierarchical level, indicated by the "has_parts_directly" object property. Further, all software and hardware hierarchical levels are disjoint. Further hardware, software, properties and functions are also each disjoint classes. It needs to be noted that the parts are structured with "only" or universal (all) restrictions, which are more restricting and make the approach in section 4 more efective. Figure 1 as well as 7 were created with Chowlk Visual Notation. For symbol clarification we refer to the documentation. 1
We used datasets from related articles in the domain for both tagging for Bi-LSTM neural network as well as automatic transformer-based methods. These include Wikipedia articles like "central processing unit", "analog digital converter", "automatic cruise control", and comprises currently around 20 articles. The articles were selected to represent the domain of electronics and its most prevalent terms.
Combining the ML methods and our ontology reference model we construct our microelectronics
knowledge graph according to the methodology in [
This paper is structured as follows: In section 3 we introduce state of the art in the related ifeld and draw comparisons and conclusions to our work. In 4 we show the approach and some 1https://chowlk.linkeddata.es/notation.html implementation details, in section 5 we show the results, outline an evaluation and finish with a discussion. In section 6 we conclude and mention future work.
Building knowledge graphs either manually with a systematic requirements and knowledge
engineering approach or automatically using algorithms and programming or machine learning
is a focus of past and current research [
Some approaches do automate the construction process in using machine learning approaches as in [6]. They populate an ontology with a relation extraction to a certain domain from text. Another approach in [7] constructed ontologies from online ontologies. Further ontology ranking, mapping, merging and evaluation were used in addition and in complement. Automating ontology creation can increase ontology adoption, use and put creating knowledge graphs in the hands of many due to its added value. Those ontologies however may sufer from inconsistencies and contradictions. So reasoning is necessary to assure a certain coherence and quality for and of the ontology. Using OWL also provides well defined and formal semantics to represent logical base assumptions of knowledge. OWL is relatively expressive and makes reasoning on it available through well know reasoners like Konclude [8] and Pellet.
Hybrid AI approaches with neural networks and OWL-DL 2 reasoning like proposed in this work are rare. This and the next paragraphs establish a representative baseline and context for current comparable works. [9] uses machine learning to summarize text and combines this with a measure to calculate several centrality metrics, which improves finding and selecting the best node / class. Instead of using metrics, we employ OWL-DL 2 reasoning to find more accurate classes and solve contradictions. Some related work employs graph embeddings for neural networks [10, 11]. Others usually construct the knowledge graphs without reasoning. [12, 13] are some selected examples. [14] does a kind of reasoning in an interactive navigation 2https://drive.google.com/file/d/1A3Y8T6nIfXQ_UQOpCAr_HFSCwpTqELeP/view robot system with question answering by matching a commonsense ontology with a home ontology and inferring relationships and classes.
There are various reasons that approaches like this one are rare. To name probably the first
one is reasoning speed with large graphs and networks which is a challenge [15]. Further
most ontologies are not build expressively and are rather used as domain models, where the
atomic class is the central part. For example, [16] and [17] give an overview of reasoning
over knowledge graphs and there are other contributions, which focus on diferent kinds of
reasoning, e.g. rule based reasoning [
In this section we will go step by step through the concept and the framework. Figure 3
illustrates the approach. We start in the middle with the knowledge graph stored in OWL. The
3https://www.emse.fr/˜zimmermann/KGBook/Multifile/quality-assessment/
graph needs to be represented in OWL in order to execute reasoning on it. We translate from
two formats into OWL. On the right to the knowledge graph we see the "System Markdown
Language" [
The transformer finally based on the dataset generates predictions on new input text and with a python library creates a pickle (.pkl) graph. We export the pickle graph into JSON. The second format then is JSON, which again gets translated into the OWL knowledge graph. Relationships that are generated from JSON to OWL are all established using the "some" restriction. In combination with the disjoint classes and the "only" restrictions from the vocabulary the reasoning is accomplished. For the translation we used Owlready2, which is similar to the OWLAPI, but based on python instead of Java. This then closes the loop. The output of our natural language processing (NLP) model provides us with triples comprising a head, type, and tail. Additionally, we leveraged the results of the tagging process from the GBO ontology to establish basic subclass/superclass relations. For example, classes that were tagged were assigned those tags as superclasses in the graph. Basically, if the term cpu is tagged as a hardware part, then the term cpu will have the hardware part class from GBO as its parent class. Some other subclass assignments were established that create obvious taxonomic relationships. The following sums up the approach for generating graphs: 1. Feeding the transformer model REBEL our dataset 2. Adding the classes, instances and object properties between classes to our ontology 3. Reason on the ontology to extract the machine-generated consistent ontologies 4. Store the parts of the graph in OWL files as well as Neo4j database 5. Update the dataset according to the reasoning results
Wikipedia articles and other scientific text pieces, that dealt with the microelectronic domain,
were used to train and classify text based on the GBO classes. The first step was to merge the
results of the tagging system and the generated triples. The triples were first generated using a
bidirectional long short-term (bi-LSTM) neural network model. Bi-LSTM models allow inputs
to travel backwards and forward. The hidden layers of the network are connected to the same
output. The task was divided into two parts. The first part was to classify sample occurrences
into our predefined classes. This task could be referred to as Named Entity Recognition (NER)
with our custom GBO classes. The second step was to generate a graph that aligned with the
relationships in the GBO. The NER task used the bi-LSTM model with 5 layers and 21 diferent
types of labels. All hyper-parameters such as the LSTM units and dropouts were adjusted
with experimentations. The training was carried out for 50 epochs with a batch size of 32. We
reached a validation accuracy of 94%. With the predictions of the bi-LSTM, we could generate
graphs. The graphs generator used a rule-based model. We defined a "context" that represented
a small part of the text that kept the interrelated conditions intact. We settled for a context
size of around 10 sentences. The identified (or the predicted) words were then extracted from
the sentences. A simple rule-based model (consisting of a few if-else blocks) was then used to
create relationships between these extracted words [
We also used the transformer to generate graphs. This gave us a better insight into the
results produced by the bi-LSTM model. It also helped us compare and choose the better option
in the end. The transformer [
In this method, we converted an original sentence comprising entities and implicit connections
among them into a group of triples that explicitly referred to all those connections. As a result,
we represented the triples as a group of tokens that the model could decode. In order to
produce the correlations in the sentence as triples and reduce the number of tokens that must be
decoded, we developed a reversible linearization, incorporating special tokens. The data from
the dataset served as the input for REBEL, and the output was the linearized triples. REBEL
[
The objective was to combine the results of the NLP phase (bi-LSTM and the triples generated by REBEL) into a single database and make necessary revisions. Each time a term was tagged and if the triple (term, subclass of, tag) was missing, it would be added to our database. In addition to this, we used the triples from the REBEL model which resulted in a coherent ontology that has a clear base structure. After conducting an analysis of the results of the REBEL model, we identified certain issues that required attention and subsequently resolved them before using the database to build the ontology. At first, the transformer generated two relations - use and uses, so we changed every relation use to uses in the database. Another problem was faulty triples, where one of them is empty, so those triples were deleted. Also, we had some terms that were the same except one was uppercase and the other was lowercase. As a solution, all uppercase letters were changed to lowercase to avoid having two classes representing the same concept. Some classes had multiple inheritances for one superclass causing an "Uncaught Exception in thread ’partitioning’ " error. To resolve this, we deleted the first occurrence of subclass of that resulted in a cycle. This was an adhoc decision and would be evaluated further in the future. So the result of this part is our new database that contains various relations including subclass of relations which would be used next to build the ontology.
We had two approaches for reasoning. The two are slightly diferent but aim for the same results, which is to extract a machine-generated consistent graph and machine-generated inconsistent graph (both abbreviated mgg). And both used the well-known reasoner Konclude. • First implementation: Our first implementation was to reason after the complete graph is created. So after we have our graph built, we reason on all the classes and delete each and every inconsistent class from our machine-generated graph to develop the machine-generated consistent ontology. Then we link those deleted classes to the GBO ontology to create the machine-generated inconsistent graph. The downsides or results of this approach are that some classes that are a subclass of an inconsistent class will be lfagged as inconsistent. Another downside is that some of the GBO classes can still be inconsistent. Thus another approach for the implementation was taken. • Second implementation: Due to the downsides of the first approach, the second one was developed. Instead of reasoning after creating the graph, we reason while creating it. So after adding each class, we check with konclude, if a class / individual is consistent or not. If it’s not consistent, then we delete it from our graph / ontology. But that in itself is not suficient, since we also need to check after adding each relation if one of the axiomatically associated / linked classes became inconsistent. If it did become inconsistent, we removed it as well. Also if the class is from the GBO ontology, we removed the relation between the two classes, instead of deleting the GBO class itself. We have again generated the machine-generated consistent graph file and then we used the deleted classes to build the machine-generated inconsistent graph.
We used labels for synonyms and use natural language spaces, also because the IRI of a class isn’t allowed to include symbols. Further adding labels with natural language spaces, improves the readability and is a best practice. The github repository of the implementation can be found here .5
This ontology graph is not our only result but also we have corrected the tags of our BI-LSTM using the inconsistent ontology file. So every class in the inconsistent ontology file was there because it had the wrong tags. So as a correction we did go through all those classes and then changed the tags in our csv file. This closes our loop. From tagging and creating the triplets to building our ontology to using this ontology to change the tags. By repeating this cycle, we can continuously enhance our outcomes.
One method of closing our feedback loop is to use human interaction. The python script opens exactly in the position of the wrong tag and gives the user options to change the tag manually or to write or to type which tag should be in this position.
As we see in figure 4, at first the exact line 7 was opened then the feedback "hwp" (which is the tag for hardware part) was given so cpu´s tag was changed to B-hwp. Then the exact line 35 was opened and the input "all cpu" was given. Then all the occurrences of cpu will be tagged as hwp. After this the script will go to the next inconsistent class until the user fixes all the incorrect tags. Since the current context is only microelectronic documents, we included that aid, as we expect wrong corrections rather improbable.
Another way to close the loop is done by making machine automatic corrections. Not only is the human in the loop, but the results of the machine, e.g. the reasoning process are considered. 5https://github.com/OntologyResearcher/Supplementarymaterial For example, the reasoner classifies something as inconsistent in the ontology, e.g. a hardware part that needs to be a system. The created inconsistency, which is the wrongly classified class as the tag from the dataset, will then first be removed. By having the loop the dataset is updated and as a consequence the predictions also change and improve, which again influences other tags.
We explore first the results of creating the ontology. We use protege to visualize the ontology, metrics are given in figure 5. The ontology has 3623 classes, 93 object properties, 239 individuals, 17399 axioms, and 8373 "subclass of" relations. Which is considerably larger than the manually developed models for specific use cases, that we were able to create in the past with considerable amounts of invested time and manual efort.
We already introduced an excerpt of the complete graph in Figure 2. Figure 6 shows the "hardware element" or hardware class from GBO. It has "hardware component","hardware elementary subpart" ,"hardware part" and "hardware subpart" as subclasses. Here we show just the hardware part class of the inconsistent graph. We see that this class has many subclasses like adc, clock generator, cpu, crystal resonator, disks flash memory, serial ata and others. Which are often indeed hardware parts, but are inconsistent. Again those classes are machine-created and also machine classified as hardware part.
For example the class "clock generator" is inconsistent, because we have the rule that "hardware component", "hardware elementary subpart", "hardware part" and "hardware subpart" are disjoint classes. And the class clock generator is directly a subclass of "hardware component" and "hardware part." For this reason it is deleted from the main ontology and added to the inconsistent graph file.
We evaluated the results according to the several metrics of conciseness and consistency. Conciseness refers to "avoiding schema and data elements irrelevant for the domain." (see footnote 3) Consistency measures the degree to which "(i.e., inconsistencies) with respect to the particular logical entailment considered." (see footnote 3) In order to handle complexity, we used only the transitive "has_part" relationship for classification rather than the "has_part_ directly" object property. This is equivalent with a less strict sorting out. First, we quantitatively measured the percentage of classes that the reasoner categorized. We counted the inconsistent and consistent classes with proteges metrics and subtracted the classes of both BFO and GBO. Whereas in the whole graph we did not subtract, as we see both ontologies as an integral element; unclassified classes were also counted directly. Table 2 gives an overview. Second, the amount of spared time by the knowledge engineer is significant. As the amount of data and its connections are hard to comprehend. This methods makes this process supervisable. Apart from the number of connections, some evaluations and considerations might simply be missed. The next paragraph gives examples. Third, we decided to qualitatively analyze and evaluate the results and discuss their significance. Here, we exemplify the results based on examining the correctness through disjointness of the classes as well as the categorization through the "has_parts" relationship. Figure 7 gives a sample of a few inconsistencies detected by the reasoner. In the following we will give explanations to each (1-5) case: 1. "moving coil motor" appears to be an unexpected classification by the neural net transformer, which puts it as a subclass of actuator (not hardware component) and contradicts our vocabulary. 2. "rop" appears at first glance to be an abbreviation of a hardware part. Examining the training dataset clears up the reason for the inconsistency. Rop is once classified as an abbreviation of "render output pipelines", a hardware part. And on another occurence as an abbreviation of "return oriented programming", a software. Both terms are merged in our graph and thus create the inconsistency, because the abbreviation is referring to two diferent entities. 3. "calculation pipeline" was identifiably falsely assigned as a function in the tagging dataset by our trainee. The transformer in this case was correct by classifying it as a type of alu (arithmetic logic unit). In combination with our ontology the error was recognized. 4. here we see the "converter" class and its relationships. This class has as a part ("has_part") a clock. For example, in digital-to-analog converters (DACs) and analog-to-digital converters (ADCs), a clock is often used to synchronize the conversion process. Also this class has as a part a comparator. In ADCs, the comparator is used to compare the input analog signal to a reference voltage and generate a digital output based on the result of the comparison. And this class uses some signal processing. In the context of converters, signal processing is used to transform the input signal into a format that can be processed and converted into the desired output signal. The color green indicates that the converter in itself is a consistent class. However, its relationships where removed, since both the comparator and the clock where found to be inconsistent classes. 5. "snr" (signal to noise ratio) was correctly tagged in the dataset as a quanitity and thus correctly placed in the graph. The transformer neural network however, decided due to context to place the snr as a subclass of adc (as well as create the "facet_of" relationship).
An adc is a hardware part though, creating the inconsistency. Overall the dataset was quantitatively improved by 4.7 % in coherency with inconsistency analysis alone. The question of relevancy is of similar importance to improve graph quality. The removing of unclassified classes achieved an improvement of 62,7 % in conciseness. Altogether our method improved our knowledge graph about 67,4 %. Table 3 summarizes the results.
This subsection puts the results into context and discusses them. Unclassified classes might be
part of the correct graph but are just not classified, thus being falsely not classified. We always
also have to weigh the NLP side of the classification when we want to assess a somewhat more
complete or so called "absolute" correctness. It is important to put the results into context and
discuss them to ascertain the meaning and understand the significance. For one we need to
consider the current NLP accuracy of our transformer and training, which depends on the actual
word of the vocabulary and is between 36-78 percent (F1 score, see [
This paper proposed a novel approach to building knowledge graphs with integrated reasoning. The approach sorts out inconsistent classes, relationships and instances and thus improves iftness for purpose of our knowledge graph in context of above discussion in a margin of up to 67,4 percent. By separating the graph into parts, the knowledge engineer is presented with only the erroneous parts, thus simplifies his work and makes it easier to oversee. As the approach is integrated with the dataset, the NLP accuracy is improved automatically by the knowledge engineers revision. The approach is demonstrated in the area of system engineering and electronics, but can easily be extended to other domains. To reapply the approach can already be done with a few disjoint classes that are used for training. Neural networks, such as transformers, large language models, Bi-LSTM or deep learning and machine learning can be applied in general. A more expressive ontology further increases the impact of our approach, as well as a systematic building approach with an upper ontology. Our work has various steps, but we made sure it is reusable for future engineers and people interested in using the approach. What is to be adapted is creating another ground ontology similar to GBO, that suits the aimed domain and tagging with other classes. Then the according automation can be used to finish generating the graph and recreate the mentioned feedback loop. This allows to create various ontologies and graphs in various domains with less inconsistencies. For example using a general "component" definition, which includes all of our parts. That can then also be matched accordingly. In future works we plan to incorporate more quality assurance aspects in automated ways and want to scale our approach. Rules, algorithms and reinforcement learning is further planned automation. [5] N. F. Noy, D. L. McGuinness, Ontology Development 101: A Guide to Creating Your First Ontology, Technical Report, 2001. URL: http://www.ksl.stanford.edu/people/dlm/papers/ ontology-tutorial-noy-mcguinness-abstract.html. [6] A. Ayadi, A. Samet, F. de Bertrand de Beuvron, C. Zanni-Merk, Ontology population with deep learning-based nlp: a case study on the biomolecular network ontology, Procedia Computer Science 159 (2019) 572–581. URL: https://www.sciencedirect. com/science/article/pii/S1877050919313961. doi:https://doi.org/10.1016/j.procs. 2019.09.212, knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 23rd International Conference KES2019. [7] H. Alani, Position paper: Ontology construction from online ontologies, in: Proceedings of the 15th International Conference on World Wide Web, WWW ’06, Association for Computing Machinery, New York, NY, USA, 2006, p. 491–495. URL: https://doi.org/10.1145/ 1135777.1135849. doi:10.1145/1135777.1135849. [8] A. Steigmiller, T. Liebig, B. Glimm, Konclude: System description, Web Semantics: Science, Services and Agents on the World Wide Web 27-28 (2014). doi:10.1016/j.websem.2014. 06.003. [9] G. E. Trouli, A. Pappas, G. Troullinou, L. Koumakis, N. Papadakis, H. Kondylakis, Summer: Structural summarization for rdf/s kgs, Algorithms 16 (2023). URL: https://www.mdpi. com/1999-4893/16/1/18. doi:10.3390/a16010018. [10] M. Gao, J. Lu, F. Chen, Medical knowledge graph completion based on word embeddings, Information 13 (2022). URL: https://www.mdpi.com/2078-2489/13/4/205. doi:10.3390/ info13040205. [11] J. Ma, C. Zhou, Y. Chen, Y. Wang, G. Hu, Y. Qiao, Tecre: A novel temporal conflict resolution method based on temporal knowledge graph embedding, Information 14 (2023). URL: https://www.mdpi.com/2078-2489/14/3/155. doi:10.3390/info14030155. [12] F. Al-Aswadi, H. Y. Chan, K. H. Gan, Automatic ontology construction from text: a review from shallow to deep learning trend, Artificial Intelligence Review 53 (2020) 1–28. doi:10.1007/s10462-019-09782-9. [13] S. Yuan, J. He, M. Wang, H. Zhou, Y. Ren, A review for ontology construction from unstructured texts by using deep learning, in: P. Kar, S. Guan (Eds.), International Conference on Internet of Things and Machine Learning (IoTML 2021), volume 12174 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, 2022, p. 121741D. doi:10.1117/12.2628713. [14] A. Schalkwijk, M. Yatsu, T. Morita, An interactive virtual home navigation system based on home ontology and commonsense reasoning, Information 13 (2022). URL: https: //www.mdpi.com/2078-2489/13/6/287. doi:10.3390/info13060287. [15] B. Parsia, N. Matentzoglu, R. Gonçalves, B. Glimm, A. Steigmiller, The owl reasoner evaluation (ore) 2015 competition report, Journal of Automated Reasoning 59 (2017). doi:10.1007/s10817-017-9406-8. [16] X. Chen, S. Jia, Y. Xiang, A review: Knowledge reasoning over knowledge graph, Expert Systems with Applications 141 (2020) 112948. URL: https://www.sciencedirect. com/science/article/pii/S0957417419306669. doi:https://doi.org/10.1016/j.eswa. 2019.112948. [17] R. Guimarães, A. Ozaki, Reasoning in Knowledge Graphs, in: C. Bourgaux, A. Ozaki,