This paper addresses a major weakness of current technologies for the Semantic Web, namely the lack of a principled means to represent and reason about uncertainty. This not only hinders the realization of the original vision for the Semantic Web, but also creates a barrier to the development of new, powerful features for general knowledge applications that require proper treatment of uncertain phenomena. We propose to extend OWL, the ontology language recommended by the World Wide Web Consortium (W3C), to provide the ability to express probabilistic knowledge. The new language, PR-OWL, will allow legacy ontologies to interoperate with newly developed probabilistic ontologies. PR-OWL will move beyond the current limitations of deterministic classical logic to a full first-order probabilistic logic. By providing a principled means of modeling uncertainty in ontologies, PR-OWL will serve as a supporting tool for many applications that can benefit from probabilistic inference within an ontology language, thus representing an important step toward the W3C's vision for the Semantic Web.
# ·The author's affiliation with The MITRE Corporation is provided for identification purposes only, and is not intended to convey or imply MITRE's concurrence with, or support for, the positions, opinions or viewpoints expressed by the author. being exchanged. The SW is meant to fill this gap, and the realization of its goals will require major improvements in technologies for data exchange.
One of the main technical differences between the current World Wide Web and the Semantic Web is that while the first relies on syntactic-only protocols such as HTTP and HTML, the latter adds meta-data annotations as a means to convey shared, precisely defined terms. That is, semantic awareness is exploited to improve interoperability among Web resources. Semantic interoperability requires shared repositories of precisely defined concepts. Such repositories are called ontologies.
One can find many different definitions for the concept of ontology applied to information systems, each emphasizing a specific aspect its author judged most important. Our focus is on ontology’s role as a structured form of knowledge representation. Thus, we define an ontology as an explicit, formal representation of knowledge about a domain of application. This includes: types of entities that exist in the domain, properties of those entities, relationships among entities, and processes and events that happen with those entities. In this definition, the term entity refers to any concept (real or fictitious, concrete or abstract) that can be described and reasoned about within the domain of application. Ontologies are used for the purpose of comprehensively describing knowledge about a domain in a structured and sharable way, ideally in a format that can be read and processed by a computer.
Semantically aware schemes must be able to represent and appropriately process semantic differences between syntactically identical terms (e.g., “Grape” as a fruit versus John Grape the person). This is not a trivial task. Semantic interoperability requires shared sources of precisely defined concepts, which is exactly where ontologies play a key role. Yet, a traditional ontology can at best list multiple possible senses for a word such as “Grape,” with no ability to grade their relative plausibility in a given context. This is inadequate for an open world environment where incomplete information is the rule and plausible reasoning is required.
Current generation Semantic Web technology is based on classical logic, and is
lacks adequate support for plausible reasoning. For example, OWL, a W3C
Recommendation [
A similar historical progression occurred in Artificial Intelligence (AI). From its
inception, AI has struggled with how to cope with incomplete information. Although
probability theory was initially neglected due to tractability concerns, graphical
probability languages changed things dramatically [
This path is not yet being followed. The lack of support for representing and reasoning with uncertain, incomplete information seriously limits the ability of current Semantic Web technologies to meet the requirements of the Semantic Web. Our work is an initial step toward changing this situation. We aim to establish a framework that enables full support for uncertainty in the field of ontology engineering and, as a consequence, for the Semantic Web. In order to achieve this goal, we choose to focus on extending OWL so it can represent uncertainty in a principled way.
One of the main reasons why Semantic Web research is still focused on deterministic
approaches has been the limited expressivity of traditional probabilistic languages.
There is a current line of research focused on extending OWL so it can represent
probabilistic information contained in a Bayesian Network (e.g. [
Another popular option for representing uncertainty in OWL has been to focus on
OWL-DL, a decidable subset of OWL that is based on Description Logics [
Description logics are highly effective and efficient for the classification and
subsumption problems they were designed to address. However, their ability to represent
and reason about other commonly occurring kinds of knowledge is limited. One
restrictive aspect of DL languages is their limited ability to represent constraints on the
instances that can participate in a relationship. As an example, suppose we want to
express that for carnivore to be a threat to another carnivore in a specific type of
situation it is mandatory that the two individuals of class Carnivore involved in the
situation are not the same. Making sure the two carnivores are different in a specific
situation is only possible in DL if we actually create/specify the tangible individuals
involved in that situation. Indeed, stating that two “fillers” (i.e. the actual individuals of
class Carnivore that will “fill the spaces” of concept carnivore in our statement) are
not equal without specifying their respective values would require constructs such as
negation and equality role-value-maps, which cannot be expressed in description
logic. While equality role-value-maps provide useful means to specify structural
properties of concepts, their inclusion makes the logic undecidable [
Although the above approaches are promising where applicable, a definitive solution for the Semantic Web requires a general-purpose formalism that gives ontology designers a range of options to balance tractability against expressiveness.
Pool and Aiken [
The acknowledged standard for logically coherent reasoning under uncertainty is
Bayesian probability theory. Bayesian theory provides a principled representation of
uncertainty, a logic for combining prior knowledge with observations, and a learning
theory for refining the ontology as evidence accrues. The logical basis for PR-OWL is
MEBN logic [
MEBN inference begins when a query is posed to assess the degree of belief in a target random variable given a set of evidence random variables. We start with a generative MTheory, add a set of finding MFrags representing problem-specific information, and specify the target nodes for our query. The first step in MEBN inference is to construct a situation-specific Bayesian network (SSBN), which is a Bayesian network constructed by creating and combining instances of the MFrags in the generative MTheory. When each MFrag is instantiated, instances of its random variables are created to represent known background information, observed evidence, and queries of interest to the decision maker. If there are any random variables with undefined distributions, then the algorithm proceeds by instantiating their respective home MFrags. The process of retrieving and instantiating MFrags continues until there are no remaining random variables having either undefined distributions or unknown values. A SSBN may contain any number of instances of each MFrag, depending on the number of entities and their interrelationships. Next, a standard Bayesian network inference algorithm is applied. Finally, the answer to the query is obtained by inspecting the posterior probabilities of the target nodes.
MEBN logic overcomes the limitations of the attribute-value representation of standard BNs. To understand this limitation, consider a relational database in which some entries are uncertain. A BN can represent only probabilities for a single table, and treats the rows of the table independently of each other. For example, in a medical system, the “Patient” table might include information such as age, smoking history, family history, and whether the patient has emphysema. A BN might represent the probability of emphysema as a function of smoking history, age, and family history. If a patient’s family history were unknown, the BN could estimate the probability of emphysema using probabilities for the family history. However, a BN cannot represent relational information such as the increase in the probability of emphysema for all siblings upon learning that one of their parents had emphysema. To incorporate this kind of knowledge in a coherent manner, we need to combine relational knowledge (e.g., siblings have the same family history) with attribute-value knowledge (e.g., family history of emphysema increases the likelihood of emphysema).
To draw generalizations about individuals related in various ways, we need
firstorder expressive power. Description logics are attractive because they provide limited
first-order expressivity, yet certain reasoning problems such as classification and
subsumption are decidable. Many researchers have worked to identify decidable
classes of problems for which efficient probabilistic algorithms exist (e.g., Naïve
Bayes classification, in which features are modeled as conditionally independent
given an object’s class). The ontology language P-SHOQ(D) [
We have chosen to base PR-OWL on MEBN logic because of its expressiveness: MEBN can express a probability distribution over models of any finitely axiomatizable first-order theory. As a consequence, there are no guarantees that exact reasoning with a PR-OWL ontology will be efficient or even decidable. On the other hand, a future objective is to identify restricted sub-languages of PR-OWL specialized to classes of problems for which efficient exact or approximate reasoning algorithms exist. It is our view that a general-purpose language for the Semantic Web should be as expressive as possible, while providing a means for ontology engineers to stay within a tractable subset of the language when warranted by the application.
Before presenting our probabilistic ontology language, we begin by defining a probabilistic ontology. Intuitively, an ontology that has probabilities attached to some of its elements would qualify for this label, but such a limited definition is inadequate for our purposes. Merely adding probabilities to concepts does not guarantee interoperability with other ontologies that also carry probabilities. More is needed than syntax for including probabilities if we are to justify a new category of ontologies.
A probabilistic ontology is an explicit, formal knowledge representation that expresses knowledge about a domain of application. This includes: (i) Types of entities that exist in the domain; (ii) Properties of those entities; (iii) Relationships among entities; (iv) Processes and events that happen with those entities; (v) Statistical regularities that characterize the domain; (vi) Inconclusive, ambiguous, incomplete, unreliable, and dissonant knowledge related to entities of the domain; and (vii) Uncertainty about all the above forms of knowledge. In this definition, the term entity refers to any concept (real or fictitious, concrete or abstract) that can be described and reasoned about within the domain.
Probabilistic Ontologies are used for the purpose of comprehensively describing knowledge about a domain and the uncertainty regarding that knowledge in a principled, structured and sharable way, ideally in a format that can be read and processed by a computer. They also expand the possibilities of standard ontologies by introducing the requirement of a proper representation of the statistical regularities and the uncertain evidence about entities in a domain of application. 5 PR-OWL PR-OWL is an extension that enables OWL ontologies to represent complex Bayesian probabilistic models in a way that is flexible enough to be used by diverse Bayesian probabilistic tools based on different probabilistic technologies. That level of flexibility can only be achieved using the underlying semantics of first-order Bayesian logic, which is not a part of the standard OWL semantics and abstract syntax. Therefore, it seems clear that PR-OWL can only be realized via extending the semantics and abstract syntax of OWL. However, in order to make use of those extensions, it is necessary to develop new tools supporting the extended syntax and implied semantics of each extension. Such an effort would require commitment from diverse developers and workgroups, which falls outside our present scope.
Therefore, in this initial work our intention is to create an upper ontology to guide
the development of probabilistic ontologies. Daconta et al. define an upper ontology
as a set of integrated ontologies that characterizes a set of basic commonsense
knowledge notions [
The PR-OWL upper ontology for probabilistic systems consists of a set of classes, subclasses and properties that collectively form a framework for building probabilistic ontologies. The first step toward building a probabilistic ontology in compliance with our definition is to import into any OWL editor an OWL file containing the PR-OWL classes, subclasses, and properties.
From our definition, it is clear that nothing prevents a probabilistic ontology from being “partially probabilistic”. That is, a knowledge engineer can choose the concepts he/she wants to include in the “probabilistic part” of the ontology, while writing the other concepts in standard OWL. In this case, the “probabilistic part” refers to the concepts written using PR-OWL definitions and that collectively form a MEBN Theory. There is no need for all the concepts in a probabilistic ontology to be probabilistic, but at least some have to form a valid MEBN Theory. Of course, only the concepts that are part of the MEBN Theory will be subject to the advantages of the probabilistic ontology over a deterministic one.
The subtlety here is that legacy OWL ontologies can be upgraded to probabilistic ontologies only with respect to concepts for which the modeler wants to have uncertainty represented in a principled manner, make plausible inferences from that uncertain evidence, or to learn its parameters from incoming data via Bayesian learning. While the first two are direct consequences of using a probabilistic knowledge representation, the latter is a specific advantage of the Bayesian paradigm, where learning falls into the same conceptual framework as knowledge representation.
The ability to perform probabilistic reasoning with incomplete or uncertain information conveyed through an ontology is a major advantage of PR-OWL. However, it should be noted that in some cases solving a probabilistic query might be intractable or even undecidable. In fact, providing the means to ensure decidability was the reason why the W3C defined three different version of the OWL language. While OWL Full is more expressive, it enables an ontology to represent knowledge that can lead to undecidable queries. OWL-DL imposes some restrictions to OWL in order to eliminate these cases. Similarly, restrictions of PR-OWL could be developed that limit expressivity to avoid undecidable queries or guarantee tractability. Possible restrictions to be considered for an eventual PR-OWL Lite include (i) constraining the language to classes of problems for which tractable exact or approximate algorithms exist; (ii) restrict the representation of the conditional probability tables (CPT) to express a tractable and expressive subset of first-order logic; and/or (iii) to employ a standard semantic web language syntax to represent the CPTs (e.g. RDF). As an initial step, we chose to focus on the most expressive version of PR-OWL, which does not have expressivity restrictions and provides the ability to represent CPTs in multiple formats.
An overview of the general concepts involved in the definition of a MEBN Theory in PR-OWL is depicted in Figure 1. In this diagram, the ovals represent general classes; and arrows represent major relationships between classes. A probabilistic ontology must have at least one individual of class MTheory, which is a label linking a group of MFrags that collectively form a valid MEBN Theory. In actual PR-OWL syntax, that link is expressed via the object property hasMFrag (which is the inverse of object property isMFragIn).
Individuals of class MFrag are comprised of nodes, which can be resident, input, or context nodes (not shown in the picture). Each individual of class Node is a random variable and thus has a mutually exclusive and collectively exhaustive set of possible states. In PR-OWL, the object property hasPossibleValues links each node with its possible states, which are individuals of class Entity. Finally, random variables (represented by the class Nodes in PR-OWL) have unconditional or conditional probability distributions, which are represented by class Probability Distribution and linked to its respective nodes via the object property hasProbDist.
The scheme in Figure 1 is intended to present just a general view and thus fails to show many of the intricacies of an actual PR-OWL representation of a MEBN Theory. Figure 2 shows an expanded version conveying the main elements in Figure 1, their subclasses, the secondary elements that are needed for representing a MEBN Theory and the reified relationships that were necessary for expressing the complex structure of a Bayesian probabilistic model using OWL syntax.
Reification of relationships in PR-OWL is necessary because of the fact that
properties in OWL are binary relations (i.e. link two individuals or an individual and a
value), while many of the relations in a probabilistic model include more than one
individual (i.e. N-ary relations). The use of reification for representing N-ary relations
on the Semantic Web is covered by a working draft from the W3C’s Semantic Web
Best Practices Working Group [
Although the scheme in Figure 2 shows all the elements needed to represent a
complete MEBN Theory, it is clear that any attempt at a complete description would
render the diagram cluttered and incomprehensible. A complete account of the
classes, properties and the code of PR-OWL that define an upper ontology for
probabilistic systems is given in [
In its current stage, PR-OWL contains only the basic elements needed to represent any MEBN theory. Such a representation could be used by a Bayesian tool (acting as a probabilistic ontology reasoner) to perform inferences to answer queries and/or to learn from newly incoming evidence via Bayesian learning.
However, building MFrags and all their elements in a probabilistic ontology is a
manual, error prone, and tedious process. Avoiding errors or inconsistencies requires
very deep knowledge of the logic and of the data structure of PR-OWL. Without
considering the future paths to be followed by research on PR-OWL (i.e. whether it
will be kept as an upper ontology or transformed into an actual extension to the OWL
language), the framework discussed here and in greater detail in [
This brief idea of an operational concept barely scratches the surface of the many possibilities for the technology presented here. Implementing a plugin such as the one envisioned here is a development task that is a topic for future research. Nonetheless, the PR-OWL upper ontology definitions take an important first step toward making probabilistic ontologies a reality. By opening the door to wide use of PR-OWL probabilistic ontologies, the present research makes a significant contribution to realizing the Semantic Web vision.
This paper describes a coherent, comprehensive probabilistic framework for the Semantic Web, that provides a means of representing probabilistic knowledge and providing web services such as plausible inference and Bayesian learning. The proposed framework is an initial step towards a more comprehensive effort focused on representing uncertainty in the Semantic Web.
A PR-OWL plugin for current OWL ontology editors is a priority for future efforts. The process of writing probabilistic ontologies can be greatly improved via automation of most of the steps in the ontology building, not only for defining MFrags to represent sets of related hypotheses, but also for consistency checking, reified relations and other tasks that demand unnecessary awareness of the inner workings of the present solution. Once implemented, such a plugin has the potential to make probabilistic ontologies a natural, powerful tool for the Semantic Web.
Finally, the most important requirement for adoption of a language is the standardization process. This process goes significantly beyond academic research and thus falls outside the scope of the present work. Nonetheless, we are confident of its feasibility, which we believe we have demonstrated in this effort, and of its desirability, given its potential to help solve many of the obstacles that stand in the way of realizing the W3C’s vision for the Semantic Web.