Both symbolic knowledge representation systems and machine learning techniques, including arti¯cial neural networks, play a signi¯cant role in Arti¯cial Intelligence. Interweaving these techniques, in order to achieve adaptation and robustness in classi¯cation problems is of great importance. In this paper we present a novel architecture that can provide e®ective connectionist adaptation of ontological knowledge. The proposed architecture can be used to improve performance of multimedia analysis and man machine interaction systems.
Intelligent systems based on symbolic knowledge processing, on the one hand,
and arti¯cial neural networks, on the other, di®er substantially. Nevertheless,
they are both standard approaches to arti¯cial intelligence and it is very
desirable to combine the robustness provided by neural networks, especially when
data are noisy, with the expressivity of symbolic knowledge representation. This
has contributed decisively to the growing interest in developing neural-symbolic
systems [
In this paper we focus on developing connectionist adaptation of ontological knowledge. Section 2 presents the proposed architecture that mainly consists of the formal knowledge and the knowledge adaptation components, which are described in sections 3 and 4 respectively. Conclusions and ongoing research involving semantic multimedia analysis applications are reported in section 6. Knowledge stores the knowledge base components that describe the problem under analysis. More speci¯cally, the Ontologies module formally represents the general knowledge about the problem (TBox) generated during the Development Phase by knowledge engineers and experts.
Moreover, the Formal Knowledge contains the World Description that is actually a representation of all objects and individuals of the world, as well as their properties and relationships in terms of the Ontology (ABox). It is evident that most of the above data involve di®erent types of uncertain information and, thus, they can be represented as formal (fuzzy) assertions connecting the objects and individuals of the world with the concepts and relationships of the Ontology. This operation is performed by the Semantic Interpretation module.
In real environments however, this is a rather optimistic claim. Unfortunately, there may be lot of reasons that cause inconsistencies in the Formal Knowledge. For example, it is impossible to model all speci¯c environments and thus, in some cases, con°icting assertions can arise. In such cases, the Knowledge Adaptation component of the system tries to resolve the inconsistency through a recursive learning process. Adaptation improves the knowledge of system by changing the world description and to some degree the axioms of the terminology. The new information as represented in a connectionist model and, with the aid of learning algorithms, is adapted and then re-inserted in the knowledge base through the Knowledge Extraction and Semantic Interpretation modules for system adaptation.
The focus of the proposed system architecture in Figure 1 is on adaptation of the
knowledge base, so as to deal with contextual information and raw data
peculiarities obtained from multimodal inputs. In the paper we deal with interweaving
of formal knowledge representation and neural-symbolic integration. In
particular, we use recent results that extract parameter kernel functions for individuals
within ontologies [
The family of kernel functions kpF : Ind(A)£Ind(A) ! [0; 1], for a knowledge base K = hT; Ai consisting of the TBox T (set of terminological axioms of concept descriptions) and the ABox A (assertions on the world state); Ind(A) indicates the set of individuals appeared in A, and F = fF1; F2; : : : ; Fmg is a set of concept descriptions. These functions are de¯ned as the Lp mean of the, say m, simple concept kernel functions ·i , i = 1; : : : ; m, where, for every two individuals a,b, and p > 0, ·i(a; b) = 8 1 (Fi(a) 2 A ^ Fi(b) 2 A) _ (:Fi(a) 2 A ^ :Fi(b) 2 A) <
0 (Fi(a) 2 A ^ :Fi(b) 2 A) _ (:Fi(a) 2 A ^ Fi(b) 2 A) : 21 otherwise 8a; b 2 Ind(A) kpF (a; b) :=
i=1 · Xm ¯¯ ·i(a; b) p¯¸1=p
¯ ¯ m ¯ (1) (2)
The rationale of these kernels is that similarity between individuals is determined by their similarity with respect to each concept Fi , i.e, if they both are instances of the concept or of its negation. It has been shown that kpF is a valid kernel, based on its composition through simpler matching kernels and on the closure property with respect to sum, multiplication by a constant and kernel multiplication.
It should be stressed that the reasoning engine, included in Figure 1, is of
major importance for the whole procedure, because it assists the operation of all
knowledge related components. First, during the knowledge development phase,
it is responsible for enriching manual generation of concepts and relations. In
the operation phase, it interacts with the semantic interpretation layer and the
connectionist system for knowledge adaptation to local environments. Both crisp
and fuzzy reasoners can form this engine, we use the FIRE engine [
We use fuzzy reasoning because a fuzzy assertional component permits more
detailed descriptions of a domain. In order to compute (1), (2) the greatest lower
bound (GLB) reasoning service of FiRE de¯ned in [
In the proposed architecture of Figure 1, let us assume that the set of
individuals (and corresponding features), that have been used to generate the formal
knowledge, is provided, by the Semantic Interpretation Layer, to the
Knowledge Adaptation component. Support Vector Machines constitute a well known
method which can be based on kernel functions[
Let the system be in its -real life- operation phase. Due to local or user
oriented characteristics, real life data can be quite di®erent from those of the
individuals used in the training phase; thus they may be not well represented by
the existing formal knowledge. Whenever a new individual is presented to the
system, it should be related, through the kernel function to each individual of
the knowledge base w.r.t to a speci¯c concept - category; the input data domain
is, thus, transformed to another domain - taking into account the semantics that
have been inserted to the kernel function. However, the kernel function in (1), (2)
is not continuous w.r.t individuals. Consequently, the values of the kernel
functions when relating a new individual to any existing one should be computed.
To cope with this problem, it is assumed that the semantic relations, that are
expressed through the above kernel functions, also hold for the syntactic
relations of the individuals, as expressed by their corresponding low level features,
estimated and presented at the system input. Under this assumption, a feature
based matching criterion using a k-means algorithm, is used to relate the new
individual to each one of the cluster centers w.r.t the low level feature vector.
The SVM or Kernel Perceptron can be retrained - including the new individuals
in the training data set, while getting the corresponding desired responses by
the User or by the Semantic Interpretation Layer - adapting its knowledge to
the speci¯c context and use. An hierarchical, multilayer kernel perceptron, the
input layer of which is identical to the trained kernel perceptron can also be used
[
Knowledge extraction from trained neural networks has been a topic of
extensive research [
In this paper we presented a novel architecture for connectionist adaptation
of ontological knowledge. We are currently performing experimentation of the
system performance for solving an image/video segmentation and classi¯cation
problem [