Our overall goal is to build dialogue systems for rational interaction with technical application systems. We want to achieve the satisfaction of user's goals in a given (ideally open) domain by conducting spoken dialogues where it should be possible in principle to augment them by other forms of multi-modal interaction. We assume that the satisfaction of user goals within the thematic framework of a particular application domain is to be achieved with the help of a dialogue system proper in cooperation with a technical application which we also call the “domain problem solver”. Such a technical application can be an information or reservation system, a system for controlling electronic devices, etc. The knowledge base of the system is represented in a Description Logic, where the terminological part, the formal ontology, is a combination of three concept hierarchies. Its first branch represents the linguistic domain, i.e. lexical semantics and linguistic units, and is therefore language-specific. The second branch represents the discourse domain, containing dialogue-related knowledge, i.e. knowledge about vaious dialogue types like questionanswering or negatiation independent of particular application domains. The third branch, most important by size, represents the application domain concepts and its properties, i.e. the domain model. We describe, how these components are being integrated and used in the dialogue system and address some problems we encountered, in particular with WordNet, which is our lexical taxonomy.
Our overall goal is to build dialogue systems for rational interaction with technical application systems. We want to achieve the satisfaction of user’s goals in a given (ideally open) domain by conducting spoken dialogues where it should be possible in principle to augment them by other forms of multi-modal interaction like gestures or the selection of items from a menu on a screen. Interactions are called “rational” because we apply rationality principles (at the knowledge representation level) to optimally select appropriate communicative actions. For dialogue modelling, we follow a plan-based approach which provides the means to conduct task- or goal-oriented dialogues focussed on accomplishing concrete tasks. We claim that only a general planning approach enables cooperative response behaviour (pragmatic adequateness, overanswering) and the ability for negotiation.
For the interpretation of dialogue, we are clearly commited to a (computational) logic framework, in particular Description Logics (DL). Of course, humans act incoherently and even inconsistently, and common sense reasoning can only to a certain extent be understood in terms of logic, but we are convinced that a coherent and consistent rational reconstruction is the best we can do about it. Such a constructive perspective has the advantage of enabling us to begin with a well understood framework for knowledge representation and reasoning upon which we can attempt to build rule systems for still idealized, but more realistic patterns of argumentation in specific domains. We believe that there is a potential to succeed in a variety of prevailingly instrumentalized contexts as it is the case with technical applications — that will be discussed in more detail below — or, to take up another example, in forensic argumentation.
To a large extent, our research and development work in the field of dialogue systems has been done within the German joint project EMBASSI (“Elektronische Multimediale Bedienund Service-Assistenz”), which aims to provide easy access for everybody to complex technical systems (A/V home theatre, car devices and public terminals), encouraging multi-modal user input. Besides a chunk parser for spoken utterances, our contributions to this project are first, the dialogue manager, second, formal ontologies for several application domains and third, a language generation component to communicate system utterances to the user. 2
Under the assumption that a “language as action” perspective provides a flexible and extensible framework for rational dialogues, whose aim is to satisfy user goals in a given application context, we need means to identify such goals and to represent them formally within an explicit representation of an initial situation. We also need methods to decompose a goal into subgoals to be satisfied by the application system, and to control the satisfaction process. In other words, we formulate a complex planning problem which comprises at least two levels: planning on the dialogue level w.r.t. interactions between the dialogue system and the user, and planning on the level of the application system.
For planning on the dialogue level, we need an explicit representation of dialogue situations
which on the one hand include statements representing what the system could extract from the
interaction with the user so far and on the other hand assumptions about the user’s knowledge of
the actual situation as well as on goals, their subgoals and the actual state of their satisfaction.
The planning process consists in the application of dialogue operations which have
preconditions defining their applicability and assertions about their effect, i.e., how the dialogue situation
develops when they are applied.1 Planning in dialogues is based on partial knowledge. Each
1A general logic-based approach for representing and processing dialogue situations on the epistemic level has
been developed by Cohen, Levesque and others (cf. [
A comprehensive framework for discourse planning has been established by Grosz and Sidner in their pioneering
investigations [
Discourse planning, i.e. the determination of a sequence of dialogue steps, has to take into account that the application subsystem influences that sequence by reacting on preconditions of operations, and generating effects which change the actual state. So, a description of the dialogue step sequence requires representations of time, the “actual state”, the terminology of the application, and the operations, their preconditions and effects. 3
Except for trivial cases, a direct mapping from a user utterance to a system command cannot be accomplished. In general, we have to take complex speech acts into account, where the interpretation of the utterance’s propositional content is determined by its (local) linguistic-pragmatic context in the first place. This, in turn, is to a large extent influenced by (global) discoursepragmatic features which provide constraints based on the dialogue history and the actual place of the utterance in the dialogue, as, e.g., being the expected answer to a question. Furthermore, the application provides further constraints by restricting the meaning of words and phrases to their particular use within a given thematic framework. Therefore, we have to distinguish several — interleaved — levels in the analysis of user utterances:
Linguistic analysis on the utterance-local level, which in turn consists of several levels of
syntactic and semantic construction (cf. [
Discourse-pragmatic analysis — a proper function of the dialogue manager.
For parsing, we build upon “chunks” which provide a first grammatical segmentation of
utterances. Grammatical structure analysis is incrementally tied with the semantic interpretation of
chunks, which in turn consists of three phases: First, we identify the domain-independent word
and intra-chunk semantic information. The second step consists of the grammatical
determination of inter-chunk relations. The third part is to perform semantic construction by means of
construction operations associated with the chunk grammar rules into Discourse Representation
Structures (DRSs). For the latter we use -DRT, a fully compositional derivative of Kamp’s
Discourse Representation Theory (DRT) [
These theoretical studies have been very influential for a lot of systems, e.g. Rich’s et al. COLLAGEN system
[
The ultimate goal in this part of analysis is to transform the domain-independent semantic representation into a description of the discourse situation which is specialized to the respective application domain of our dialogue system. First of all we need to access the domain-specific concepts which are available through a link between the general lexical semantic information and the specific semantics of the application domain in the lexicon. We then have to instantiate the respective domain concepts with discourse referents of the extensional semantics by mapping chunk structures into relations between concept instances. 4
Our fundamental design decision amounts to a clear functional separation between the language model, the dialogue model and the domain model. Accordingly, there is a division of labour between the dialogue manager and the application (the “domain problem solver”), which implies the following interaction steps: the dialogue manager “formulates a task” for the application; the application executes the task; the application decides whether it is necessary to inquire the user; the application sends task results and further inquiries to the dialogue manager such that it can execute appropriate dialogue operations.
As far as the administration of application-specific user goals, and in particular conflict resolution among them is concerned, it has to be provided by the application — as opposed to the administration of dialogue goals cared for by the dialogue manager. Application and dialogue manager are planning separately. The exchange of data must guarantee consistency between the application and the dialogue situation which, of course, requires semantic compatibility. This in turn presupposes that both, dialogue manager and application, have access to the same domain model. Another consequence of the separation is that it leads to a classification of utterances w.r.t. their functionality to change the dialogue situation. 4.1
The underlying formal ontology is a combination of three concept hierarchies. Its first branch represents the linguistic domain, i.e. lexical semantics and linguistic units, and is therefore language-specific. The second branch represents the discourse domain, containing dialoguerelated knowledge, i.e. knowledge about various dialogue types like question-answering or negotiation, independent of particular application domains. The third branch, most important by size, represents the application domain concepts and its properties, i.e. the domain model.
In particular, the application knowledge, which is used in application situation descriptions, consists of concept descriptions of domain objects, and concept descriptions of domain actions. These concepts are instantiated in application situation descriptions that are used to represent which objects of which types currently exist and which actions are possible in the current situation. So, the application concept hierarchy represents formally reconstructed technical or scientific knowledge, combined with elements of common sense under a technical perspective. From the viewpoint of reusability and configurability, it it worthwile to consider a division between a generic base ontology which is suitable for a variety applications, the “upper level”, and a problem specific part, the “lower level”, which is specific to a particular application domain and has to be replaced with any new application. In specific application domains it may be possible — as it is the case for EMBASSI — that a considerable part of the application concept hierarchy, i.e. the device-specific concepts, can be gained automatically from a source provided by the application engineers, which in this special case was given as a Java class hierarchy implementing the device control system2.
Analogously, the dialogue knowledge used in dialogue situation descriptions, is built up from concept descriptions of dialogue objects (expressions, enumeration of alternatives, dialogue goals), and concept descriptions of dialogue actions (speech acts). Dialogue situation descriptions contain instances of those objects; they are extracted from the semantic representation, in our case Discourse Representation Structures (DRSs).
The common roof for the latter both hierarchies consists of a generic base model, for which
we chose the SUMO formal ontology [
The bridge between linguistic analysis and application level planning is a mapping from DRSs composed during parsing to DL ABoxes representing propositions on the current application situation. This means that ABoxes have to be consistent with respect to a given TBox. A rationale for using DL for constructing natural language semantics is the possibility to eliminate hypotheses constructed by the parser if the corresponding ABoxes are inconsistent. 5.1
EUROWORDNET (EWN) In order to ease the adaptability of the dialogue system to different domains and to reflect general and domain independent usage of language from that of a specific application, the semantics of chunks is expressed in terms of concept expressions taken from the EWN terminology. EWN has been developed on the basis of the WORDNET semantic net which — in version 1.5 — encodes the meaning of about 80.000 nouns, 60.000 verbs and 16.000 adjectives and adverbs. Beyond 2HAVi (Home Audio Video Interoperability) which represents a common, openly-licensable specification for networking digital home entertainment products.
3based on WordNet, cf. [
Constraints on complements and modifiers of German words are expressed in terms of case frames which state the valencies of a word and their possible semantic filler types. In general, thematic roles are used in a number of case frames, not just in one. This means, more than one general concept inclusion (GCI) axiom has to be included in the linguistic terminology that is used to encode the use of German words that take complements or modifiers. Thematic roles are defined to be features as the relation between discourse referents is functional.
The interpretation of thematic roles in terms of roles in the application domain is encoded as the application specific part of the case frames describing the language usage in the application domain. Given two DRSs, with the help of an ABox consistency test, one has to validate the application specific reading constructed by the parser. 6
In order to encode the semantics of a natural language expression in our DL domain, we had to search in EWN for this expression, and if it was found, we had to manually follow up the hyperonym chain until we arrived at a super concept that is already defined in our domain, and then begin from that point to encode the subtree we expanded in the last step. This task is time consuming and can be a source of errors, like encoding some concepts with their trees more than once, or forgetting subnodes within a hyperonym chain, not to mention typing mistakes, missing parentheses, etc., which makes the domain-model inconsistent and the processing difficult or rather impossible.
Furthermore, we use our approach to semantics construction in different applications. Consequently, we gathered a huge amount of semantic definitions (i.e. concept chains) and case frames (i.e. thematic roles) defined by these applications. Some of these data are specific to a given application, whereas others are used in several applications at once. This made the need for a tool that enables efficient storage and easy and fast access, as well as preparing the data required by the parser be of prime importance. Therefore, we developed a lexicon tool that helps editing semantic data, checks their coherence and visualizes them as well. 6.1
The lexicon tool can be considered as an interface between our application system and the semantic resources mentioned above, because on the one hand, it stores the expressions used by the different applications and presents them as entries, to which the corresponding case frames — i.e. the valencies in the syntactic (e.g. subject), semantic (e.g. agent), and pragmatic (e.g. user) sense — are assigned and which are needed by the parser. On the other hand, it stores for each entry the underlying semantic concept as it is represented in EWN together with its derivation chain.
The interface provides an easy access to the stored information with the help of navigation tools like pop-up menus, text fields, lists, etc. It also enables the user of adding new entries to the data base and define its word class, syntactic function, thematic role, and semantic concept (after obtaining it from EWN). While doing this the lexicon-tool offers lists with options that help the user determinig the most appropriate category by which the selected gap (text field) can be filled, and in the case of ill-formed or inappropriate input it returns detailed error messages with suggestions for improvement.
One of the most valuable features in our lexicon-tool is the possibility of controlling and checking the coherence of entries both in terms of the complete conceptual hierarchy with regard to our linguistic domain, and appropriate thematic roles with regard to the application domain. So if the user wants to check consistency or dependency relations between some concepts he can do that by typing the required sequence of concepts into the corresponding text field and getting the response after the check performed by the RACER inference server. Similarly, on adding new entries to the data base, if the given concept doesn’t exist or collide with other concepts it won’t be added, and subsequently the tool produces a corresponding error message and propose possible solutions. 7
As already pointed out, our knowledge base contains the complete SUMO ontology encoded in DL, the EWN upper ontology, and the concept definitions specific to EMBASSI applications. However, many SUMO and EWN concepts can be removed from the knowledge base as they are not used by the application specific part. For a performance evaluation, we compiled a big knowledge base consisting of 1165 concept definitions and a large number of additional disjoint statements. During parsing, a consistency check has to be executed which involves the computation of the most specific concepts a DRS head element. With the actual configuration4 that single function call took about two seconds — which is not acceptable for parsing natural language, as, given a complex word lattice, hundreds of such calls have to be performed for parsing one lattice under the constraint of real time behavior of the overall system. Obviously, however, many SUMO and EWN concepts could be deleted from the knowledge base as they were not used by the application specific part of the knowledge base. In an automatic precompilation step, 862 concepts were deleted, which are only defined, but not used as part of another definition — many among them about insects and bacteria which are not considered relevant for the application. The performance test was then repeated, taking only 134 ms. But there is even a better message: The new version 1.7.6 of RACER is even an order of magnitude faster!5 8
We are well aware of methodological problems of the approach taken by the authors of WordNet.
Slodzian [
Missing expressions: EWN is mainly limited to nouns, verbs and adjectives. However, meanings are not just expressed by these elements. Definitions for adverbs, temporal and spatial expressions, function words (e.g. auxiliary verbs, modal verbs, prepositions, etc.), not to mention multi-word elements (e.g. phrasal and prepositional verbs), idioms, collocations, and widely used abbreviations (e.g. “CO” for company) are generally not accounted for in EWN. Therefore we had to expand the linguistic domain model to include concepts for temporal and spatial expressions – to mention only the most prominent ones. It is evident that these elements are essential within the domain of EMBASSI in particular and similar systems in general, because, on the one hand, they function as fillers of roles in the application specific domain, which, in turn, helps determinig the sort of action to be triggered off as a response to an utterance. On the other hand, in a language like German, prepositions, for instance, determine the case of the following noun. This fact can be used to enhance the mechanisms employed for disambiguation and sense-differentiation. Missing or inappropriate senses: Another problem was the case in which the word being searched for already exists in EWN, but not all its senses are defined. A definition of the word “part”, for instance, in the sense of “member of a group” doesn’t exist. Also, the word “subscribe” is only defined in the domain of financial transactions, so when we were searching for the same word in the sense of “being a member or join (e.g. a mailing list or so)” the corresponding definition couldn’t be found. In such cases, we had to get the required sense by using synonymous words, despite the fact that the required word is already defined in EWN but not in all or at least not in the most dominant senses of it. Conceptual gaps: The definitions of some verbs (e.g. contain, glow, test, treat, sweat, apply, charge,...) and most adjectives are so short, that they don’t lead to the superset of all concepts that already exist in EWN. Concequently, gaps in the conceptual hierarchy may arise. In order to fill in the gaps in the hierarchy, we added general concepts like DO, CHANGE, CAUSE, STATE, QUALITY, MODAL-PROPERTY, MENTAL-PROPERTY and others to our knowledge base. On the one hand, these concepts function as subnodes of already defined concepts in EWN, on the other hand, we can derive the required or rather the missing concepts from them.
Long definition chains: In contrast to verbs and adjectives, some nouns have very long derivation chains (see, for example the definitions of “mall”, “tour”, “cloth”, “stuff’), which makes their encoding in DL and hence the consistency control rather difficult, not to mention the storage place and processing time they may take. We by-passed this problem by taking the definition of the underlying synonym, which usually has a shorter derivation chain. A side effect of this stratgy is that some of the semantic properties of the word get lost, which leads to inacurracy in the semantic representation. Also the synonym definitions always imply a kind of generalization, which may be a source of ambiguity. Antonymy: Antonyms that can be regularly built by using some negation prefixes like (un-, in-, anti-, dis-,. . . ), in general, are poorly represented in EWN. For example, the word “subscribe” exists, but not “unsubscribe”, the same holds for “scented” and other words. So it would be very helpful, if EWN would pick up those antonyms as separate entries or rather assign to every word the corresponding antonymous form or prefix, so that a large amount of antonyms is covered in EWN.
Derivations: Like antonyms, many standard (rule-based) derivations are not existent in EWN. To illustrate this, take for example the word “moisturizer”; it is not defined, although the verb “moisturize” already exists. So the possibility to account for derivations either statically or dynamically in EWN is essential for building a uniform and balanced taxonomical hierarchy.
Insufficient syntactic coverage: By “syntactic coverage” we mean syntactic features like valencies of a verb; case, gender, number of nouns, and so forth. Such features are not represented in EWN. In a system for natural language processing these features are essential not only on the syntactic but also on the semantic level.
Compounds: Like derivations, there are only few entries for compound words in EWN, and there is no way to generate them dynamically. In our application, we dealt with this problem either by combining the concepts of the individual constituents making up the compound expression, provided the constituents are already defined in EWN, or by searching for synonymous expressions, each consisting of a single word in order to take its definition as a substitute for the compound being actually searched for. The disadvantage of this method is that it makes the semantic construction more difficult and the semantic representation very complex and even inaccurate in some cases as well. This problem becomes more obvious in languages like English, where the constituents of a compound expression are separated by spaces. Consequently, it is sometimes difficult to recognize compounds as such. Therefore generating all possible conceptual combinations dynamically would be of a great advantage.
Orthographic variants: As there are no uniform orthographic rules, it would be a big plus for EWN if it would account for possible orthographic variants of an expression like email / e-mail, anti-perspirant / antiperspirant, web page vs. website), which will accelerate search and retrieval.
The research presented in this paper has been carried out and tested in the framework of the EMBASSI project, sponsored by the German Federal Ministry of Research (Grant No.: 01IL9904F8), providing multi-modal assistance for controlling audio and video equipment.