Dialogue systems for spatially situated tasks need to provide referential descriptions of spatially located objects and understand such descriptions from users. To construct such dialogue systems, it is useful to investigate how humans describe object locations in their immediate environment and how they ask about object locations in remote environments. In this paper, we address the semantic classi cation of the localization utterances found in the CReST corpus, which is a dialogue corpus of humans performing a cooperative, remote, search task. The aim is to explore the relation between speci c semantic con gurations and the dialogically and situationally embedded linguistic forms employed. Speci cally, we rst extracted di erent types of localization utterances from the corpus and then paired these with semantic categories provided by the linguistically motivated spatial ontology GUM. The paper concludes with a discussion of the characteristics of di erent types of localization expressions on the basis of spatial concepts and descriptions employed.
The localization of objects in an indoor environment poses a considerable
challenge for dialogue-based intelligent systems, which are employed in a variety of
tasks, such as mapping and localization Kruij et al. [
Spatial localization has two main aspects: location description and location
query. The recently collected human-human dialogue corpus CReST
(cooperative, remote, search task) of Eberhard et al. [
The Generalized Upper Model (GUM) is a general task and domain
independent \linguistically motivated ontology" that provides linguistic semantics
for spatial expressions. It serves as an intermediate \interface ontology"
mediating between linguistic forms and contextualized interpretations. In previous
work, Bateman et al. [
Corpus Analysis
The CReST corpus [
The CReST corpus is particularly relevant for the localization problem because the experiment directly involves several distinct localization subtasks. In one subtask, the searcher was to report the locations of eight green boxes in the environment and the director was to mark their locations on the map. Hence, the director needed to learn about the boxes' locations through dialogue. In a second subtask, the director was to direct the searcher to the cardboard box at the furthest point in the search environment, while the searcher was to collect blocks from blue boxes and put them into the cardboard box. Here, the director needed to track the searcher in the environment (from dialogue only, with no visual feedback) and then give further instructions to direct the searcher to the blue boxes.
In order to extract probable relatum and locatum from the dialogues, three wordlists were created. The rst wordlist (BP) contains the words related to the places in the environment (room, cubicle, o ce, hallway, doorway, etc), the second (FO) words related to the xed objects (door, wall, steps/stairs, stage, booth, etc), and the third (MO) words related to the movable objects (computer, chair, table, shelf, cabinet, etc). The localization-related utterances of the directors were rst extracted on a word-by-word basis through comparing each word with those in the wordlists. These utterances were then processed using a set of tools (e.g., tokenizer, part-of-speech tagger, and regular expression based NP chunker) developed in NLTK toolkit1, in order to nd the meaningful noun-phrase(s) (NP) of individual utterances. For instance, NP [the/DT little/JJ tiny/JJ room/NN] is the meaningful phrase of the utterance \is it in the little tiny room?".
Analyzing the number of utterances of each group, we nd that rooms of the wordlist BP and doors of the wordlist FO are most frequently used. Moreover, we found that \small", \big",\tiny", " rst", etc. (see Table 1) are often used to distinguish rooms. 1 http://nltk.org/ Group Examples BP FO MO
Descriptions from the corpus are you still in the smaller room? little, small, big, long, large, okay now you're in the long hallway? tiny, initial, rst, third, etc there's a open door in front of you right? open, close, single, double, is there a door with a single door? small, two, etc are there ling cabinets you see? ling cabinets, are there two chairs at the desk? two chairs, etc 3
Semantic Classi cation of Localization Expressions
The GUM ontology [
SpatialLocating an open door GeneralizedLocation hasSpatialModality: FrontProjectionExternal relatum: you(person)
Exploring the di erent forms of localization expressions, we found several utterances (e.g, 19.5% utterances of SpatialLocating), in which the locatum (L) is unknown and the relatum (R) is either an object or a person (e.g, the searcher). As can be seen from table 2, the locatum of the sentence \what do you see immediately to your left?" (Con guration: SpatialLocating, Locatum: unknown, Relatum: person, SpatialModality: LeftProjectionExternal) is unknown. From the analysis of spatial relations between a locatum and a relatum, we have found that there are many utterances (e.g, 33.8% utterances of SpatialLocating) related to the GUM concept Containment of a person (e.g., \Are you in a hallway or small room?") or an object (e.g., \Is there like a room that has a bunch of desks with tables and chairs?") in a spatial place (i.e., BP). This is because locating the person rst and then the entities that can be seen from that place is essential for solving localization problems in a search task. The GUM concepts ProjectionRelation (e.g., \on the left"), MultipleDirectional (e.g., \it's in the bottom right or bottom left?"), and GeneralDirectional (e.g., \on the opposite side of the ling cabinets") are frequently used for directional relations in the corpus, together with clauses containing words such as \face" or \look" for orientation; for example: \Okay three steps and now when you look to your right there's another open door." and \if you look outside the cubicle there should be a door to your right is that correct?".
Many utterances (e.g, 35.1% utterances of SpatialLocating) in CReST contain
what in GUM terms are called ComplexCon gurations, in which the director
describes two or more spatial locations in a single utterance. The speci cations
of such utterances usually combine several SpatialLocating con gurations with
conjunction (\SLCjSL") or disjunction (\SLDjSL") as shown in table 2. For
example, by saying \there's a cubicle on your right hand side and then straight
in front of you there's also a door?", the director required the searcher to con rm
a spatial setting which only matches with the room in the upper left of the setup
shown in Figure 1. Table 2 shows that the complex sentence can be divided into
two sentences: \there's a cubicle on your right hand side" and \there is also a
door straight in front of you" respectively.
In this paper, we analyzed location descriptions and location queries in the
CreST corpus which naturally includes di erent forms of localization utterances
in a dialogue context. First, to analyze the probable locatum and relatum, we
extracted localization utterances which are more frequent in the corpus by
classifying the environment into places (BP), xtures (FO) and objects (MO). Second,
we extracted descriptions of each group in order to explore the levels of
descriptions attached with locatum and relatum. Finally, we paired these utterances
with semantic categories provided by the GUM ontology to explore the speci c
semantic con gurations and linguistic forms employed in localization query. The
ndings include (1) the adjectives and clauses are used to describe the
locatum and relatum, (2) the entities of the locatum and relatum are often found
unknown or variable, (3) the spatial relations such as Containment and
Projectional are found frequently used, and (4) the query which describes two or more
spatial locations are used to uniquely identify a spatial setting in the
environment. Since linguistic semantics of localization expressions formalized in GUM
is domain-independent, the semantic classi cations and ndings of this research
o er a general conceptualization for relating place-sensitive natural expressions
to their spatial semantic interpretation and therefore, provide a sophisticated
foundation for the contextualization and generation of place-aware natural
expressions in situated dialogue systems, like DAISIE (see Ross & Bateman [