=Paper=
{{Paper
|id=Vol-149/paper-4
|storemode=property
|title=Context-Aware Computing System for Heterogeneous Applications
|pdfUrl=https://ceur-ws.org/Vol-149/paper04.pdf
|volume=Vol-149
|dblpUrl=https://dblp.org/rec/conf/ubipcmm/IsodaKI05
}}
==Context-Aware Computing System for Heterogeneous Applications==
https://ceur-ws.org/Vol-149/paper04.pdf
ubiPCMM 2005 17
Context-Aware Computing System for
Heterogeneous Applications
Yoshinori Isoda, Shoji Kurakake, and Kazuo Imai
illustrate the motivation of our work.
Abstract— It is a very important to develop context-aware
systems that can handle, at the same time, multiple heterogeneous A. Motivation Scenario
applications that require different contexts with different levels of User A is at home in the morning. He launches a home
abstraction. This paper proposes a framework for such systems. monitoring application, which records his physical activity
To handle the heterogeneity of the context required by the context at home throughout the day and also alerts him if
applications, we introduce a user activity context detection
method based on the combination of a multi spatio-temporal pre-registered events are detected within the house. While
description of measured sensor data, a description of detected preparing breakfast for himself and his dog, he starts a cooking
context with multiple levels of abstraction, and an order-sensitive assist application. He carries an RFID tag. Motion detection
description of the context model required by an application. We sensors, touch detection sensors and other sensors are attached
also introduce an algorithm that implements the context detection to the dishes, pans and other kitchenware. Cameras are sited
method by reflecting the context detection capabilities of any around the kitchen. Once he selects a recipe for breakfast, the
given environment. We build a prototype system by embedding
sensors into an experimental house; evaluations show it promise. cooking assist application shows him cooking instructions
step-by-step according to the selected recipe on a near-by
Index Terms— context recognition, decision tree, screen. He can interact with the application by touching or
spatio-temporal representation, ubiquitous moving the appropriate kitchenware. While general soup
making instructions are displayed on the screen, he can request
more detailed instructions by shaking a soup pan. He can skip
I. INTRODUCTION to the next instruction by tapping the pan. If his body is too
close to a pan of boiling water, the home monitoring application
C ontext-aware computing systems are being studied, and
many prototypes have been implemented over the years [1].
The popularity of context-aware computing research indicates
flashes an alert light in the kitchen. When he goes to the family
room to feed his dog, the cooking assist application is
that systems that can identify the user's context and that of the automatically suspended. If the movement of some dish is
surrounding environment have the profound potential to detected while he is in the family room, the home monitoring
provide services that are much more user-specific. Context application sends an alert message and visual information
awareness is especially relevant to mobile and ubiquitous captured by the cameras in the kitchen to the screen closest to
applications because some of those applications are suitable him in the family room. When he goes back to the kitchen, the
only at a particular time and/or location for users in a specific cooking assist application is resumed. His dog wears an RFID
situation. The ability to detect context in different environments tag. RFID tag readers are mounted on walls, and motion
is essential for a system that provides context-aware mobile detection sensors are mounted on doors and some furniture.
ubiquitous applications over a wide area. Time, location, The home monitoring application can track the location of the
identity and activity have been proposed as the primary dog and detect if the dog touches doors or furniture. The
elements of context [2]. Because it is relatively easy to detect application passes messages to the screen nearest to him when
time and location, a lot of location-aware systems such as his dog moves from one room to another. When the application
guides for city tours [3], [4] and guides for museums [5] have notifies him that his dog is trying to get into the study room, he
been designed. Activity is much more difficult to identify, but stops cooking, rushes to the study, stops the dog, and carries it
some aspects of activity can be detected by placing sensors in back to the family room. In addition, a daily chores reminder
the environment. Advanced context-aware applications using application is always running. It reminds him of tasks to be
activity context information have been realized for a specific done. Just after he wakes up, the application tells him to take his
smart environment [6]. Prior researches tend to focus on medicine. When he is about to leave home, the application
specific applications in specific environments. The goal is, indicates which windows are still open and confirms that he
however, to support different context-aware applications wants them to remain open. It also suggests that he take an
simultaneously in the same environment and across different umbrella since the weather forecast is for rain. While cooking,
environments. We believe that context-aware systems must he found that the rice would run out within a week. He enters
become capable of handling multiple heterogeneous this into the daily chores application. He has registered with
applications and environments if such systems are to be brought several context detection services at shopping malls that he
to market. We will use the following scenario to further often visits. This means that his daily chores application knows
Yoshinori Isoda, Shoji Kurakake, and Kazuo Imai 18
where to buy rice; it will keep reminding him every time he is context detection method in a given environment. The
physically close to a shop with rice until he buys it. When he is algorithm learns the context detection capabilities of the
out of his home, the home monitoring application automatically sensors at each site. This enables the heterogeneity in
changes its behavior and notifies him, via his cellular phone, environment with respect to sensing capability to be well
only when the dog leaves the house or unauthorized people try handled. We have also developed a framework that allows a
to enter it. context-aware application to adapt to the different context
We make three observations from this scenario. detection capabilities. We assume that an ID is assigned to each
class of context and that the IDs are shared among the
1) Different contexts will have significantly different spatial applications and context detection environments. Thus the
and temporal resolutions. The time period of the touching required context designated by an application is mapped to
kitchenware context is a couple of seconds, and that context detected by a local context detection system by using
context happens within the kitchen. The context of leaving context ID.
home may occupy a couple of minutes. The context of We have built an experimental environment house into
looking for rice may extend over a week and the context which we placed sensors such as floor-mounted weight sensors
should be detected by using sensors spread city-wide. and RFID tags with touch sensor. Tests were conducted in this
2) The same sensor data can be used to detect the different house and show that our methods and framework are
contexts required by different applications. For example, promising.
the data from the motion detection sensor attached to a dish The remainder of the paper is organized as follows: Section 2
is basically used by the cooking assist application. The presents the design of overall system, Section 3 describes a user
same data, however, can be used by the home monitoring activity context detection method, Section 4 presents an
application to detect suspicious events. Note that even the algorithm for implementing the context detection method,
same application may require different levels of context to Section 5 describes the experimental house, the sensors, and the
be abstracted from the same sensor data. system. Section 6 describes the experiment and analysis of the
3) Some of the context-aware applications that are triggered results, and Section 7 draws our conclusions.
by the detection of the same pre-registered multiple
contexts require an ordered sequence of context detection,
other do not. The cooking assist application, for example, II. DESIGN OF THE SYSTEM
requires the appropriate contexts to be detected in a The design of the proposed system is based on decoupling
pre-defined order. The daily chores application does not the description of measured sensor data from the description of
care which open window was closed first. detected context, and the description of required context model
associated with an application. The benefit of this approach is
B. Our Approach that users can employ context-aware applications in various
To address the issues identified in the scenario above, we ubiquitous environments (UE).
developed a user activity context detection method that can, at The framework of the system is shown in Fig. 1. The global
the same time, cope with multiple context-aware applications network stores and manages application templates. When a user
that may require different kinds of context with various levels wants to use a specific application, the user downloads the
of abstraction. Our method decouples the description of corresponding application template to the user’s terminal (Fig.
measured sensor data from the description of detected context, 1, (1)). Each application template consists of a pointer to the
and the description of required context model associated with application and a description of the required context, which
an application. Note that the terms “detected context explains the necessary conditions in terms of the contexts that
description” and “required context model” refer to different will trigger, suspend, and resume the application. We call the
concepts. User activity context descriptions have multiple description of the required context model because context itself
abstraction levels for describing detected user context. The is referred to as the abstract context concept.
model of required context is instantiated by available In each UE, sensor data is mapped to context by using a
abstraction levels of context depending on the detection decision tree. At the each abstraction level of the decision tree,
capability of each environment and the level of context required each leaf is assigned to a user state, and each node has a
by the applications. Our sensor data description is flexible in description of decision making conditions with respect to the
terms of temporal and spatial resolution. The duration of sensor value of sensor data to further split the user state of a node into
activation is the most important feature to detect user context, more specific user states. Thus, the decision tree describes
and the sensor data description can adapt the time scale detectable user states with multiple abstraction levels by using
according to the duration of sensor activation of interest. We sensors in the environment. Some of the user states in the
describe the required contexts to control an application based decision tree are associated with context. In practice, some
on regular expressions. This enables the description of ordered leaves in the decision tree are treated as context leaves. The
and semi-ordered contexts as condition elements for satisfying decision tree is stored in a context server. The context server
application requirements. receives sensor data and maps it to the detected context
We also introduce an algorithm that can implement the descriptions. If a user wants to use a specific application and
has already downloaded its application template to his mobile
ubiPCMM 2005 19
terminal, the user compares the required context model and the state. Context models take the form of regular expressions.
decision tree stored in the context server at the site that he will Each state consists of context models partitioned by commas.
visit. By using the shared context class ID, the user can For example, the ordered sequential state transition sequence
determine which abstract level of context needs to be detected [S1→S2 →S3] is represented as “^S3, S2, S1”. However, when a
at the site for each context described in the required context user performs a specific task, it is not always necessary for the
model. The user can carry out this process over the net or at the states associated with the task to occur consecutively. For
site. When the user is in some UE, the user receives the detected example, user states S1, S2 are elements of one context model.
context description from the context server (Fig. 1, (2)). The relationship between S1, S2 is ordered, but they don’t
Thereafter, the user allocates the detected context to the always occur consecutively. In this case, the context model is
elements of the required context model in the application represented as “^S2, ([^,]*,) S1”. Moreover, sometimes a user
template (Fig. 1, (3)). For example, when the application needs performs different tasks in parallel. Therefore, the context
to know only whether the user is indoors or not, the user selects models in the user activity model are managed separately, and
a user state with high abstraction level in the decision tree. On checks are performed for multiple applications to determine
the other hand, when the application requires some specific which ones should be executed.
behavior in some specific place (e.g. cooking in the kitchen) as
context, the user must select a state from a lower abstraction time = t1
level. As a result of this procedure, the appropriate user states at Tag reader(1) Tag reader(2) tag
tag11, ,tag
tag22
appropriate abstraction levels are instantiated in all required detection area detection area
Objects in the
context model elements. The application is then controlled user user’s vicinity
tag1 tag3
according to the description in the application templates in the
tag2
mobile terminal. taguser tag4
Area(1) Area(2)
Global Network
Application templates time = t2
Required context Model
Application (Context required by Application) tag
tag22, ,tag
tag33, ,tag
tag44
(1) Download application
Ubiquitous Environment templates
Application Required context Model
tag1 tag2 tag3
Context taguser
Server tag4
Sensors
(2) Download detected Area(1) Area(2)
Mobile
context description
terminal
Detected context
Application templates description Fig. 2. Changes in RFID tags output associated with movement of the user.
(Downloaded) Abstraction Level
RFID tags are attached to objects and the user.
Required Context Model 1
Application
2
B. Description of User States
(3) Instantiate context model
N
Each UE offers a different combination of sensors (type and
User activity
context description Context decision tree number). The sensor data descriptions should support the use of
many types of sensors. In this paper, sensor information is
UE-ID, Abstraction Level
Application
transformed into spatio-temporal attributes. Each attribute
consists of temporally continuous sensor data, and the attributes
are arranged to support each spatial resolution needed.
Fig. 1. Framework of the proposed system Complex sensors, such as cameras have already been used for
recognizing user state. Computer visions tracking [7], [8] and
behavior recognition [9], [10], [11] often work in the laboratory
but sometimes fail in real environments due to the lighting
III. USER ACTIVITY CONTEXT DETECTION METHOD
variations and occlusions that are frequent in natural
A. Context Model environments. Given their relative immaturity, we restrict
As we mentioned above, each context model is used to ourselves to simple sensors such as RFID tags and weight
trigger a specific application and each model consists of context sensors to achieve robust user state detection.
elements that describe specific user states. In the real world, As an example, we explain the description of user states
users perform a wide variety of tasks. In this study, a task refers when RFID tags and floor sensors are employed as sensors.
to work performed by a user; work here consists of performing RFID tags have recently attracted interest as a means of
specific state transitions, such as checking that doors are locked detecting objects in the vicinity of the user. In this case, the
in multiple locations, or preparing a meal by following a series attributes describing the user’s state are derived from
of procedures. When the user performs the tasks, the time information obtained from the RFID tags carried by the user
ranges of user states included in the task change accordingly. and attached to objects. As shown in Fig. 2, the RFID tag
Therefore, the context model deals with only transitions in user information detected by each RFID tag reader is collected. This
Yoshinori Isoda, Shoji Kurakake, and Kazuo Imai 20
yields attributes that relate the state of each user to the objects Loc
Loc(u,
(u,tt)) ::User
Useruuhas
hasbeen
beeninincells
cellswith
withmulti
multispace
spaceresolutions
resolutions
in the user’s vicinity. In addition to information on the presence cell7S
or absence of objects in each user’s vicinity, the temporal
cell8S
continuity of the presence of these objects is also used as Space resolution: S
attributes of the user’s state. In other words, the user’s state is
Space resolution: M
described by attributes that include information on how long cell1M cell2M
each object has been present in the user’s vicinity due to the
Space resolution: L cell1L
user’s movements and whether or not the objects in question
are being carried around by the user. The overall set of
cell1S cell2S cell3S cell4S cell5S cell6S cell7S cell8S cell1M cell2M cell1L
attributes describing the state of user u in relation to these
0 0 0 0 0 0 d2S 0 0 d2 M d1L
objects at time t is as follows:
Space resolution: S Space resolution: M Space resolution: L
Tag (u, t ) = [ tag1 (t ), tag 2 (t ), ... , tag i (t ), ... , tag n (t ) ] (1)
Fig. 4. Attribute representation of user position with multi spatial resolution.
Here, tagi(t) is a positive integer value indicating how long
object i has been remained (continuously) in the vicinity of
user u at time t. If object i does not exist in the user’s vicinity, Loc(u, t ) = [ cell11 (t ), cell21 (t ), cell31 (t )...,
(2)
the value of tagi(t) is zero. Here, the user’s vicinity corresponds cell1R (t ),..., cell Rj (t ), ... , cellmQ (t )]
to the detection range of the tag reader that detects the tag
carried by the user, so all objects detected by the same tag
reader are regarded as being in the user’s vicinity. When plural Here, celljR(t) is also a positive integer value indicating
users occupy the same detection range of a tag reader, it is whether or not the user has been continuously detected in cell j
difficult to distinguish which objects are in each user’s vicinity. with space resolution R at time t.
Therefore, the detection range of the tag reader is restricted When the above two kinds of sensors are employed in a UE,
from several tens centimeters to one meter by using an the entire set of attributes A(u,t) representing the state of user u
attenuator. In this way, the user’s state is identified, in part, at time t is as follows:
from the attributes expressing which objects are in the user’s
vicinity, and if so, for how long. For example, Fig. 3 shows the A(u, t ) = [ Tag (u, t ), Loc(u, t ) ] (3)
attributes of objects around the user at times t1 and t2 in Fig. 2.
C. User State Decision
Tag
Tag(u,
(u,t1t1)) ::When
Whenthe
theuser
useruustays
staysfor
foraawhile
whileininarea
area(1)
(1) Several methods to discriminate user state have been
tag1 tag2 tag3 tagp tagn proposed. Combinations of neural networks and lookup tables
d1 d2 0 0 0 were used to predict the temporal mobility patterns of a user in
the Neural Network House [12], [13]. However, since neural
dk : positive integer value indicating how long tagk has nets do not provide information about the underlying model of
existed continuously in the vicinity of user u
the prediction process, it is difficult to extend them to
Tag
incorporate prior knowledge. Specific types of Dynamic
Tag(u,
(u,t2t2)) ::Directly
Directlyafter
aftermoving
movinginto
intoarea
area(2)
(2)
Bayesian networks [14] have been used to track the daily
tag1 tag2 tag3 tagp tagn activities of users from sensor information such as active
0 d’2 d3 dp 0 d’2 > d2 badges [15] and cricket-devices [16] embedded in a house [17].
The algorithm can differentiate the activities according to
Fig. 3. Representation of attributes relating to objects. duration and user location. Dynamic Bayesian networks have
proven to be one of the most powerful and efficient ways of
RFID tags provide information on the spatial relationships representing temporal events and fusing information from
between the user and objects. In some UE, sensors that detect multiple sensors [18]. However, the complexity of networks
information on the absolute position of the user will be and learning algorithms for Hidden Markov models make it
employed. In this case, the multiple absolute user positions, difficult to apply them to cases involving many sensors.
needed to satisfy the multiple spatial resolutions, are used as Each application requires a different level of user state
attributes representing user state. As Fig. 4 shows, one floor can abstraction as context. Some applications simply need user
be partitioned into several cells. Several cells are grouped into position with coarse resolution; others may need more detailed
larger cells. Additional attributes are used to express whether or user states such as user behavior. Therefore, user states should
not the user is present in each cell, and if so, for how long. The be detected with various levels of abstraction. With the goal of
attributes related to the position of user u at time t are as detecting user state hierarchically, we use a decision tree that
follows. discriminates user states with multiple levels of abstraction.
The benefit of using a decision tree is that it generates
understandable rules. Therefore, if high-level knowledge about
ubiPCMM 2005 21
discriminating user states exists, the decision tree can be and the corresponding information gained by an attribute A
modified manually. In each UE, the detected context with k outcomes as
description from sensor data is described as a decision tree that
detects user states hierarchically with different abstraction k
| Di |
levels. If more than one person is in a UE, the decision tree is Gain( D, A) = info( D) − ∑ × Info( Di ) (5)
i =1 D
managed to detect each user context separately.
The information gained by an attribute is strongly impacted
IV. CONTEXT DETECTION METHOD by the number of outcomes and is maximal when there is one
case in each subset Di.. On the other hand, the potential
It is difficult to create the rules that form a decision tree information obtained by partitioning a set of cases is based on
manually because each UE will have different sensors and the knowing the subset Di into which a case falls; this split
user would have to consider the meaning and significance of information
sensor information and the relationships between sensors.
Moreover, given the large number of sensors in one UE, some k
| Di | | Di |
sensors will be prone to error. This means that the user would Split ( D, A) = −∑ × log 2 ( ) (6)
have also considered the robustness of sensor data. Our solution i =1 D D
is to employ the learning-based approach to construct the
decision tree semi-automatically. tends to increase with the number of outcomes of an attribute.
All sensor data is recorded while a user performs actions The gain ratio is expressed as
such as using the toilet, bathing, grooming and so on. These
actions typically consist of specific user states. For example, GainRatio( D, A) = Gain( D, A) Split ( D, A) (7)
the action of bathing consists of states such as standing in front
of a bathroom, standing inside the bathroom, using the bath, The gain ratio criterion assesses the desirability of an
and so on. However, identifying these states depends on the attribute as the ratio of its information gain to its split
sensors in the UE. Our context detection method extracts information. The gain ratio of every possible attribute is
hierarchical states of each action as leaves of the decision tree. determined and, among those with at least average gain, the
Sensor data recorded in each time interval is transformed to split with maximum gain ratio is selected.
spatio-temporal attributes according to (1)-(3) and the data for In a decision tree constructed in this way, the attributes that
each time interval is used as one training data set for are valid for discriminating the user states identified by the
constructing a decision tree. At first, the user allocates action actions are learned as nodes of a decision tree. Leaves of the
labels to the specific time regions of the training data by hand. decision tree are the user states to be discriminated. Applying
These time regions correspond to specific actions such as the C4.5 algorithm to each state recursively, our system extracts
bathing. Next, our system finds user states identified by the the hierarchical states of each action. In the following, we detail
actions by using the C4.5 algorithm [19]. The C4.5 algorithm the procedure by using an example. As shown in Fig. 5, the user
constructs a decision tree by calculating the information gain allocates label 1A to the specific time regions that correspond to
ratio of each attribute from a set of training data, and the specific action in the training data. Time regions other than
successively employing the attributes with the highest values those labeled 1A are assigned label 1E. This is the first level of
(e.g., tagi(t)) as the nodes of the decision tree. Moreover, the the decision tree. By using this set of training data, the C4.5
C4.5 algorithm can treat various kinds of sensor information as algorithm constructs a decision tree that decides valid attributes
attributes. This makes it easier to integrate different kinds of of sensor data as nodes and states of the action 1A as leaves.
sensors and to use them to discriminate user states. In the real In Fig. 6, action 1A is divided into n leaves in the decision
world, sensor error can occur at any time. If the training data tree at the second level. These leaves are taken as user states at
contains instances of sensor error, the C4.5 algorithm selects abstraction level 2 of action 1A, and assigned the label of 2Si
(1 ≤ i ≤ n) . In the training data, each time region of state Si is a
the attributes of sensors that are able to discriminate user state 2
robustly. region in which sensor attributes that discriminate 2Si hold (Fig.
Let D denote a set of cases of training data. Each case is 6). At the third level of the decision tree, new decision tree
labeled with its state class cj (1 ≤ j ≤ N ) . Some attribute A with branches are created from each state 2Si. In this level, the time
mutually exclusive outcomes A1, A2,..., Ak is used to partition D regions corresponding to 1A in the first level are used as new
into subsets D1, D2,..., Dk, where Di contains those cases that training data (Fig. 7). Sensor attributes that were used to
have outcome Ai. p(D, cj) is the proportion of cases in D that discriminate 2Si are deleted from the new training data. That is,
belong to the jth class. The residual uncertainty about the class the sensor attributes used in the second level aren’t used in the
to which a case in D belongs can be expressed as third level. The time regions other than those labeled 2Si are
re-labeled 2Ei. The decision tree at the third level is constructed
Info( D) = −∑ p (D, c j )× log 2 ( p (D, c j ))
N
(4) from this new training data set. For example, 2Si is divided into
j =1
m leaves. Each new leaf is assigned the label 3Sij (1 ≤ j ≤ m) ,
and together they are considered as the user states derived from
Yoshinori Isoda, Shoji Kurakake, and Kazuo Imai 22
2
Si at abstraction level 3. These procedures are executed
Decision tree Training data
recursively to construct sub-trees. For example, the kth node at at level 1 Yes No
the fourth level of decision tree is labeled 4Ski,j. The result is one 1A 1E
Labeling manually
decision tree that can discriminate user states at multiple levels.
Fig. 8 shows an example of one such decision tree. Decision tree Attribute
In the context detection phase, the resulting decision tree and at level 2 No
Yes
the current information from sensors are used to discriminate Attribute
the current user state in each abstraction level. Yes No
time
Extracting 2S 2S 2S 2S
Level 1 1E 1A 1E 1A 1E Automatically 1 2 i n
Sensor 1
Decision tree
Sensor 2
at level 3 Re-labeling automatically 2S 1E
i
OFF ON Attribute
Sensor No
Yes
Attribute
Sensor X Yes No
Time interval of acquiring sensor data
(This sensor data is transformed to one training data) Extracting 3Si 3Si 3 Si 3Si
Automatically 1 2 j m
A set of training data Decision tree 4Si,j
at level 4 k
Fig. 5. User state allocation in specific time intervals; manually performed at
the first level.
Fig. 8. Example of a decision tree that discriminates user states at multiple
levels.
time
Level 1 1E 1A 1E 1A
V. USER ACTIVITY SENSING AND DETECTION ENVIRONMENT
extraction extraction IN UBIQUITOUS EXPERIMENTAL HOUSE
We have developed an experimental house in which multiple
Level 2 2S 2S 2S 2S 2S
(automatically) 1 2 n 1 n types of information about human activities can be acquired
[20]. It contains various embedded sensors and output devices.
Sensor i Fig. 9 shows its floor plan. To support a wide variety of
Sensor j experiments, this house has workspaces set into the floor, attic,
Sensor k and inside walls (striped areas in Fig. 9). These workspaces
make it easy to set up and wire many sensors and devices. Fig.
Fig. 6. User states are extracted at second level of decision tree. 10 shows the configuration of the prototype system constructed
inside the experimental house. The next subsection details
system elements.
New set of training data
time Overview
Level 1 1A 1A 1A
WC
Washroom Bathroom
Delete sensor attributes that discriminate 2Si Entrance
Attic
Level 2 2Ei 2Si 2E
i
2S
i
2E
i
2S
i
2E
i Staircase
(2Si) Workspace
Re-labeled training data Kitchen
2
Fig. 7. New training data for extracting use states at level 3 for Si. Inside wall
Living Sofa
Room Dining Room
Closet TV
Fig. 9. Overview of Ubiquitous Experimental House.
ubiPCMM 2005 23
Global network Magnetic
Application reed switch
templates
RFID tag
RFID tag reader
RFID tag Info. User position Info. Door Faucet Towel
RFID tag (a) Sensors detecting human behavior of operating objects
Spatio-temporal
Mobile
description transformer terminal RFID tag Object
Spatio-temporal
description
Decision tree Decision tree Floor sensor
constructor
(Training phase) Touch sensor
Electrode Receiver Mug cup Toilet seat
Context server (b) Sensors detecting human behavior of touching objects
Fig. 10. Prototype system configuration. RFID tag
A. RFID Tags
As shown in Fig. 11, the house has an active type RFID tag
Small weight sensor Chair Chair Sofa
system consisting of RFID tag readers and RFID tags attached
(c) Sensors detecting human behavior of sitting and lying
to the users and to various objects. We extended some of the
RFID tags to suit the object. For instance, we attached a
Fig. 11. Sensors detecting human behavior associated with objects.
magnetic reed switch to an RFID tag to yield a position sensor.
These RFID tags can detect human behavior when using
objects such as turning a faucet, opening a door, using a towel
Bathroom
and so on. Other RFID tags were modified with touch sensors. Shoes box BACK Toilet seat
When the user touches electrodes on the object, the touch
sensor detects the action and the RFID tag is activated. We can INSIDE Faucet
detect human behavior that involves touching objects such as
holding a cup, holding a receiver, sitting on the toilet, and RFID tag Door Large size
resolution name
brushing teeth and so on. Moreover, other RFID tags were Door FRONT
modified with small weight-sensitive sensors to detect user ENRANCE
behavior such as sitting on a chair, lying on a sofa and so on. User ENTRANCE HALL
This tag system has low directionality in the tag detection BATHROOM
ranges, and by attaching attenuators we were able to set the tag Door PASSAGE
detection radius from a few tens of centimeters to a few meters.
Each reader communicates with a PC by TCP/IP via a protocol
converter. The RFID tags transmit at intervals of 0.4 seconds. Fig. 12. Experimental environment.
B. Floor Sensor tags carried by the users, and the decision tree is used for each
To recognize human activity, information of the user’s user. The states of each user thus detected are sent to the user’s
position is useful. Therefore, the entire floor is covered with a mobile terminal. On the mobile terminal, user states are
large number of pressure sensors. Each pressure sensor covers received and applications are executed by matching the user
18 x 18 cm (a unit cell), and the binary value output by each cell states and context elements.
is read into a PC via a serial port at 0.4 second intervals. These
detector cells were grouped into blocks (e.g. 10×10, 5×5, 3
×3), so user position is detected at multiple resolution levels. VI. EXPERIMENTAL EVALUATION
To confirm the automatic construction of a decision tree, we
C. Context Server
performed an experiment that focused on bathroom activities as
The context server operates in one of 2 phases. One is the the first step.
phase of constructing a decision tree that discriminates user
states with multiple levels of abstraction. Information from A. Experimental Conditions
RFID tags and floor-mounted weight sensors are acquired and Fig. 12 shows the experimental environment in the bathroom.
used as training data. A user performed a series of actions such as entering the
In the other phase, it users this decision tree to perform state bathroom, using the bathroom, washing hands, leaving the
decisions based on the current information provided by the bathroom and walking around the house. We acquired these
RFID tags and floor-mounted weight sensors. When plural series of actions five times as training data. One RFID tag was
users are in the house, each user is distinguished by the RFID continuously carried by the user. Other RFID tags were
Yoshinori Isoda, Shoji Kurakake, and Kazuo Imai 24
attached to objects. Five RFID tags were extended with a
Floor_BATHROOM_Duration > 0 No
magnetic reed switch, and attached to doors and faucet in the Yes
User state
bathroom. Level 1 TOILET OTHER
Attribute
The information acquired by the floor-mounted weight Floor_INSIDE_Duration > 0 No
sensors was represented with multiple levels of spatial Yes Floor_FRONT_Duration > 0 No
resolution. At the largest resolution level, the experimental area Yes Floor_BACK_Duration > 8
Yes No
was divided into four spaces: ENTRANCE, ENTRANCE
Level 2 INSIDE FRONT BACK OTHER
HALL, BATHROOM, and PASSAGE. The medium level of
resolution corresponded to the areas consisting of 5×5 blocks. Floor_BACK_Duration > 0 No
The smallest level of resolution corresponded to areas of 3×3 Yes
Rftag_Faucet_Duration > 0
No
Yes
blocks. Floor_FRONT_Duration > 0
Yes No
B. Experimental Result Rftag_ToiletDoor_Duration > 0
Yes No
Fig. 13 shows the decision tree that was constructed Rftag_ToiletDoor_Duration < 12
Level 3
automatically, and Fig. 14 shows the decision results for each Yes No
time interval at each level of the decision tree. INSIDE&BACK WASH HANDS JUST ENTER OPEN DOOR OTHER1 OTHER2
At the first level of the decision tree, two leaves were
extracted as user states. Investigating the attributes of decision Fig. 13. Decision tree constructed in the experiment.
tree, one considers that the user is near the bathroom. The other
considers that the user is not near the bathroom. They are
time
named TOILET and OTHER, respectively. The largest space Level 1
resolution, which corresponds to the area of the bathroom, was OTHER TOILET OTHER
selected as the attribute to discriminate the two states at this Level 2
OUTSIDE INSIDE BACK INSIDE OUTSIDE
level.
At the second level of the decision tree, four leaves were Level 3
JUST INSIDE INSIDE WASH OPEN
extracted as user states. Investigating the attributes of the ENTER &BACK &BACK HANDS DOOR
decision tree, we see that the first considers that the user is in
front of the bathroom, the second considers that the user is Fig. 14. User state decision for each time interval.
inside the bathroom, the third considers that the user is at back
of the bathroom, and the fourth considers that the user is in
some other state. They are named FRONT, INSIDE, BACK, abstraction levels are extracted by using the C4.5 algorithm.
and OTHER respectively. At this level, three areas in the These user states, extracted as leaves of decision tree, are
bathroom with medium level of resolution were selected as detectable by using sensors in the experimental environment.
attributes. According to the decision tree and application requirements,
At the third level of the decision tree, no new leaves were the user instantiates the appropriate user states at the
extracted from the user states of FRONT and BACK. However, appropriate abstraction levels in all required context model
six leaves were extracted from the user state of INSIDE. The elements. However, this preliminary experiment considers only
first leaf from INSIDE considers that the user is standing on simple activity. As the next step, we will confirm the validity of
both the cell of BACK and that of INSIDE simultaneously. This decision trees constructed from more complex activities.
leaf is named INSIDE&BACK. The second leaf considers that
the user is washing hands. This leaf is named WASH HANDS.
The third leaf is detected from the attributes of the floor sensor VII. CONCLUSION
at the area in front of the bathroom and the area inside the In this paper, we proposed a framework for context-aware
bathroom. This leaf is detected only when the user is just systems that allows context-aware applications adapt to the
entering the bathroom (Fig. 14). Investigating the training data, difference in context detection capability present in each
the floor sensors corresponding to FRONT and INSIDE were environment. A user activity context description, which has
not active simultaneously when the user left the bathroom. It is multiple abstraction levels in terms of user context, is used. It is
considered that the position relationship between the door, the instantiated according to the level of context detection
faucet, and toilet seat influence human behavior. It can be said capability in each environment and application requirements.
that this extracted leaf well matches real human behavior in the We also presented a user activity context detection method that
environment. Therefore, this leaf is named JUST ENTER. The can detect user context by using a multi spatio-temporal
fourth leaf from INSIDE considers that the user is just leaving description of measured sensor data, a description of detected
the bathroom. When the user is entering the bathroom, the same context with multiple levels of abstraction, and an order
attributes can be used. However in the case of entering the sensitive description of required context model associated with
bathroom, the decision tree detects it as JUST ENTER. So, this an application. The description of detected context takes the
leaf is named OPEN DOOR. form of a decision tree. It is constructed automatically by the
From these results, reasonable user states at different C4.5 algorithm.
ubiPCMM 2005 25
We introduced a ubiquitous experimental house in which [16] Nissanka B. Priyantha, Anit Chakraborty, and Hari Balakrishnan, “The
cricket location-support system,” In Proceedings of MOBICOM 2000,
various human activities can be acquired from the data of pages 32–43, Boston, MA, August 2000. ACM, ACM Press.
embedded sensors. We performed a preliminary experiment in [17] R.Want, A. Hopper, V. Falcao, and J. Gibbons, “The active badge
this house in which a decision tree that detects context with location system,” ACM Transactions on Information Systems, 10(1),
1992.
multiple abstraction levels was automatically constructed from [18] A. Garg, V. Pavlovic, and J.M. Rehg, “Audio-visual speaker detection
actual human activity data; the results were shown to be using dynamic bayesian networks,” IEEE International Conference in
reasonable. Automatic Face and Gesture Recognition, 2000.
Issues for further study include evaluating the performance [19] R. Quinlan, “C4.5: PROGRAMS FOR MACHINE LEARNING,”
Morgan Kaufmann Publishers Inc., 1993.
of automatic decision tree construction with respect to more [20] Y. ISODA, S.KURAKAKE, and K. Imai, “Ubiquitous Sensor based
complex situations, and modeling the required context Human Behavior Recognition using a Spatio-temporal Representation of
descriptions of more complex tasks including unordered state User States,” International Journal of Wireless and Mobile Computing
series. It would also be worthwhile to evaluate the feasibility of (IJWMC), Special Issue on Applications, Services, and Infrastructures for
Wireless and Mobile Computing (accepted in Dec 2004)
our prototype system by adapting it to support real applications.
Yoshinori ISODA Senior Research Engineer, Network Laboratories, NTT
DoCoMo, Inc. He received the master’s degree from the Department of
REFERENCES Systems Engineering, the University of Osaka in 1993. Since joining NTT in
[1] D. Saha, A. Mukherjee, “Pervasive Computing: A Paradigm for the 21st 1993, he has been researching sensor processing systems and ubiquitous
Century,” IEEE Computer, IEEE Computer Society Press, pp. 25-31, computing systems. He is a member of the Information Processing Society of
March 2003. Japan (IPSJ) and the Robotics Society of Japan.
[2] Dey, A. K., Abowd , G. D. , “Towards a Better Understanding of Context
and Context-Awareness,” Proc. of the CHI 2000 Workshop on The What, Shoji KURAKAKE (M’00) Executive Research Engineer, Network
Who, Where, and How of Context-Awareness, The Hague, Netherlands, Laboratories, NTT DoCoMo, Inc. He received the master’s degree from the
April 2000. Department of Mathematical Engineering and Information Physics, the
[3] Abowd, G.D., Atkeson, C.G., Hong, J., Long, S., Kooper, R., Pinkerton, University of Tokyo. Since joining NTT in 1985, he has been researching
M.: Cyberguide, “A mobile context-aware tour guide,” Wireless character recognition, image analysis, video media handling, seamless
Networks: special issue on mobile computing and networking: selected applications, and ubiquitous services. He is a member of IEICE，IEEE, and the
papers from MobiCom .96, Vol. 3, No 5. (1997) 421-433 Association for Computing Machinery (ACM).
[4] Cheverst, K., Davies, N., Mitchell, K., Friday, A., Efstratiou, C.,
“Developing a Context-Aware Electronic Tourist Guide: some issues and Kazuo IMAI (M’84) Executive Director of Network Laboratories, NTT
experiences, “ Proc. CHI, The Hague, Netherlands, April 2000 DoCoMo, Inc. He received the B.Sc. and M.Sc. degrees from Kyoto University
[5] Broadbent, J., Marti, P., “Location-Aware Mobile Interactive Guides: in 1974 and 1976, respectively. Since he joined NTT Laboratories in 1976, he
Usability Issues,” Proc. Inter. Cultural Heritage Informatics Meeting, has been engaged in various R&D activities including digital data exchange
Paris, France, 1997 systems design, ISDN user-network interface definition, and broadband
[6] Abowd, G. A. Bobick, I. Essa, E. Mynatt, and W. Rogers, “The Aware switching and access systems research. Most recent activities at NTT were to
Home: Developing Technologies for Successful Aging,” Workshop held lead the research into and implementation of a network services platform for
in conjunction with American Association of Artificial Intelligence NTT's information sharing services. Since April 2000, he has been with NTT
(AAAI) Conference 2002, Alberta, Canada, July 2002. DoCoMo where he leads the next generation networking research for mobile
[7] S. Stillman, R. Tanawongsuwan, and I. Essa, “A system for tracking and communications.
recognizing multiple people with multiple cameras,” in Proc. 2nd
International Conference on Audio-Vision-based Person Authentication.
1999.
[8] I. Haritaoglu, D. Harwood, and L. Davis, “W4: Who, When, Where,
What: A real time system for detecting and tracking people,” In Third
International Conference on Automatic Face and Gesture. Nara, 1998.
[9] H. Ishiguro, T. Nishimura, T. Sogo and R. Oka, “VAMBAM: View and
Motion-based Aspect Models for Distributed Omnidirectional Vision
Systems,” Proceedings of the Seventeenth International Joint Conference
on Artificial Intelligence (IJCAI-01), No.651, 2001.
[10] J.W. Davis and A.F. Bobick, “The representation and recognition of
action using temporal templates,” In Proc. the IEEE Conference on
Computer Vision and Pattern Recognition, pages 928–934. IEEE Press,
1997.
[11] X. Sun and C.W. Chen, “Probabilistic motion parameter models for
human activity recognition,” IEEE International Conference on Pattern
Recognition (ICPR02), 1, 2002.
[12] M. Mozer, “The neural network house: an environment that adapts to its
inhabitants,” In Proceedings of the AAAI Spring Symposium on
Intelligent Environments, Technical Report SS-98-02, pp.110–114,
AAAI Press, Menlo Park, CA, 1998.
[13] M. Mozer, “Lessons from an adaptive house,” In D. Cook & R. Das (Eds.),
Smart environments: Technologies, protocols, and applications. J.
Wiley & Sons, 2004.
[14] K. P. Murphy, “Dynamic Bayesian Networks: Representation, Inference
and Learning,” PhD thesis, University of California, Berkeley, 2002.
[15] H. Kautz, O. Etziono, D. Fox, and D. Weld, “Foundations of assisted
cognition systems,” Technical report cse-02-ac-01, University of
Washington, Department of Computer Science and Engineering, 2003.