=Paper=
{{Paper
|id=Vol-149/paper-17
|storemode=property
|title=Experiences with Developing Context-Aware Applications with Augmented Artefacts
|pdfUrl=https://ceur-ws.org/Vol-149/paper17.pdf
|volume=Vol-149
|dblpUrl=https://dblp.org/rec/conf/ubipcmm/KawsarFN05
}}
==Experiences with Developing Context-Aware Applications with Augmented Artefacts==
https://ceur-ws.org/Vol-149/paper17.pdf
ubiPCMM 2005 111
Experiences with Developing Context-Aware
Applications with Augmented Artefacts
Fahim Kawsar, Kaori Fujinami, Tatsuo Nakajima
2) Rather than providing an analog/digital sensor value,
Abstract—Context-Awareness is a key concept of future sentient artefacts provide a “statement” to the interested
ubiquitous computing. Although various academic and industry applications, for example a sentient chair may provide a
researches are going on around the world to realize context-
statement like “The chair is being used by Rob”.
awareness, we are yet to achieve any substantial success. In this
paper we have reported our experiences of dealing with context 3) Finally, sentient artefacts can also be actuators in some
awareness. We have presented our approach for building cases.
context-aware applications by demonstrating a real life
application development process. Furthermore we have Some services like scheduler or weather forecast monitor
presented two context-aware application frameworks that we etc. are also considered as virtual sentient artefacts. By
have deployed in the roadmap of our research towards context-
augmenting sensors, we make these belongings (micro
awareness. We have discussed our findings and learned lessons
on various issues of context-awareness. component of the environment) smart. Eventually this process
Index Terms— Context-Awareness, Middlewares, Sentient recursively makes our environment smart and context aware in
Artefacts, Smart Environment. a bottom up approach. For context aware services we need to
handle this environment by modeling them in multiple
applications. These applications typically use various
I. INTRODUCTION components of the environment. In our approach, these
components are the sentient artefacts.
U BIQUITOUS computing envisions a future environment
that will be aware of its operating context and that will be
adaptive to ease our interaction [1][3][9][24]. We approach
In this paper we have reported our experience on
developing the context aware applications using these
such an environment by the environment itself. That means artefacts. We have specified the approach centered on the
taking the building blocks of the environment and making sentient artefacts that we have taken and the problems that we
them smart and context aware by capturing people’s implicit have faced during development process. For the ease of
interaction. We have been developing such building blocks, development we have written two context aware application
specifically everyday life objects by augmenting various kinds frameworks from different points of view. These two
of sensors. We call them sentient artefacts. Our vision is to middlewares’ goal is to provide a seamless platform for
utilize these objects for value added services in addition to application development automating many recurring tasks.
their primary roles. For example, consider a frying pan, its However we could not achieve a substantial degree of success.
primary use is in the kitchen. However we can utilize the In the paper we have reported the probable causes that limit
frying pan by augmenting it with some sensors/tags to infer our performance. For clearly demonstrating our approach, we
that its owner is in the kitchen or he/she is cooking while the have selected a context aware application and have shown the
frying pan is being used. Usually these artefacts differ from development process of that application. We have developed
the explicit sensors in three ways: two versions of this application utilizing the two frameworks
to identify the pros and cons of the frameworks. Rather than
1) Sentient artefacts require a small operating specifying the ideal requirements, we will focus mainly on
software/device driver that captures values from multiple sharing our experience of application development in the
sensors embedded in the artefacts and process these values in paper.
a logical way to provide information about its state of use,
properties or anything the software/driver author wants to The remaining paper is organized as follows: In Section II
provide. we have specified the steps that we have followed for
application development. Section III demonstrates our
approach by presenting a hypothetical scenario followed by
Manuscript submitted June 17, 2005.
Fahim Kawsar is a MS student in Computer Science Department at the capability requirements for the scenario functionalities and
Waseda University, Japan (e-mail: fahim@ dcl.info.waseda.ac.jp). the implementation of the scenario. In Section IV we have
Kaori Fujinami is an Assistant Professor of Computer Science Department presented two middlewares and their integration with the
at Waseda University, Japan (e-mail: fujinami@dcl.info.waseda.ac.jp).
Tatsuo Nakajima is a Professor of Computer Science Department at application. Section V discusses the problems and our
Waseda University. (e-mail: tatsuo@dcl.info.waseda.ac.jp).
Fahim Kawsar, Kaori Fujinami, Tatsuo Nakajima 112
experiences. In Section VI we have cited the related works, solving the complexity. We will discuss in detail about this
finally Section VII concludes the paper. complexity in the discussion section.
II. OUR APPROACH III. SAMPLE APPLICATION
In our lab we are constantly drawing real life practical The first step that we follow is to identify an application
scenarios. We use these scenarios as the design base for scenario. In this section we have presented a scenario, then we
deploying the environment components with augmented have demonstrated how we have implemented the scenario.
sensors and for developing integrated applications to
A. Another Morning for Binti
implement those scenarios. We believe capturing users’
context implicitly by their natural interaction with the
environment is a key issue for context awareness. Here natural Binti is a broker at the New York Stock Exchange. During
interaction means interacting with natural interfaces like her daily morning routine in the bathroom, while she is
everyday objects. A natural interface activates the cognitive brushing her teeth and putting on her make-up, her mirror
and cybernetic dynamics that people commonly experience in provides all the information she needs to start her day. During
real life, thus persuading them that they are not dealing with these activities she can watch her daily schedule. Besides that
abstract, digital media but with physical real objects. This she also sees what the weather will be like, so she can dress
results in a reduction of the cognitive load, thus increasing the fittingly. Furthermore she finds out if the subway, which she
amount of attention on content [28]. So our approach is to usually takes from her house to the Stock Exchange, is
make the artefact aware but not to make the user aware of this running properly. The subway is often delayed or closed for
fact by keeping the artefact’s primary role and interaction maintenance, in which case the mirror shows her an
technique intact. Users only use daily life objects in the way alternative route making sure that she does not have to rush to
they are used to. However our infrastructure captures these be on time for the morning breakfast meeting with her team.
natural interactions in order to generate user’s context.
B. Scenario Implementation: What To Sense and How
From our experience of development, we have identified the
essential steps that we followed for building context aware Once we have drawn the application scenario, then we have
applications. These are: to identify the context information and the capability
requirements of the artefacts that we can use to extract the
1) Fixing the goal and the target location of the application, required contexts. This scenario requires a smart mirror to be
that is where and what services to perform, where and what installed in the washroom that can capture users context
information to provide etc. It means drawing the application (specifically detecting user’s identity and user’s presence at
scenario. mirror’s location) and can present him/her with some useful
2) Identifying the context information that is required for information. We have deployed an application AwareMirror
achieving the goal or implementing the scenario. for the functionalities. In the following table we have
3) Identifying the source of the context information that is summarized what is required to be sensed for the proper
how to extract the context information from the target functionalities of the application and what sentient artifacts we
environment. This step can be further sub-divided to answer have used to capture them:
Table I
the following questions: Functionality mapping from requirements to sentient artefacts
a) What context sources are available in the
environment that can provide the required context? Scenario Required Augmented
b) How these sources are acquiring context? If sensors Functionality Capability Artefact
Used
are used then how to model the context from the raw
Display weather, Detecting user’s Mirror augmented
sensor data? schedule, and presence and with proximity
c) Who is responsible for modeling the context, the transportation position. sensors
application or the context source? information related
W to the user on the Identifying user Toothbrush as an
4) Identifying the suitable interaction technique for the end
A mirror authenticator of
users and providing a suitable way for reflecting end users’ S the user
preference. H Extracting Web service
R schedule, treated as virtual
O weather and sentient artefact
These steps are not mutually exclusive and may overlap each
O transport
other. The success of the application depends on satisfying M information
them with the highest degree of precision. However from our Displaying Mirror augmented
experience we have found that it is very difficult to satisfy extracted with acrylic
these steps. There are many issues that contribute to this information magic mirror
board
complexity thus limiting the success of context-aware
applications. In fact we cannot pick a single problem for
ubiPCMM 2005 113
AwareMirror [16] is a smart mirror installed in the is used for tracking the user’s schedule. We have used a
washroom as shown in Figure 1. In addition to its primary task dummy web service for the transportation information.
of reflecting someone’s image, it can also provide some useful
information related to the person who is using the mirror. It D. Functional View
uses two sentient artefacts: a mirror and a toothbrush. It also The application’s control flow can be stated as follows:
uses three web services to collect information about the users’
schedule, transportation information and weather forecasting. 1) During the system initialization, all the components are
When a toothbrush is used, its user is identified and the useful enumerated accordingly.
information related to the user is extracted from various web 2) The user initializes the system by providing necessary
services and is shown on the mirror. information.
3) If the user uses the toothbrush in the morning, while the
system is running, the user is identified by the system and
his/her preference information is loaded.
4) Accordingly the web services are contacted to collect the
information.
5) The system then renders the information to the Mirror.
6) The display has two modes, initially the mirror displays
very abstract information in appropriate positions within the
mirror, making sure that information does not cover the main
portion of the mirror.
7) The slider can be used to change the mode of the display.
By using the slider fabricated in the mirror, the user can
navigate to the detail mode that shows detailed information. In
this case the mirror actually turns into a mere display.
In the next section, we have discussed how these
Fig. 1. AwareMirror in Operation, the mirror displaying weather, functionalities are achieved by utilizing the two different
transportation and schedule information in abstract mode. frameworks.
C. Component View IV. UNDERLYING INFRASTRUCTURE
The following sentient artefacts and sensors have been used
It is obvious that we indeed need a stable platform that
in the AwareMirror application:
makes the application development simple and rapid by
automating many recurring tasks. We have developed two
1) AwareMirror: The mirror is constructed using an acrylic
infrastructures for providing the platform support to context-
magic mirror board and an ordinary computer monitor. The
aware application developers. These two frameworks attack
acrylic board is attached in front of the monitor and only
the problem in different manners. The first one “Bazaar” is a
bright colors from the display can penetrate the board. Two
centralized architecture that attempts to model the
proximity sensors have been embedded into the mirrors. These
environment in a bottom up approach exploiting self-
are used to infer users’ distance/position from the mirror.
descriptive objects, central repository and an inherent location
Furthermore a slide sensor is fabricated on the mirror that is
model. Bazaar is more motivated to provide a suitable world
used to navigate between the abstract and detail mode of the
model than providing support for contextual application.
displayed information.
However it is flexible enough to provide support for handling
2) Toothbrush: A Toothbrush is augmented with a two-axis
contextual information. The second one “Prottoy” is a
accelerometer. It can detect start and end of brushing. This is
distributed one in approach that attempts to model the
achieved by monitoring zero crossing through the
environment in a top down manner and primarily focuses on
differentiation between two latest measurements, i.e. from
providing generic access to the environment for the
plus to minus and vice versa. In addition an RFID tag is
developers. Prottoy is completely dedicated to provide a
fabricated into it, which can be used to detect the toothbrush in
seamless platform for context-aware applications.
a specific location. As a toothbrush is a highly personal
belonging, which is rarely shared with others, we can infer
We have implemented two versions of AwareMirror
that only its owner will use it. This fact we have utilized to
applications using these two frameworks. However before
infer the identification of a person (the owner) by the
giving our experience reports on the development, let us
toothbrush’s state of usage and location.
introduce the two architectures briefly and their integration
3) Web Services: For three categories of information we
with AwareMirror.
have used 3 distinct web services. Yahoo Japan has been used
for the weather forecasting. iCalendar based scheduler service
Fahim Kawsar, Kaori Fujinami, Tatsuo Nakajima 114
A. Bazaar application exists which aggregates information from the two
artefacts and makes this information available to the
Bazaar [17][18] is an infrastructure for modeling the AwareMirror Controller. Upon receiving appropriate context
physical world populated with various sentient artefacts. It information, this controller communicates with the web
provides an application developer with a shared physical space services to extract proper information related to the user and
information repository and with high level APIs to access present them on the mirror.
information and notification of interesting events. The details
of communication between the application space and the
physical space are hidden in Bazaar.
Currently, Bazaar manages and provides the following
types of information: 1) type, 2) location, 3) state-of-use, 4)
owner, 5) timestamp of detection, and 6) IP address/port
number and additional information to control a sentient
artefact. Since Bazaar manages these kinds of information as a
key-value pair, a new one can be easily added.
As can be seen in Figure 2 Bazaar consists of five major
parts. This includes, 1) an identifiable artefact that serves as a
source of low level contextual information and an actuator if Fig. 3. Implementation of AwareMirror using Bazaar
any, 2) an ID detector that provides the approximate location
of a detected artefact within its sensing range, 3) a shared So, here Bazaar acts as the physical space manager, it has the
information repository, 4) a contextual extraction framework snapshot (location, state of use, properties etc) of the self-
that interprets and makes the low level contextual information descriptive artefacts (AwareMirror, Toothbrush) available in
available to the application as highly abstract information, and the environment. The artefacts are identified by the ID
5) an application logic that a developer has to implement. detector component of Bazaar and accordingly their state of
Every artefact is identified using a tag like RFID, where its use and property information is aggregated in the central
related information and the location of the detector are linked repository. The consumer application AwareMirror controller
together. Application can receive artefact specific information exploits this information accordingly to render the output on
specified earlier by querying the shared repository. the mirror.
C. Prottoy
Prottoy [7][8] provides a generic interface to the
applications for interacting with sentient artefacts in a unified
way regardless of their type and properties Prottoy is
composed of few core components and few pluggable
components as shown in Figure 4.
Fig. 2. The architecture of Bazaar, demonstrating major components
B. Implementation of AwareMirror using Bazaar
Figure 3 shows the overall architecture of Aware Mirror on
top of Bazaar. AwareMirror, sentient toothbrush and an
application “AwareMirror Controller” are the components that Fig. 4. The architecture of Prottoy, demonstrating major components
are glued with Bazaar. On top of Bazaar, an integrated
ubiPCMM 2005 115
Core Framework Components: deals with both types of artefacts (physical like toothbrush
and virtual like web service) using the virtual artefact
1) Resource Manager: As the name implies, it simply interfaces. That means application creates instances of virtual
registers the properties, services and context information of artefact by providing the properties of the required artefact.
the artefacts. When application query comes via virtual Virtual artefact internally communicates with the resource
artefacts it responses accordingly manager to locate the AwareMirror, the toothbrush and the
web services. Once these are found, then the application
2) Artefact Wrapper: It encapsulates the sentient artefacts, subscribes to the toothbrush and AwareMirror for context
sensors, actuators or virtual sensors like weather services, information. When the application receives appropriate
scheduler etc. We have provided a template for the developers context information (interpreted by the interpreter), it
to wrap their device drivers or software into this component. communicates with the web services through the virtual
Developers provide the artefact property and location artefact and then actuates the rendering service of
information into this template during deployment. Artefact AwareMirror to display the information. Application uses the
wrapper has its own resource manager that can advertise its interpreter to interpret the context information received from
capabilities when the global resource manager is absent. In the artefacts to map into application specific values. Also the
addition it has a simple security measure using IP filtering, end users of the AwareMirror application can control the
that allows an artefact to control access to its service and participation of the artefacts by the preference manager
protect information from malicious applications. component.
3) Virtual Artefact: It abstracts the smart environments and
provides a unified view. Application constructs virtual artefact
instances from specific context or service requirement. Virtual
artefact communicates with the resource manager and if an
artefact is found virtual artefact communicates with that
artefact. If everything goes fine virtual artefact represents the
artefact in the application. Application can subscribe to this
artefact or can poll for contextual information. Application can
also execute services of the physical artefact. If storage is
enabled, virtual artefact creates storage in the application
layer. If proxy is enabled then the proxy service of virtual
artefact activates when the physical artefact is absent. The
proxy provides the application with a calculated low accurate
context value using the storage.
Components Pluggable to Application:
1) Interpreter: It maps the context value to the interpreted Fig. 5. Implementation of AwareMirror using AwareMirror
value. We argue that context interpretation is highly
application dependent as the same context can be interpreted Virtual artefact solely hides all the communication details
in different ways based on the application requirements. So we and isolates all access issues thus simplifying application
put this component in the application layer. development considerably.
2) Preference Manager: This component is designed for the E. Feature Comparison
end users of the applications, which are developed on top of
Prottoy. It provides the facility to enable or disable the Bazaar’s primary focus is on providing a world model in a
participation of any artefact of the environment in the transparent way centering on the sentient artefact. For that
application based on their preference. purpose Bazaar’s approach is different from Prottoy. As we
have shown Bazaar provides various information of the real
world beyond location to facilitate suitable modeling.
D. Implementation of AwareMirror using Prottoy However modeling the real world is slightly different than the
daily life context aware applications. Thus we felt for another
Figure 5 shows the architecture of AwareMirror using framework concretely focusing on the context aware
Prottoy. The AwareMirror and the toothbrush are wrapped applications and that's how Prottoy came into the scene. In the
into the artefact wrappers. The three web services are also following (Table II) we are providing a comparison of the
wrapped in three distinct wrappers. During the deployment, features between these two frameworks.
these artefacts communicate with the resource manager and
register themselves by providing their identities. Application
Fahim Kawsar, Kaori Fujinami, Tatsuo Nakajima 116
Table II We are not offering them new devices, rather augmenting their
Comparison between Bazaar and Prottoy daily life objects with sensing capability. As a result our
Feature Bazaar Prottoy
approach is well accepted by the end users. This acceptance
Motivation Providing a world Providing a seamless test was done for several applications that integrate multiple
model. platform for context- artefacts [7][16][17].
aware application
development Major Observations:
Approach Centralized and Distributed and top-
bottom-up. Artefact down, Application
itself informs/notifies queries artefact from
However, there are a few issues that we have identified
it’s information. information/event and which require further discussions. First and most
notification important is how can we make a generalized artefact that can
Location-Model Inherent location model No location model is be utilized in multiple scenarios. We must not develop
is present. present. Location application/scenario specific artefacts. Thus deploying
information is statically
artefacts with “one to many” attribute is a big challenge. From
updated.
Attribute Independence Complete. Application Partial. Application
our experience we have identified that it is very difficult to
does need to know any needs to know the generalize an artefacts capability for providing contextual
attribute about the artefacts capability to information as same artefact can be used in different
artefact. acquire the applications in different manner. Still now we could not come
information. up with a guideline that can help us to build a generalized
Security Support No support. Supported.
artefact.
Plug-n-Play Support No support. Full Plug-n-Play via the
Resource Manager
Event Notification Partly event based. Completely event The next issue is what types of sensors are suitable for an
Application needs to based. Application can artefact and where to fabricate it. From our experience we
query for events. be notified instantly have found that, to answer this question first we have to
about low-level events. observe the target environment and the available artefacts.
Context History No historical Context historical
Then we have to identify what information we are looking for
Support information is support is provided.
provided. and what sensors are appropriate for providing the
Proxy Support Not supported. Supported. information. Once this analysis is performed, then we can pick
Preference Support No support. Supported. appropriate artefact and can fabricate the sensors to them. In
most of the cases there are several alternatives, so we cannot
However, since Bazaar’s initial goal is to provide a real confine strict rules here. For example: In AwareMirror we
world model, some features like proxy, security or preference need to identify the presence of a person. For that we used the
support etc. should not be considered as evaluation criterions toothbrush augmented with an accelerometer and an RFID tag.
between the two frameworks. Of course both the architectures These two provides state of use and location. Thus combining
support all other standard requirements like context these information we can infer users’ presence. However we
specification, separation of concern, transparent can also use a shaver or a comb for the same purpose.
However, the reason for which we picked the toothbrush is
communication (spatial and temporal independence), constant
that toothbrush is a personal belonging that is rarely shared
availability etc.
than the other two. So such ad hoc reasoning may lead to the
selection of appropriate artefacts.
V. DISCUSSION
This section discusses about several issues. We have Final issue related to the sentient artefact is what and how
provided our experience on the basis of the performance, context information can be modeled. In our approach we have
advantages, and drawbacks of sentient artefacts and the two provided state-of-use, location and properties (owner, type,
middlewares. However one point to note here is that: though color etc.) of the artefact as the contextual information. The
we have discussed only one application in this paper, our device driver running on the sentient artefacts aggregates this
actual experience has been aggregated from various information. Also some of these are predefined or can be
applications [7][8][17][18][26] that we have developed. manually changed. However we are not claiming these are
sufficient. A new application may come up with a new
requirement. So, we need a guideline that will help us to
A. Sentient Artefact model the low-level sensor data into higher-level context. One
alternative can ontological based context modeling from raw
For developing context-aware application, drawing practical sensor data at individual artefact. Further more same sensor
real life scenario plays the key role. By augmenting data can be modeled in different manner for providing
appropriate artefacts with context sensing capability and by different context. We cannot yet confine any rule for
integrating these artefacts in one or more applications, we can prescribing such context calculation. Thus this issue poses a
eventually implement those scenarios. The major strength of great challenge for the researchers.
considering everyday object as context source is the natural
interaction. Users need not to learn any new technology; they
are using the artefacts in the same manner they are used to.
ubiPCMM 2005 117
B Bazaar specification to the developer during the development time to
Performance and Observation: reduce the probability of such errors.
C. Prottoy
The primary motivation behind the development of these
Performance and Observation:
frameworks was to provide a seamless platform for integrating
multiple sentient artefacts in context-aware applications.
To compensate some of these issues we have build the
Bazaar was our first attempt where we tried to model the real
world exploiting the features of sentient artefacts. Bazaar is second framework. Prottoy’s primary goal is to provide a
successful in hiding the heterogeneity among the artefacts and seamless platform for the application development exploiting
providing a shared repository to the applications. Application it’s distributed nature. This is because during application
developers can use the APIs provided by Bazaar to manipulate development we felt the difficulty of accessing/interfacing
the artefact in an intuitive manner. They don't need to consider with multiple sentient artefacts that have different access
the low level detail of communication or context management. protocols. If the application developer has to handle these
Very simple query semantics are provided by which heterogonous protocols, application development becomes
application can query specific property of the artefacts. Also really cumbersome. One of the major goals of Prottoy is to
these properties are implemented as a key-value pair, thus hide this heterogeneity by providing a unified interface.
adding new property to the artefacts is relatively simple. Application development on top of Prottoy is fairly simple. To
be specific, developers only provide the context to action
The shared repository can support multiple applications at mapping rules. Once the artefacts are wrapped in the artefact
the same time with current context information. So integrating wrapper template, applications can communicate with them in
multiple artefacts has no effect on the applications’ a unified way via the virtual artefact interface. Even the
performance. Thus application development process is very application does not need to know the artefact specific
simple, as we have seen in the AwareMirror application. Once information or even the resource manager. Such simplicity and
the appropriate artefact is deployed, the control flow of the isolation of the access issues make the application
application is clear and smooth due the high level abstraction
development very simple and rapid. Because of its distributed
support from Bazaar. Application only queries the repository
nature, scalability is not a problem. So any number of artefacts
to know the context information and actuate service
can be integrated in a seamless manner to several applications,
accordingly. So, application deals with the repository for
contextual information. Further more Bazaar removes the as long as the artefacts are wrapped in artefact wrapper.
typing conflict and attributes dependency. By providing a
bottom up hierarchy Bazaar relieves the developer from The virtual artefact and artefact Wrapper in conjunction
knowing and analyzing the artefacts’ classes and attributes in provide the generic interface for everything from a sentient
advance. Such simplicity makes Bazaar very flexible to artefact to a single sensor to a web service and to an actuator.
handle. The artefact wrapper provides the generalization that allows
the actual artefact to be replaced anytime with another one.
Drawbacks of Bazaar: The proxy service is a unique feature of Prottoy. Some of the
existing systems provide storage functionality at the artefact
However, Bazaar has some drawbacks also. The layer, our argument is that if the artefact itself is absent in that
centralized approach lacks from the fact of single point of case the storage is also absent. We think the best use of the
failure. Also because of the single-entity, context management context storage or history is the prediction of the context, so it
is difficult when artefacts increase thus paying a scalability should be somewhere that can be accessible when the artefact
penalty. Also Bazaar does not support plug-n-play features. is absent. Virtual artefact perfectly solves the problem by
Thus automatic join/leave of the artefact could not be hosting the storage and providing proxy service. While using
supported. As we have seen, in the AwareMirror application, Prottoy, application developers are free from network
Bazaar could not handle the virtual sentient artefacts like web management issues. The three-layer architecture separates the
services. So the application developer needs to manipulate
application from the physical space completely. Prottoy’s
them manually. This requirement actually was not identified
overall communication model is event based. All the events
when we deploy Bazaar, when our focus was on real world
are handled at real time with proper functioning. Prottoy
modeling. Also complex querying is not well supported in
Bazaar. provides both subscribe-publish and request-response event
models. Furthermore application code is completely
Though Bazaar isolates attribute based access to real world, independent of Prottoy.
from our experience we have identified that developers often
made some mistakes that are revealed at run time. For Prottoy’s distributed nature compensates the problems of
example, if a developer wants to use a “chair”, but by mistake centralized approach of Bazaar; also the resource manager
he/she types “chari”. Bazaar could not provide support for (both global and local) can handle the issue of plug-n-play
such errors. However, as we have mentioned in the future support. Further more in Prottoy any number of
work section that we are working on providing artefacts’ information/properties regarding artefact can be provided at
deployment time or later. Also Prottoy hides the difference
Fahim Kawsar, Kaori Fujinami, Tatsuo Nakajima 118
between the physical artefacts and virtual artefacts like web applications. We are trying to figure out more appropriate
service. Thus application developers can use them in a unified mechanism for the security measure. Regarding the proxy
manner. So Prottoy could handle some of the issues that service, currently we have used only the mean of the stored
Bazaar suffers from. values and the most recent value, but more sophisticated
technique may lead to better predictions. Also, what sort of
Drawbacks of Prottoy: applications are the appropriate clients for the proxy service
and to what extent? The preference component in the current
In spite of having these features, from our experience we version only provides a selection-based approach via a GUI.
have identified that Prottoy suffers from some problems that However, we don’t feel such GUI based preference is suitable
limit its performance. There is no inherent location model in for a context aware application. We are investigating to make
Prottoy; so artefacts location information cannot be updated this component more realistic and effective. One alternative is
dynamically. Currently this information is manually updated. preference by demonstration. We feel this preference policy is
One solution to this problem is adopting Bazaar’s location very important for the practical deployment of the context
detector component. Prottoy cannot advertise automatic aware applications.
contextual information like Bazaar. (The local resource
manager in Prottoy advertises artefacts capability) So Complex query support is another issue. Rule based
applications need to explicitly subscribe/query the artefacts technique has already in the literature but we believe such rule
information. This is a vital issue when we consider the real increases the cognitive burden of the developers. We are
world model. We need substantial level of information from currently working on a prototype physical space programming
the underlying environment beyond location. In this respect IDE [20]. We feel physical space IDE will reduce the
Bazaars performance is well justified. Also Prottoy’s complexity of queries. With such IDE the developer will be
functionality is vastly dependent on artefact wrapper. All automatically informed of available artefacts as he/she
sentient artefacts must need to be wrapped by artefact wrapper plugged into local environment thus isolating the necessity of
for Prottoy’s functionality. complex queries. We are investigating all these issues with
great interest and hope to come up with some interesting
Another issue is the query support. Currently Prottoy results soon.
provides only AND operation, that is artefacts properties and
capabilities can only be concatenated for artefact searching. VI. RELATED WORK
Prottoy cannot handle other combinations (like OR or XOR
etc.) However from the application development experience In this paper we have demonstrated our approach: sentient
we have figured out that our mere AND support is not enough. artefact as context source and middlewares for supporting
application development. So, in this section we have cited the
Prottoy attempts to provide a generic interface by hiding the related work from two points of view; firstly current trend
communication detail, however we cannot prescribe this related to context source and secondly existing context-aware
generalization for heterogeneous sensors. That means how can application frameworks.
we generalize the sensor specific APIs. We could not find any
suitable answer to this question. Also hiding communication A. From Context Source Point of View
from the applications does not always provide optimum
performance. For example how can we ensure such Most of the context aware projects use artefacts that are
communication in a generic way when conserving energy is a either traditional general purpose computing platforms ranging
major requirement (like to communicate with the artefacts from small handheld to large sized high end computers like
connected via wireless network)? Also the proxy and security ParcTabs, or dedicated artefacts designed for providing
component of Prottoy is immature in terms of functionalities. specific contextual information like Active Badge infrared
B. Future Work sensor. However our work is different from these two
approaches as we concretely focus on everyday objects for
In the previous sub-section we have described the problems context capturing without compromising their primary role.
that we are facing currently. Our current research Digital Décor [12] project augments traditional drawer and
investigations are challenging those issues. We are focusing coffee pots to use as a smart storage and a media for informal
on finding a suitable location model that can be adopted for communication respectively. However users are responsible
optimum functionality. The sentient artefact generalization for explicitly using these artefacts for their services. Also they
and context modeling is another big hurdle that we are trying only provide some services (searching, communicating with
to cope with. Several other features that we have introduced in people etc.) rather than any contextual information. Tangible
our two frameworks seek further clarification. For example, Bits [10] project attempts to bridge the physical world and
the access controls mechanism or proxy module’s logic. We virtual world by providing interactive surface, graspable
believe that the simple IP filtering technique used in the objects and ambient media. However such explicit dedicated
current Prottoy prototype is not suitable for all the interfaces violates natural interaction paradigm and natural
augmentation of conventional everyday objects.
ubiPCMM 2005 119
Recently one Internet service [27] provides similar notion addition, by introducing features of the artefact wrapper and
as ours by providing activity information of remote elderly by the virtual artefact, it provides the developer much more
capturing the state of coffee pot. Although they have control and flexibilities over the physical space and the
augmented everyday object the consumer of this information application development process. Bazaar’s information
is not the person who uses the system. It is a kind of repository actively performs the task of Device Agents and
monitoring system, which does not provide any contextual Active maps. However the other components of Bazaar further
information. Paradiso’s work [15] in wearable computing enhance the application development process.
arena matches our vision as he has exploited sensor-
augmented footwear to obtain contextual information. TEA Among the distributed ones, Context Toolkit [2] focuses on
[11][22] project attempts to embed various sensors to augment the component abstraction by providing the notion of Context
handheld devices to provide contextual information. However Widget and Context Aggregator. Discoverer manages these
they only focus on handhelds. MediaCup [21] projects and its components and additionally there is a Context Interpreter
succeeding SmartIts [29] provide insight into the component that performs the task of context interpretation.
augmentation of artefacts with sensing and processing. Our Context Toolkit provides very low-level abstraction.
work is greatly influenced by them and exploits the Aware Developer needs to provide the details about the context
Artefact model introduced in [11]. However our sentient source like location, port etc. Also, the application is
artefacts do not require any explicit interaction as MediaCup inherently dependent on the framework as the application is
or SmartIts based artefact requires. Our approach is to make tightly coupled with the architecture. That means application
artifact aware but not their user aware of this fact. Sentient needs to extend the architecture component and manipulate
artefacts are mere everyday artefacts without any noticeable accordingly. Such low level complexity makes application
feature. Users manipulate them in the natural way they are development very cumbersome. Bazaar approach is different
used to with. They don't need to do something explicitly to than Context Toolkit. Application can access the sentient
make something happen. This natural feature distinguishes our artefacts by the shared repository solely without concerning
work from other projects. any low level detail. Prottoy takes the Context Toolkit
architecture and generalizes it in a single component namely
B. From Middleware Point of View virtual artefact. Using Prottoy, the applications become
independent from the context infrastructure as the virtual
Currently there exist a number of context aware application artefact handles all the lower level tasks. Even the applications
frameworks in the literature. Usually, two approaches have do not need to know about the resource manager. Applications
been investigated for context-aware framework. One is the only use the virtual artefact as a generic component that
centralized server approach, like Schilits System [24] or provides all the infrastructure supports. Prottoy also hides the
Contextual Information Service [14] and the other is the context implementation from the context specification
distributed approach like Context Toolkit [2] or Speitzer’s
work [23]. Speitzer [23] proposed another distributed architecture
based on multicast communication, where context information
Centralized frameworks provide fair performance from the is multicast among the members of the multicast group.
point of view of context acquisition from the sensors and However the disadvantages of their approach are the
providing interpreted context via standard APIs. However they increasing computation and communications thus paying a
suffer from the fact of single point of failure and extensibility scalability penalty. For example an application within a
concerns. Also, collecting information from several sources in multicast group will receive context information even if it
one place makes the framework complex and maintenance does not request for it. The Stick-e Notes system [13] provides
becomes difficult. Bazaar is centered on centralized simple semantics for writing rules that specify what action to
repository. However, as the context sources are distributed perform based on the acquired context, mainly focusing on
among the sentient artefacts, the repository itself is not non-programmers to author context aware services. Both
responsible for modeling or generating the context. Prottoy’s Bazaar and Prottoy generalize this context acquisition from
approach is different from these as it completely distributes programmer’s point of view. Stick-e Notes can be thought of
the context sources into multiple artefact wrappers. as an application on top of Bazaar and Prottoy.
Application can communicate to specific context sources and
can interact with them via the virtual artefact seamlessly. Thus The Sentient Computing Project [1][20] utilizes Active Bat
scalability is well supported in Prottoy. location system to provide an architectural base for indoor
application exploiting a world model. The approach of Bazaar
Schilit’s System [25] deals with the context awareness by and Prottoy is different from the point of view that they offer
Device Agents that maintain the status and the capabilities of the applications to create a context aware environment by
the devices, User Agents that maintain the user policies and constructing an array of artefacts. It means Bazaar and Prottoy
Active Maps that maintain the location information of the specialize the world model creation by allowing developers to
devices and the users. Resource manager and preference construct the model as they want. HP Cool Town [5]
component in Prottoy provide the same functionalities. In encapsulates the world by providing web presence of place,
Fahim Kawsar, Kaori Fujinami, Tatsuo Nakajima 120
people and thing and allows interaction with web presence of [12] Itiro Siio, J. Rowan, N. Mima and E. Mynatt “Digital Decor: Augmented
Everyday Things,” Graphics Interface 2003
these entities primarily exploiting RF technology. Cool Town
[13] J. P. Brown “The Stick-e Document: A framework for creating context
supports only web based context aware applications where aware applications”, In the proceedings of the Electronic Publishing ’96
Bazaar and Prottoy support any classes of context aware [14] J. Pascoe “Adding generic Contextual Capabilities to Wearable
applications. Easy Living [4] focuses on an architecture that Computers ," In the Proceedings of the Second International Symposium
on Wearable Computers, Pages 129-138. IEEE Computer Society Press.
supports the coherent user experience as users interact with
Oct, 1998
variety of devices in a smart environment. Easy Living also [15] J. A. Paradiso, K. Hsiao and A. Benbasat “A. Interfacing the Foot:
utilizes the notion of world model. Open Agent architecture Apparatus and Applications,” CHI 2000 Extended Abstracts
[6] is an agent-based system, which exploits a centralized [16] K. Fujinami, F. Kawsar and T. Nakajima. “AwareMirror: A
Personalized Display using a Mirror” In the proceedings of International
black board to support the contextual behavior. In contrast to
Conference on Pervasive Computing (Pervasive 2005), May 2005
these systems, Bazaar and Prottoy provides a more generic [17] K. Fujinami and T. Nakajima. “Towards System Software for Physical
abstraction as developer has the flexibility to construct the Space Applications” In Proceedings of ACM Symposium on Applied
model by manipulating the sentient artefacts. Computing (SAC) 2005, March 2005.
[18] K. Fujinami, T. Yamabe, and T. Nakajima. “Bazaar: A Conceptual
Framework for Physical Space Applications.” In The 2nd International
VII. CONCLUSION Symposium on Ubiquitous Computing System (UCS2004), November
In this paper we have shared our experiences on building 2004.
[19] K. Fujinami, T. Kambe and Tatsuo Nakajima, “BASE: An Interactive
context-aware applications. We have demonstrated our Location-aware Development Tool for Sentient Environments”, In the
approach by a real life application development process and Poster Proceedings of the Seventh International Conference on
have discussed our findings and difficulties. Two different Ubiquitous Computing, UbiComp2005 (Submitted)
frameworks have also been presented that we have exploited [20] M. Addlesee, R Curwen, S. Hodges, J. Newman, P. Steggels, A. Ward
and A. Hooper. “Implementing a Sentient Computing System” Cover
for developing the applications. Context-Awareness is a key Feature in IEEE Computer, Vol. 34, No. 8, August 2001 pp 50-56
concept of future computing, we believe our experience report [21] M. Beigl, H.W. Gallerson and A. Schmidt “Media Cups: Experience
will help the developers to understand the problems for with design and use of Computer Augmented Everyday Objects,”
realizing some of the issues that are limiting the success of this Computer Networks, special Issue on Pervasive Computing, Mar ’2001
[22] M. Beigl, T. Zimmer and C. Decker. “A Location Model for
key concept. We hope the workshop will provide us with a Communicating and Processing of Context,” Personal and Ubiquitous
good opportunity to share our experiences, to discuss the Computing 6(5-6): 341-357, Dec, 2002
complex issues that mentioned in the paper with the [23] M. Spreitzer “Providing Location Information in a Ubiquitous
researchers and to identify the probable path towards Computing Environment”. In Proceedings of the fourteenth ACM
symposium on Operating System Principles, pages 270-283. ACM Press
resolution. 1993
[24] M. Weiser “The Computer for the 21st Century” Scientific American,
REFERENCES Vol. 265, No.3 1991
[25] N. B. Schilit. “System Architecture for Context Aware mobile
[1] A. Harter, A. Hopper and P. Steggles “The Anatomy of a Context-
Computing” PhD Dissertation, Columbia University New York, 1995
Aware Application,” In Mobile Computing and Networking, pages 59-
[26] T. Yamabe, K. Fujinami and T. Nakajima “Experiences with Building
68, 1999
Sentient Materials using various sensors,” In the Proceedings of the 4th
[2] A. K. Dey, G. Abowd and D. Salber “A Conceptual Framework and a
International Workshop on Smart Appliances and Wearable Computing
toolkit for supporting the rapid prototyping of context-aware
(IWSAWC), pages 445-450, Mar. 2004
applications,” Human-Computer Interaction, Vol-16 2001.
[27] www.mimamori.net
[3] A. Schmidt and V. Laerhoven. “How to build smart appliances,” IEEE
[28] www.naturalinteraction.org
Personal Communications, pages 66-71, 2001
[29] www.smart-its.org
[4] B. L. Brumittet, B. Meyers, J. Krumm, A. Kern and S. Shafer “Easy
Living: technologies for Intelligent Environments” In the proceedings of
the 2nd International Symposium on Handheld and Ubiquitous
Computing ‘2000
[5] C. Deborah and P. Debaty. “Creating Web representations for Places” In
Fahim Kawsar is a MS student in Computer Science
the Proceedings of the 2nd International Symposium on Handheld and
Department of Waseda University. His research interest
Ubiquitous Computing ‘2000.
includes Context-aware Computing, Human Computer
[6] C. Philip R. A. Cheyer, M. Wang and S. C. Baeg “An Open Agent
Interaction, Operating Systems and Web Programming.
Architecture,” In the proceedings of the AAAI Spring Symposium Series
on Software Agents,’94
[7] F. Kawsar, K. Fujinami and T Nakajima “Design and Implementation of
a Software Infrastructure for Integrating Sentient Artefacts,” The 2nd
Annual International Conference on Mobile and Ubiquitous Systems: Kaori Fujinami is an assistant professor of Department
Network and Service, July 2005 (To be published) of Computer Science in Waseda University. His research
[8] F. Kawsar, K. Fujinami and T Nakajima “Prottoy: A Middleware for interest includes Context-aware Computing, Physical
Sentient Environments,” The 2005 IFIP International Conference on Space Oriented Programming, and Information
Presentation.
Embedded And Ubiquitous Computing. Dec 2005, (Submitted)
[9] G.D. Abowd “Software Engineering Issues for Ubiquitous
Computing”, In Proceedings of the 21st International conference on Tatsuo Nakajima is a professor of Department of
Software Engineering, pages 75-84. IEEE Computer Society Press 1999. Computer Science in Waseda University. His interests are
[10] H. Ishii, and B. Ullmer “Tangible Bits: Towards Seamless Interfaces Operating Systems, Distributed Middleware, Real-time
between People, Bits and Atoms,” CHI 1997 Systems and Ubiquitous Computing.
[11] H.W. Gallerson, A. Schmidt and M. Beigl “Adding Some Smartness to
Devices and Everyday Things,” WMCSA 2000