=Paper=
{{Paper
|id=Vol-146/paper-11
|storemode=property
|title=User-initiated context switching using NFC
|pdfUrl=https://ceur-ws.org/Vol-146/paper10.pdf
|volume=Vol-146
|dblpUrl=https://dblp.org/rec/conf/ijcai/Jalkanen05
}}
==User-initiated context switching using NFC==
https://ceur-ws.org/Vol-146/paper10.pdf
User-initiated context switching using NFC
Janne Jalkanen
Insight & Foresight, Nokia Corporation
PO Box 407, FI-00045 Nokia Group, Finland
Janne.Jalkanen@nokia.com
Abstract. Near Field Communication is an application of Radio Frequency
Identification technologies in very short range. In this paper we present an easy
and cheap way for users to set their context using a mobile phone equipped
with an NFC reader device. In one application, the user is able to update
presence information in a web log, in another to convey simple emotions to
another person, and in a third application, update a “mood meter” by touching.
1 Introduction
The promise of an ubiquitous context-sensitive system is that it is able to invisibly
and discreetly adapt to the user's needs, wants and wishes. In order to do this, the
context-sensitive system needs to draw information from a number of sensors and
sources, including things like the clock, calendar, location, movement, previous
information, etc. However, extracting this information can sometimes be difficult,
cumbersome, or downright impossible, resulting in a multitude of failures and
exceptions in the context-sensitive system.
One possible way of getting the more complex context information is simply to
ask the user. However, even when you do it in a relatively non-intrusive way, it can
still be an annoyance. You don’t want the Microsoft Office assistant to pop up every
five minutes and ask: “It looks like you’re writing a letter. Would you like some
help?” In order for such a proactive agent to work properly, it needs to be aware of
the user in a multitude of ways: it needs to know when the user is receptive to such
information, whether the mood is correct, and so on. This is relatively difficult, as it
requires a computer to be “tuned” to the user.
Of course, if the user is not attempting productive work, proactive context setting
becomes easier: e.g. in games and entertainment systems such an agent can be
interwoven as a part of the game, for example the context-aware agents of the game
“Black and White” [22].
The user can also set his context manually, but this is difficult for two reasons:
1. The user needs to be aware of the system and know how to use it. This is,
of course, contrary to the ubiquitous computing vision.
2. Users in general don’t remember to “flip the switch back”: it’s much
easier to forget the lights on when leaving the room, than to forget to turn
them on when entering.
This means that while a context-sensitive system can mostly rely on the
information the user gives it, it cannot trust the lack of information as continuation of
a previous state. It is conceivable that a good context-sensitive system would
combine these approaches: start with whatever the user tells you to do, then do a best-
guess estimate on what the user would like to do next.
109
2 Background
Context is traditionally redefined in every context-awareness paper, but we stick to
Dey’s definition: “Context is any information that can be used to characterize the
situation of an entity. An entity is a person, place, or object that is considered relevant
to the interaction between a user and an application, including the user and
applications themselves.” [15]. Other definitions in the literature tend to include all or
some of the following aspects, especially when talking about mobile technology [6],
[9]:
• Physical context (e.g. time, location, temperature)
• Social context (e.g. social group, friends, work/home, activities of nearby
people)
• Mental context (what the user is doing or feeling.)
Some of these contexts are more easily derived than others; time can usually be
acquired with sufficient accuracy very easily, while other physical properties can be
acquired with some limited accuracy/efficiency using different sensors. Social context
is fuzzier, but some educated guesses can be made using things like Bluetooth
neighborhood [9].
User’s mental context is however often difficult, if not impossible to determine. It
is hard sometimes even for people to pick nuances of moods from the behavior of
other people. The only remotely reliable method is to ask the person (unless, of
course, they are lying or incapable of articulating their feelings).
Since determining the context of the user is a challenging task at best, one must be
prepared for errors. In case of an error, the context sensitive system should not
produce a result that impacts the user in a way that is found disturbing. “Do no harm”
is a major imperative to any context sensitive system.
2.1 Presence
RFC 2779 defines presence is a means for finding, retrieving, and subscribing to
changes in the presence information (e.g. "online" or "offline") of other users. [16]
In practice, presence takes a form of an icon and/or a piece of text that describes
what the user’s current action is, for example “I’m available”, “Do not disturb”, or “In
a meeting, please SMS”. In many instant messaging (IM) systems users also can
include things like the currently playing song or mood.
2.2 Previous work
Using intelligence embedded in the environment in the forms of smart tags is not a
new concept [10]. For example, Siegemund and Florkemeier are using Bluetooth
beacons and RFID tags in the mobile phones [1], however, their smart tags are far
smarter than our “dumb tags”.
Bardram and Hansen [7] present a system similar to ours, where mobile devices
where used to mediate social situations in a context-aware fashion, which they call
“Context mediated awareness”. Their system works with Bluetooth beacons, and
cell-based positioning, and thus also require expensive infrastructure to function.
Another very smart system is presented by Göker et al [8], but this one, too, suffers
from large infrastructure costs.
The practice of putting URLs on objects has been explored in HP's CoolTown
project, though their work concentrates on infrared beacons [21].
110
3 The Setup
3.1 NFC Technology
Near Field Communication is a name for a particular type of communication that
occurs at close distances. It is closely related to RFID technology, but due to different
design, the maximum communication distance is very short, in the order of 10 cm. A
recent study by the German government found that even with very good equipment, it
is extremely difficult to intercept communication between two NFC devices from
more than 2m away [3],[4], even with a very aggressive antenna design.
NFC is not new as such, and in fact it is already used in many commercial
applications, such as ski passes, public transportation tickets, access keys, and
payment systems [11]. The NFC standards are maintained by the NFC Forum [18]
and standardized in ECMA 340 and ISO 18092 [13].
3.1.1 NFC Tags
NFC tags are essentially antennas with little logic, that contain memory. A small tag
contains typically ~50 bytes of user memory, while large tags can have up to 4
kilobytes of memory.
A tag can either be active or passive. An active tag contains its own power source,
whereas a passive tag draws power from the RF field of the reader. Typically most of
the tags in use are passive tags, since they are cheaper to manufacture and require no
charging or power source.
Each tag has an unique ID code, ranging from 4 to 10 bytes, depending on the
manufacturer and technology. These ID codes are usually sufficiently unique within
one manufacturer, though with multiple manufacturers collisions may occur.
3.1.2 NFC Readers
An NFC reader contains an antenna, much like the one in a tag, a microprocessor and
a power source of some kind. It typically also interfaces with a more complex
microprocessor. A device with an NFC reader is usually called an NFC device, and is
capable of bi-directional information transfer with another similar NFC device, as
defined in the NFC-IP1 standard [13].
3.2 Weblog
A weblog is a web site with periodic, time-stamped posts collected on a single page,
typically in a chronological order. They have gained popularity and visibility lately,
both as a self-expression as well as a publishing tool.
In this paper, we used the freely available JSPWiki software as our backend
system. While JSPWiki is technically a Wiki, it also has weblog capabilities [17]. It
was chosen because of its flexibility, easy interfacing from the server, and author's
familiarity with the software.
A dual-processor 1.4 GHz Ubuntu Linux server was used as the backend. All
software was developed in Java, and specifically as Java Server Pages (JSP).
3.3 Nokia NFC Shell
The Nokia NFC shell is a Near Field Communication enabled functional cover,
developed for the Nokia 3220 phone [14]. It replaces the normal battery cover with a
111
slightly thicker version, which contains an NFC reader/writer antenna and microchip.
The reader functions in the 13.56 MHz frequency range, and has a reading distance of
a few centimeters. It supports the ISO-14443-A tag standard, as well as the ISO
18092, ISO 21481, ECMA (340), and ETSI TS 102 190 standards.
The transfer speed varies from 106 kbps to 424 kbps, and the device has also a tag
emulation mode, which allows a second NFC shell or another commercial tag reader
to read the information from it.
The NFC shell comes with a pre-built Java application called “Service Discovery”,
which allows URLs, phone numbers and SMSs to be read from and stored to tags.
In this particular case, we are using both Philips Mifare Ultralight and Mifare
Classic 1k passive tags [19]. The former has a storage capacity of 48 bytes, and the
latter can store up to 1 kB.
4 The Applications
Fig 1. System overview. All applications are built around the same base system.
4.1 Presence determination
In this application, the user is able to announce his presence information using a
simple touch paradigm.
Multiple tags are encoded with a URL pointing at a JSP page residing on the same
server as the weblog. When an user touches a tag (see Fig 3), a Java midlet
application on the phone reads the URL and starts the phone native web browser. The
URL on the tag consists of the web server address as well as the tag ID. Once the
HTTP request is made, the web server parses the HTTP GET request, and determines
the user presence information from a preset table. For example: Tag code is
“048b22b9ca028000” has been defined in the server database to correspond to
presence value of “At work”.
The other possibility would be to simply embed the URL (e.g.
“http://www.ecyrd.com/P.jsp?p=At+work”) in the tag, in which case no lookup table
112
would be required. However, using the tag ID codes allows the predefined presence
strings to be arbitrarily long, due to the limited space in MIFARE Ultralight tags.
After the P.jsp page has the presence information, it interfaces with the Wiki page
repository and writes the presence message to a predefined page. The Wiki software
aggregates a number of pages on the front page as a weblog, and displays the last
received presence status to whoever is viewing the page (see Fig. 2).
Authentication is done with a simple passkey mechanism: whenever the user uses
the system the first time, he is redirected to a JSP page, which requests a passphrase.
Once a valid passphrase is given, a cookie with a long expiry time is saved in the
browser, which allows the user to bypass the authentication phase the next time he
touches a tag. Finally, the user is redirected back to the original page he tried to
access.
Fig 2. How a casual viewer would see the presence information on the weblog.
Fig 3. Touching the tag and initiating a context switch. In this case, the tag has been attached
on a mirror next to the home front door.
4.2 Instant emoticons
The second application is used to transmit moods and feelings to a loved one. A tag is
attached to the backside of a photograph, and an SMS command is written on it. The
tag, when touched, directs the phone to send a simple SMS message with the text
“*hugs*” to the recipient (see Fig. 4). This allows “low-bitrate emotions” to be sent
with little effort, letting the other person know that you are thinking of him/her at that
moment. As the tag is readable through the wallet, the picture does not need to be
removed from the wallet, and neither the wallet needs to be removed from the pocket:
the tag can be read through the fabric for quick access. (However, coins or other
metallic objects do disturb the field, and thus the outward facing side of the wallet
should not contain anything metallic.)
This is similar to the practice of sending empty SMS messages to other people, just
to tell them that one thinks of them.
113
Fig 4. Sending the “hug” SMS by touching a picture. The tag has been attached to the backside
of the picture. The user keeps this picture in his wallet, and is able to send the “hug” without
taking the picture out of the wallet.
4.3 Mood Swinger
Mood is a prolonged state of mind, resulting from a cumulative effect of emotions
[2]. In this application we allow the user to signal simple emotions which then are
aggregated to the weblog to show a mood.
The application consists of a series of tags, each one of which is encoded with a
simple emotion: “Happy”, “Sad”, etc. When a user touches the tag with the mobile
phone, a HTTP GET request to a server is initiated, and the weblog system mentioned
previously is updated.
The difference here to the Presence application is that an emotion dissipates over
time; the level of emotion is increased upon each touch, and the value dilutes over
time towards an indifferent state. For example, the first touch on “happy” will move
him to roughly halfway up in the scale (to “pretty happy”), and a second touch will
move him all the way to “very happy”.
Our system uses three basic axis, along which an emotion can vary:
1. Happy – sad
2. Strained – relaxed
3. Focused – unfocused
The mood is represented as a floating point value varying from –1.0 to 1.0. For
visualization purposes, this has been divided into five states, which are (in the
order of strength):
1. Very (0.9+)
2. Quite (0.7-0.9)
3. Pretty (0.4-0.7)
4. Somewhat (0.2-0.4)
5. Slightly (0.05-0.2)
6. (Indifferent – this is the zero state, corresponding to values less than 0.05)
For example, “Very happy” would mean a value > 0.9, whereas somewhat sad
would mean a value around -0.3.
As the mood moves towards indifferent, so does the indicator on the weblog. The
mood value is updated discreetly every T seconds and reduced by p per cent. The
values of T and p are chosen so that an emotion dissipates to indifference in slightly
under two hours.
114
Fig. 5. How the mood dissipates over time.
Fig 6. How the mood is visible in the weblog. The “Now playing” -field is extracted from
iTunes, and shows yet another new kind of presence/context information, not discussed in this
paper.
5 Discussion
We have presented a simple way of using an off-the shelf system to make the user
indicate his physical or mental context at any given time. Using a standard mobile
phone has several advantages:
• A phone is something that the user typically carries all the time.
• The device has on-demand connectivity in most areas.
• The touch paradigm is easy and intuitive, and does not require any
complicated menu traversal.
• Works well even if the user cannot see the display. This can be a benefit
to people with poor eyesight.
• Having the reader in the phone eases privacy concerns: user is in control
of his information all the time. In a typical RFID-based system anyone
could set up a reader and read passing tags, which would be a privacy
concern, as a passive tag cannot be turned off the same way as the phone
can.
The system is relatively cheap and easy to set up: the user needs a single NFC-
enabled phone, and tags are readily available in the open market. A typical ubiquitous
computing system, where the intelligence embedded in the environment requires a
complicated infrastructure is expensive to install and maintain, whereas our system
scales by multiplication of cheap tags and keeping the intelligence and connectivity in
the mobile device.
115
However, this is still a far cry from a fully ubiquitous system, where the computer
does all the work for the user. It still requires conscious user action to trigger the
context changing event, and also requires physical access to the tags (which in many
cases could be argued to be a benefit as well). In some cases it was found that no
suitable tag was present, or the phone was inconveniently located and not used. In
addition, if there is no concrete, immediate benefit (such as feedback from other
users) from switching the context (or the mood), the initial interest in it tends to wane
rapidly. Also, not having such a mobile phone would mean no access to most of the
functionalities, therefore making it difficult for casual visitors to participate.
Also, the fact that the device required confirmations for each access was
experienced as a burden. The lag between touching an object and the browser actually
starting and connecting also contributed to a non-optimal user experience.
Is this then more convenient than having the environment watch the user? Yes, for
some purposes. Since determining the user's “mental context” is difficult for even a
complex computer program, this kind of explicit context setting is a reliable way of
finding the user's context. But, as mentioned earlier, this information loses its
relevance as time goes on, as users cannot be expected to continuously update the
state of the context-sensitive system. However, it is possible to “fix” the estimates of a
system using user-input, in much the same way as a GPS device uses auxiliary
methods like accelerometers, pedometers, or dead-reckoning to compensate for the
times it has no satellite coverage.
One important thing is that the user needs some immediate benefit from setting the
context: the Remote Hug -case, while not really very complicated, nor useful at first
sight, provides the most feedback to the user by nurturing the generic human need of
being social, and therefore would also give an ubiquitous computing system the most
valuable information for a longer period. Also, making the user's mood visible in the
weblog allows other people to access this information, making feedback from people
possible (“oh, you have been feeling sad for days now; what's wrong?”) A computer
system can tap in this flow of information between people, and learn from it; rather
than forcing the user to communicate directly with the computer and figuring out the
proper responses.
5.1 Future work
The system, as presented here, is at a relatively “hackish” stage. The JSPWiki
engine is stable, and easily enhanced, but it is not exactly a highly advanced context
awareness platform. The NFC system could easily be interfaced with other similar
systems that offer a web services or SMS-based interface. Some further possibilities
include:
• Hooking up to an IM client (e.g. Apple iChat, using Applescript). This
would allow interfacing with commercial presence systems.
• More efficient modeling of emotions, using established virtual character
emotion modeling tools [5],[12],[20].
• Extending the range of emotions available to the user, while making it
configurable.
• Distributing the devices to an user group, and doing usability
observations.
The whole project was intended to explore what can be done using off-the-shelf,
cheap components by interfacing them with existing person-to-person communication
systems like SMS and weblogs, which are not context-aware as such. One further
avenue of study would be to augment existing context-aware systems, both
commercial and research prototypes.
116
5.2 Acknowledgments
This work could not have been done without the dedicated effort of the NFC teams in
Nokia. My gratitude goes to the NVO 3220 NFC shell team, Sebastian Nyström,
Carmen Kühl, and Heikki Huomo. Special thanks go to Tatu Koljonen for providing
the opportunity to write about this, and Chris Heathcote and Matt Jones for being
such good company.
6 References
1. Frank Siegemund and Christian Florkemeier: Interaction in Pervasive Computing
Settings using Bluetooth-Enabled Active Tags and Passive RFID Technology together
with Mobile Phones. Proceedings of the First IEEE International Conference on
Pervasive Computing and Communications (PerCom’03). IEEE Computer Society,
2003.
2. Sumedha Kshirsagar: Virtual Humans Personified. Proceedings Autonomous Agents
Conference (AAMAS) 2002, pp 356-359., July, 2002.
3. Thomas Finke, Harald Kelter: Radio Frequency Identification – Abhörmöglichkeiten
der Kommunikation zwischen Lesegerät und Transponder am Beispiel eines ISO14443-
Systems. Bundesamt für Sicherheit in der Informationstechnik 2004. Online document:
http://www.bsi.bund.de/fachthem/rfid/whitepaper.htm
4. Risiken und Chancen des Einsatzes von RFID-Systemen. ISBN 3-922746-56-X.
Bundesamt für Sicherheit in der Informationstechnik–BSI, Godesberger Allee 185-189,
53175 Bonn. 128p. 2004.
5. Eva Hudlicka: Beyond Cognition: Modeling Emotion in Cognitive Architectures. In
Proceedings of the Sixth International Conference on Cognitive Modeling, 118-123.
Mahwah, NJ: Lawrence Earlbaum. ICCM2004, Pittsburgh, USA.
6. Panu Korpipää, Jonna Häkkilä, Juha Kela, Sami Ronkainen, Ilkka Känsälä: Utilising
Context Ontology in Mobile Device Application Personalisation. Proceedings of the 3rd
international conference on Mobile and ubiquitous multimedia. College Park, Maryland,
2004. pp. 133 – 140. ISBN:1-58113-981-0.
7. Jakob E. Bardram and Thomas R. Hansen: The AWARE Architecture: Supporting
Context-Mediated Social Awareness in Mobile Cooperation. In Proceedings of
CSCW’04, November 6–10, 2004, Pages: 192 – 201, Chicago, Illinois, USA.
8. Ayse Göker, Stuart Watt, Hans I. Myrhaug, Nik Whitehead, Murat Yakici1,Ralf Bierig,
Sree Kanth Nuti, Hannah Cumming: An Ambient, Personalised, and Context-Sensitive
Information System for Mobile Users. Proceedings of the 2nd European Union
symposium on Ambient intelligence, 8-10 November 2004, Eindhoven, the
Netherlands. pp. 19–24.
9. Antti Sorvari, Janne Jalkanen, Riitta Jokela, Alastair Black, Kimmo Koli, Marko
Moberg, Turkka Keinonen: Usability Issues in Utilizing Context Metadata in Content
Management of Mobile Devices. Proceedings of NordiCHI '04, October 23-27, 2004
Tampere, Finland. pp.357-363.
10. Roy Want, Kenneth P. Fishkin, Anuj Gujar, Beverly L. Harrison: Bridging Physical and
Virtual Worlds with Electronic Tags. Proceedings of CHI ‘99 Pittsburgh PA, USA.
pp.370-377.
11. Jiajun Jim Chen, Carl Adams: Short-range Wireless Technologies with Mobile Payments
Systems. Proceedings of the 6th international conference on Electronic commerce. pp:
649 – 656. Delft, The Netherlands 2004. ISBN:1-58113-930-6.
12. Aaron Sloman: Beyond Shallow Models of Emotion. Workshop on Behaviour planning
for life-like avatars, 13 Spring Days Workshop, March 1999, Sitges, Spain.
13. ECMA-340: Near Field Communication - Interface and Protocol (NFCIP-1). 2nd edition
(December 2004). Available on-line at
http://www.ecma-international.org/publications/standards/Ecma-340.htm
14. Nokia NFC Shell: http://www.nokia.com/nokia/0,,66260,00.html
15. Dey, Anind: Understanding and Using Context. Personal & Ubiquitous Computing,
Vol5, Issue 1, Feb. 2001, 4-7.
117
16. M. Day, S. Aggarwal, G. Mohr, J. Vincent: RFC 2779 - Instant Messaging / Presence
Protocol Requirements. On-line document, available at
http://www.ietf.org/rfc/rfc2779.txt
17. Janne Jalkanen: JSPWiki. On-line document: http://www.jspwiki.org/
18. NFC Forum: http://www.nfc-forum.org/
19. http://www.mifare.net/
20. Ken Perlin, A. Goldberg: Improv: A System for Scripting Interactive Actors in Virtual
Worlds. Computer Graphics; Vol. 29 No. 3., 1996
21. Tim Kindberg, John Barton, Jeff Morgan, Gene Becker, Debbie Caswell, Philippe
Debaty, Gita Gopal, Marcos Frid, Venky Krishnan, Howard Morris, John Schettino, Bill
Serra, Mirjana Spasojevic: People, places, things: web presence for the real world.
Mobile Networks and Applications Volume 7, Issue 5 (October 2002) pp. 365 – 376.
ISSN:1383-469X
22. Peter Molyneux: Black & White. PC Game, published by Lionhead Games, 2001.
118