The promise of an ubiquitous context-sensitive system is that it is able to invisibly and discreetly adapt to the user's needs, wants and wishes. In order to do this, the context-sensitive system needs to draw information from a number of sensors and sources, including things like the clock, calendar, location, movement, previous information, etc. However, extracting this information can sometimes be difficult, cumbersome, or downright impossible, resulting in a multitude of failures and exceptions in the context-sensitive system.
One possible way of getting the more complex context information is simply to ask the user. However, even when you do it in a relatively non-intrusive way, it can still be an annoyance. You don't want the Microsoft Office assistant to pop up every five minutes and ask: "It looks like you're writing a letter. Would you like some help?" In order for such a proactive agent to work properly, it needs to be aware of the user in a multitude of ways: it needs to know when the user is receptive to such information, whether the mood is correct, and so on. This is relatively difficult, as it requires a computer to be "tuned" to the user.
Of course, if the user is not attempting productive work, proactive context setting becomes easier: e.g. in games and entertainment systems such an agent can be interwoven as a part of the game, for example the context-aware agents of the game "Black and White" [22].
The user can also set his context manually, but this is difficult for two reasons:
1. The user needs to be aware of the system and know how to use it. This is, of course, contrary to the ubiquitous computing vision. 2. Users in general don't remember to "flip the switch back": it's much easier to forget the lights on when leaving the room, than to forget to turn them on when entering. This means that while a context-sensitive system can mostly rely on the information the user gives it, it cannot trust the lack of information as continuation of a previous state. It is conceivable that a good context-sensitive system would combine these approaches: start with whatever the user tells you to do, then do a bestguess estimate on what the user would like to do next.
Context is traditionally redefined in every context-awareness paper, but we stick to Dey's definition: "Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves." [15]. Other definitions in the literature tend to include all or some of the following aspects, especially when talking about mobile technology [6], [9]:
• Physical context (e.g. time, location, temperature)
• Social context (e.g. social group, friends, work/home, activities of nearby people) • Mental context (what the user is doing or feeling.) Some of these contexts are more easily derived than others; time can usually be acquired with sufficient accuracy very easily, while other physical properties can be acquired with some limited accuracy/efficiency using different sensors. Social context is fuzzier, but some educated guesses can be made using things like Bluetooth neighborhood [9].
User's mental context is however often difficult, if not impossible to determine. It is hard sometimes even for people to pick nuances of moods from the behavior of other people. The only remotely reliable method is to ask the person (unless, of course, they are lying or incapable of articulating their feelings).
Since determining the context of the user is a challenging task at best, one must be prepared for errors. In case of an error, the context sensitive system should not produce a result that impacts the user in a way that is found disturbing. "Do no harm" is a major imperative to any context sensitive system.
RFC 2779 defines presence is a means for finding, retrieving, and subscribing to changes in the presence information (e.g. "online" or "offline") of other users. [16] In practice, presence takes a form of an icon and/or a piece of text that describes what the user's current action is, for example "I'm available", "Do not disturb", or "In a meeting, please SMS". In many instant messaging (IM) systems users also can include things like the currently playing song or mood.
Using intelligence embedded in the environment in the forms of smart tags is not a new concept [10]. For example, Siegemund and Florkemeier are using Bluetooth beacons and RFID tags in the mobile phones [1], however, their smart tags are far smarter than our "dumb tags".
Bardram and Hansen [7] present a system similar to ours, where mobile devices where used to mediate social situations in a context-aware fashion, which they call "Context mediated awareness". Their system works with Bluetooth beacons, and cell-based positioning, and thus also require expensive infrastructure to function.
Another very smart system is presented by Göker et al [8], but this one, too, suffers from large infrastructure costs.
The practice of putting URLs on objects has been explored in HP's CoolTown project, though their work concentrates on infrared beacons [21].
Near Field Communication is a name for a particular type of communication that occurs at close distances. It is closely related to RFID technology, but due to different design, the maximum communication distance is very short, in the order of 10 cm. A recent study by the German government found that even with very good equipment, it is extremely difficult to intercept communication between two NFC devices from more than 2m away [3], [4], even with a very aggressive antenna design.
NFC is not new as such, and in fact it is already used in many commercial applications, such as ski passes, public transportation tickets, access keys, and payment systems [11]. The NFC standards are maintained by the NFC Forum [18] and standardized in ECMA 340 and ISO 18092 [13].
NFC tags are essentially antennas with little logic, that contain memory. A small tag contains typically ~50 bytes of user memory, while large tags can have up to 4 kilobytes of memory.
A tag can either be active or passive. An active tag contains its own power source, whereas a passive tag draws power from the RF field of the reader. Typically most of the tags in use are passive tags, since they are cheaper to manufacture and require no charging or power source.
Each tag has an unique ID code, ranging from 4 to 10 bytes, depending on the manufacturer and technology. These ID codes are usually sufficiently unique within one manufacturer, though with multiple manufacturers collisions may occur.
An NFC reader contains an antenna, much like the one in a tag, a microprocessor and a power source of some kind. It typically also interfaces with a more complex microprocessor. A device with an NFC reader is usually called an NFC device, and is capable of bi-directional information transfer with another similar NFC device, as defined in the NFC-IP1 standard [13].
A weblog is a web site with periodic, time-stamped posts collected on a single page, typically in a chronological order. They have gained popularity and visibility lately, both as a self-expression as well as a publishing tool.
In this paper, we used the freely available JSPWiki software as our backend system. While JSPWiki is technically a Wiki, it also has weblog capabilities [17]. It was chosen because of its flexibility, easy interfacing from the server, and author's familiarity with the software.
A dual-processor 1.4 GHz Ubuntu Linux server was used as the backend. All software was developed in Java, and specifically as Java Server Pages (JSP).
The Nokia NFC shell is a Near Field Communication enabled functional cover, developed for the Nokia 3220 phone The transfer speed varies from 106 kbps to 424 kbps, and the device has also a tag emulation mode, which allows a second NFC shell or another commercial tag reader to read the information from it.
The NFC shell comes with a pre-built Java application called "Service Discovery", which allows URLs, phone numbers and SMSs to be read from and stored to tags.
In this particular case, we are using both Philips Mifare Ultralight and Mifare Classic 1k passive tags [19]. The former has a storage capacity of 48 bytes, and the latter can store up to 1 kB.
In this application, the user is able to announce his presence information using a simple touch paradigm.
Multiple tags are encoded with a URL pointing at a JSP page residing on the same server as the weblog. When an user touches a tag (see Fig 3), a Java midlet application on the phone reads the URL and starts the phone native web browser. The URL on the tag consists of the web server address as well as the tag ID. Once the HTTP request is made, the web server parses the HTTP GET request, and determines the user presence information from a preset table. For example: Tag code is "048b22b9ca028000" has been defined in the server database to correspond to presence value of "At work".
The other possibility would be to simply embed the URL (e.g. "http://www.ecyrd.com/P.jsp?p=At+work") in the tag, in which case no lookup table would be required. However, using the tag ID codes allows the predefined presence strings to be arbitrarily long, due to the limited space in MIFARE Ultralight tags.
After the P.jsp page has the presence information, it interfaces with the Wiki page repository and writes the presence message to a predefined page. The Wiki software aggregates a number of pages on the front page as a weblog, and displays the last received presence status to whoever is viewing the page (see Fig. 2).
Authentication is done with a simple passkey mechanism: whenever the user uses the system the first time, he is redirected to a JSP page, which requests a passphrase. Once a valid passphrase is given, a cookie with a long expiry time is saved in the browser, which allows the user to bypass the authentication phase the next time he touches a tag. Finally, the user is redirected back to the original page he tried to access.
The second application is used to transmit moods and feelings to a loved one. A tag is attached to the backside of a photograph, and an SMS command is written on it. The tag, when touched, directs the phone to send a simple SMS message with the text "*hugs*" to the recipient (see Fig. 4). This allows "low-bitrate emotions" to be sent with little effort, letting the other person know that you are thinking of him/her at that moment. As the tag is readable through the wallet, the picture does not need to be removed from the wallet, and neither the wallet needs to be removed from the pocket: the tag can be read through the fabric for quick access. (However, coins or other metallic objects do disturb the field, and thus the outward facing side of the wallet should not contain anything metallic.) This is similar to the practice of sending empty SMS messages to other people, just to tell them that one thinks of them.
Mood is a prolonged state of mind, resulting from a cumulative effect of emotions [2]. In this application we allow the user to signal simple emotions which then are aggregated to the weblog to show a mood.
The application consists of a series of tags, each one of which is encoded with a simple emotion: "Happy", "Sad", etc. When a user touches the tag with the mobile phone, a HTTP GET request to a server is initiated, and the weblog system mentioned previously is updated.
The difference here to the Presence application is that an emotion dissipates over time; the level of emotion is increased upon each touch, and the value dilutes over time towards an indifferent state. For example, the first touch on "happy" will move him to roughly halfway up in the scale (to "pretty happy"), and a second touch will move him all the way to "very happy".
Our system uses three basic axis, along which an emotion can vary: 1. Happy -sad 2. Strained -relaxed 3. Focused -unfocused The mood is represented as a floating point value varying from -1.0 to 1.0. For visualization purposes, this has been divided into five states, which are (in the order of strength):
1. Very (0.9+) 2. Quite (0.7-0.9) 3. Pretty (0.4-0.7) 4. Somewhat (0.2-0.4) 5. Slightly (0.05-0.2) 6. (Indifferent -this is the zero state, corresponding to values less than 0.05) For example, "Very happy" would mean a value > 0.9, whereas somewhat sad would mean a value around -0.3.
As the mood moves towards indifferent, so does the indicator on the weblog. The mood value is updated discreetly every T seconds and reduced by p per cent. The values of T and p are chosen so that an emotion dissipates to indifference in slightly under two hours.
We have presented a simple way of using an off-the shelf system to make the user indicate his physical or mental context at any given time. Using a standard mobile phone has several advantages:
• A phone is something that the user typically carries all the time.
• The device has on-demand connectivity in most areas.
• The touch paradigm is easy and intuitive, and does not require any complicated menu traversal. • Works well even if the user cannot see the display. This can be a benefit to people with poor eyesight. • Having the reader in the phone eases privacy concerns: user is in control of his information all the time. In a typical RFID-based system anyone could set up a reader and read passing tags, which would be a privacy concern, as a passive tag cannot be turned off the same way as the phone can. The system is relatively cheap and easy to set up: the user needs a single NFCenabled phone, and tags are readily available in the open market. A typical ubiquitous computing system, where the intelligence embedded in the environment requires a complicated infrastructure is expensive to install and maintain, whereas our system scales by multiplication of cheap tags and keeping the intelligence and connectivity in the mobile device.
However, this is still a far cry from a fully ubiquitous system, where the computer does all the work for the user. It still requires conscious user action to trigger the context changing event, and also requires physical access to the tags (which in many cases could be argued to be a benefit as well). In some cases it was found that no suitable tag was present, or the phone was inconveniently located and not used. In addition, if there is no concrete, immediate benefit (such as feedback from other users) from switching the context (or the mood), the initial interest in it tends to wane rapidly. Also, not having such a mobile phone would mean no access to most of the functionalities, therefore making it difficult for casual visitors to participate.
Also, the fact that the device required confirmations for each access was experienced as a burden. The lag between touching an object and the browser actually starting and connecting also contributed to a non-optimal user experience.
Is this then more convenient than having the environment watch the user? Yes, for some purposes. Since determining the user's "mental context" is difficult for even a complex computer program, this kind of explicit context setting is a reliable way of finding the user's context. But, as mentioned earlier, this information loses its relevance as time goes on, as users cannot be expected to continuously update the state of the context-sensitive system. However, it is possible to "fix" the estimates of a system using user-input, in much the same way as a GPS device uses auxiliary methods like accelerometers, pedometers, or dead-reckoning to compensate for the times it has no satellite coverage.
One important thing is that the user needs some immediate benefit from setting the context: the Remote Hug -case, while not really very complicated, nor useful at first sight, provides the most feedback to the user by nurturing the generic human need of being social, and therefore would also give an ubiquitous computing system the most valuable information for a longer period. Also, making the user's mood visible in the weblog allows other people to access this information, making feedback from people possible ("oh, you have been feeling sad for days now; what's wrong?") A computer system can tap in this flow of information between people, and learn from it; rather than forcing the user to communicate directly with the computer and figuring out the proper responses.
The system, as presented here, is at a relatively "hackish" stage. The JSPWiki engine is stable, and easily enhanced, but it is not exactly a highly advanced context awareness platform. The NFC system could easily be interfaced with other similar systems that offer a web services or SMS-based interface. Some further possibilities include:
• Hooking up to an IM client (e.g. Apple iChat, using Applescript). This would allow interfacing with commercial presence systems. • More efficient modeling of emotions, using established virtual character emotion modeling tools [5], [12], [20]. • Extending the range of emotions available to the user, while making it configurable. • Distributing the devices to an user group, and doing usability observations.
The whole project was intended to explore what can be done using off-the-shelf, cheap components by interfacing them with existing person-to-person communication systems like SMS and weblogs, which are not context-aware as such. One further avenue of study would be to augment existing context-aware systems, both commercial and research prototypes.
This work could not have been done without the dedicated effort of the NFC teams in Nokia. My gratitude goes to the NVO 3220 NFC shell team, Sebastian Nyström, Carmen Kühl, and Heikki Huomo. Special thanks go to Tatu Koljonen for providing the opportunity to write about this, and Chris Heathcote and Matt Jones for being such good company.