For autonomous agents to achieve their full potential they require access to detaile d private information about individuals they are designed to assist. The time is rapidly approaching when we can build systems to gather this information and monitor all aspects of an individual's life. In this paper we describe Tempus Fugit (Time Flies), an attempt to create just such a system. The reality of this technology has enormous social implications and, misused, it creates direct threats to liberty. We further describe an “e-Social Contract”, a design philosophy developed to safeguard against these threats. It is the foundation of the design philosophy behind Tempus Fugit and should be considered in the development of any agent technology.
The technology to enable people to remain in contact with their
computer systems while they roam from place-to-place and switch
from activity-to-activity is rapidly being deployed[3][
The threat before us is that the development and deployment of autonomous agent technology, coupled with new communications technologies (mobile phones) will, in the guise of helping its users, create an infrastructure capable of monitoring the activities of large Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
Conference ’00, Month 1-2, 2000, City, State.
Copyright 2000 ACM 1-58113-000-0/00/0000…$5.00. numbers of people. This infrastructure could be abused to control people in a manner never before possible.
According to Hobbes we give up basic rights for the personal
benefits we gain from living in society[
Du contrat social asserts that individuals possess certain basic rights. They may choose to delegate certain rights for the greater good because they, as individuals, also benefit from living in a free Society and working in partnership with others. However, the Government must protect the rights of all.
An e-Social Contract must recognize that individuals also have basic rights pertaining to the use of software technology. These rights are intrinsic to all individuals and should be respected in the design of that technology. Users may delegate rights to an agent for their benefit or the greater good of a community, but the technology itself must never deprive the individual of any rights. These rights, liberties, and privileges are possessions of all individuals. To protect rights under an e-Social Contract one must first define them. We ascribe the following as rights, liberties, and privileges of the individual user. varying degrees. We chose a coordinates system to independently characterize: (Axis A) the degree of Autonomy of an agent, (Axis B) the Beneficiary of an agent, and (Axis I) the Impact an agent has or is designed to have. Note that none of these axes denote the performance or effectiveness of the software, only the intent of the software design.
permission by the owner of that information.
B.
property is under the direct control of the user.
also act for a third party without user permission.
private by default and should not be exposed.
Having defined the rights of individuals, one must next understand
how different classes of “agent” technology affect them. As
Hobbes, wrote in De Cive (The Citizen), “for every thing is best
understood by its constitutive causes; for as in a watch, or some
such small engine, the matter, figure, and motion of the wheeles,
cannot well be known, except it be taken in sunder, and viewed in
parts”[
The software components that define the broad class of programs
we call agents are as varied and numerous as the parts in a
Hobbes’ watch[2][
Clearly there is a continuum of agent technology that may reside anywhere in the unit cube defined in the 3-D space (A, I, B) of Figure 2. Agents may act with varying degrees of autonomy, having varying impact, on behalf of the user or others. Based on this coordinate system, we consider 4 classes of agent technology. First, we define agent-less technology as software that uses no agents and depends solely on direct manipulation by the user. Agent-less technology corresponds to the plane A=0. Software in this plane has no active components.
Second, we define collaborative agents as technology that acts only on behalf of the user, whose intended beneficiary is the user (B=0). Purely collaborative agents reside in the A-I plane. They may seek out information or resources for a user and may act upon that information with varying degrees of autonomy. A collaborative agent might look for a source of books on the Internet based on user input. An agent that recommends books on behalf of an Internet business is not collaborative, although it could be of some benefit to a user (B>0). Third, we consider informing Agents, which only provide information. An informing agent will never take action that would affect the user without direct approval by the user. It may act with varying degrees of autonomy and may gather information for the user or for some other party. Purely informing agents reside in the A-B plane.
Finally, we define autonomous agents as fully independent software that accesses user information and takes action without even requiring the user be aware of said action. These agents act with complete autonomy. The actions may or may not be on the users behalf. The system may gather information or take action to impact the user or others. Purely autonomous agents reside in a plane parallel to the B-I plane with a normalized autonomy value of 1.0 (100% autonomous).
To apply an e-Social Contract, one must first determine within this
space, how these technologies affect the rights of a user. An
individual user of a collaborative-informing agent must trust the
software if the user is to rely only on the agent for information and
services. However, the risk accepted by the user of an informing
agent is limited. The agent requires permission of the user to spend
money or give out personal information (such as a credit card
number, billing address, and telephone number). Agents that take
autonomous action on a user’s behalf require a greater degree of
trust. In 1994, Pattie Maes proposed a system wherein users begin
to trust an agent to act autonomously once their confidence in the
software rises above some fixed threshold (e.g. 80%)[
The e-Social Contract dictates that the individual must retain the rights to define and restrict the behavior of potentially invasive technology. The permissions a user chooses to grant or deny to a software system will depend on the properties of the system with respect to the (A, I, B) coordinates. Figure 3 represents a possible permission-surface defined by a hypothetical user. In this example the user allows a high degree of autonomy only for components that act entirely on the users behalf. For any agent that acts for a different beneficiary, the allowed autonomous behavior drops exponentially. This particular user permits significant information to be gathered on behalf of some external beneficiary (e.g., another member of the system) but these permissions are also severely restricted as soon as the information gathering begins to affect the user. Agent behaviors enclosed by this permission surface are enabled for this user. Agents whose activities extend beyond this surface are disabled. Individuals could, no doubt, choose a different surface. The e-Social Contract insists that the power to define the permission-surface must remain with the user.
The active and autonomous components of Tempus Fugit were designed to conform to the proposed e-Social Contract. Below we describe the Tempus Fugit system, its implementation, and how it fulfills the e-Social Contract.
At its core Tempus Fugit [4] is a "smart" electronic personal
information management (PIM) system. The use of electronic PIMs
has grown dramatically in the past few years with widespread
adoption of Personal Digital Assistants (PDAs) (e.g., Palm Pilot,
Palm PC, etc.), mobile phones with integrated calendars, and
"groupware" such as Lotus Notes and Microsoft Exchange[
The Tempus Fugit PIM records the details of a user’s life in an accessible electronic form that can easily be processed. Tempus Fugit knows the past, current and future activities of its users. It knows where its users are, where they are scheduled to be, what they are scheduled to do, and with whom they plan to meet. From the "to do" list, it knows the tasks its users want to accomplish, and from the “resource book” (a more generalized extension of the idea of an address book), Tempus Fugit knows the people the users know and the objects they own. In addition, Tempus Fugit incorporates other information not usually associated with PIMs that help it more effectively assist its users. In particular it incorporates the ability to track and record the physical and virtual locations of users, objects and other resources known to the system. Tempus Fugit goes beyond conventional PIM systems in other dimensions as well. Tempus Fugit is also a Social Information Management. (SIM) system. A SIM manages the social connections users have to their personal communities. Tempus Fugit attempts to make its users more effective in managing their social relationships.
The value that users derive from Tempus Fugit can be partitioned into two areas: automation and awareness. The combination creates several modes of functionality: personal automation, social automation, personal awareness and social awareness. The area of personal automation leverages the information users maintains about their schedule (calendar), goals (“to-do list”) and people they know (address book), to implement features that automate tasks for the user. In the calendar, for instance, the system will automatically generate background intelligence information for a user’s meetings. This involves aggregating available information such as news reports, business summaries and stock information. The calendar also exploits the location tracking information collected by the system to predict attendance and arrival times for meeting participants. For instance, the system will predict that someone will be late and provides an estimated time of arrival given their current velocity.
The social automation features of Tempus Fugit allow users to discover other people that may wish to meet. This discovery can be manual, or with Tempus Fugit’s help automatic. For instance, a user being informed of a new person joining an organization who has characteristics that match theirs (e.g., they speak Finnish). The personal awareness features of Tempus Fugit concentrate of providing access to the system through mobile devices (i.e., mobile phones). For instance, the system is fully accessible to WAP/WML enabled mobile phones. To expand this interface we have also developed automatic voice summarizations of the system contents represented in VoiceXML? that can be rendered in a variety of formats. In particular we create MP3 files of text-to-speech output. The social awareness features of Tempus Fugit concentrate on informing users of the status of other users in their self-defined social circle.
Tempus Fugit is implemented with a scaleable architecture designed to support large numbers of users. The core of the system is implemented using Java? Servlets and Enterprise Java Beans. A relational database is employed as the system's persistent storage. It runs on both Windows 2000 and Linux. The architecture includes a layer of XSLT translation between end users and the main system that allows great flexibility in supporting different output formats and devices; Tempus Fugit generates HTML, WML and VoiceXML? from the same XML data simply by providing three different XSL style sheets.
The architecture of Tempus Fugit includes an event distribution mechanism based on the Java? Messaging Service (JMS). All external information enters Tempus Fugit initially as an event and many "state changing" operations that occur within Tempus Fugit are made visible as events in this system. For instance, as location transponders report the locations of users, these are turned into events that propagate through JMS and into the core Tempus Fugit database of user positions. The flexibility of this architecture has allowed us to integrate a "rule engine" into Tempus Fugit as a subscriber and publisher of events. This rule engine enables many other powerful features that are beyond the scope of this paper. The location-tracking features of the system are exercised by five vehicles equipped with "PinPoint" CDPD GPS location transponders from AirLink Communications[1]. These send a UDP packet containing the latitude and longitude of the vehicle to Tempus Fugit every 5 minutes. The system also tracks the location of individual laptop computers that participate in a wireless Ethernet LAN. In that case, a utility running on the laptop beacons the id of the wireless access point (i.e., the "cell") it is connected to. Full support for national languages and localization is designed into the core of Tempus Fugit. Currently Tempus Fugit supports US and Canadian English, Swedish, German and Finnish.
A guiding principle in the system's implementation is that it contains
no proprietary interfaces; instead the project will adopt and
implement whatever standards are successful. For instance, for
communication with other PIM systems, Tempus Fugit implements
the iCalendar standard[
We have incorporated into the infrastructure of Tempus Fugit the following controls to safeguard the rights of its users with respect to the principles defined in the e-Social Contract. Through these controls, every user can create an individualized permissionsurface (Figure 3).
1. 2.
To protect their rights, users can specify access privileges to any personal data. The default access right is “private”. The user can set the access level at the granularity of his/her choosing – from protecting an individual data object to protecting all data. The user can grant or deny access to individuals, groups, or to agents of the Tempus Fugit system. These privileges not only determine whether or not other users can access the data but also whether the active agent components can make use of it.
Autonomous agents are disabled until activated by the user. Informing-collaborative agents are enabled.
behavior of enabled autonomous features.
invasive functionality (e.g., location tracking).
not known by the system, known by the system but marked private by the user, or not known to the system because its collection is disabled by the user. This masks the user’s intention to hide their information.
relationship data are just as important as individual user data. The default access right for relationships is private. The user can set the access level at the granularity of his/her choosing – from protecting an individual relationship to all relationship data. The user can grant or deny access to individuals, groups, or to agents of the Tempus Fugit system.
Tempus Fugit embodies many agent types through out the continuum of agent behaviors. In Figure 4 we show a two dimensional space of collaborative agents (the A-I plane of Figure 2). Even for agents designed only to support a single user, the user accepts a larger risk when relying on an agent that exercises greater autonomy or creates greater impact. As information about an individual becomes available to other clients of the system, the actions of the agent begin to affect multiple individuals and multiple permissions may be required to constrain agent behavior. Examples of agents affecting multiple clients are extruded out of the A-I plane in Figure 4. Below we describe some of these Tempus Fugit features with respect to these different agent behaviors. The screenshots below (Figures 5-7) illustrate the Tempus Fugit interface we call “Mozongo”. 4.3.1 Form Data Entry Form entry is a mechanism used to gather information from a user. Form filling can be a tedious task. An agent can assist in this task with varying degrees of autonomy. An agent-less form would not assist the user at all. An informative agent could provide look-ahead features and order likely options. Tempus Fugit can examine the repository of knowledge it maintains about its users and predict the values most likely to be entered (e.g., likely destinations for travel events, usual participants for certain types of events). These values can then be presented to the user as possible choices in order of likelihood.
Tempus Fugit moves along the collaborative plane towards autonomous form entry by pre-selecting likely values. Pre-filling and pre-selecting data in a form can readily affect a user to a greater extent than populating a selection box with likely choices. In accordance with the e-Social Contract design philosophy, only the user should have the authority to grant Tempus Fugit permission to pre-fill/pre-select a form. Without this permission the autonomous behavior is not allowed.
Another example, demonstrates a Tempus Fugit autonomous action. As part of its PIM functions, Tempus Fugit maintains an address book for each of its users. A user can manually create contacts and group them, via form entry. When a user first joins Tempus Fugit from within an organization, the system discovers, if possible, the user’s position within the organization and pre-fills the address book with department members, the user’s manager, and secretary. This requires no request or validation by the user. In this case the autonomous action is informative. If the autonomous agent is ineffective in putting useful information in the users address book, the user can simply remove the new entries or groups. Pre-loading the user’s address book lies close to the A axis in the A-I plane (Figure 4). 4.3.2 People Matching One of the more intriguing aspects of Tempus Fugit is its social automation capability. Tempus Fugit enables the discovery of potential social connections between people. For instance, a new person joining an organization such as a company or a university may be completely unaware of the traits and characteristics of the other individuals. Tempus Fugit uses the intimate knowledge it maintains about its users to identify compatible (or incompatible) people.
A first time user of Tempus Fugit is prompted to enter personal traits and characteristics (e.g., languages spoken, hometown, marital status, interests, expertise, skills) that the user is willing to share with others. Users are never coerced to reveal this information. A user can query Tempus Fugit to find others with a particular trait or interest (e.g., spoken language) or an expertise in a subject. This is agent-less technology. With the use of a collaborative agent, Tempus Fugit can pre-determine the best matches for a user and inform that user (Figure 5). The user can then follow up on Tempus Fugit’s “recommendation”, by clicking on the link to learn more about the matched user(s). Without the use of an autonomous agent Tempus Fugit could, having identified a possible match, offer to request a meeting between the users. Since this action requires validation by the user, the user’s rights are protected. The matched colleague is affected by this possible action. The colleague permitted the possible action by choosing to share personal information with others in the system. Having made this decision, the colleague is free to decline any meeting. Notice that there are various levels of impact depending upon the level of autonomy. By informing user ‘Sally’ that user ‘Bob’ is a good match for her, Bob is mildly affected since Sally is now aware of Bob. Tempus Fugit would have a more significant affect if it were to automatically set up a meeting between Sally and Bob. In accordance with our design philosophy, before Tempus Fugit could automatically send out a meeting invitation, both Sally and Bob would have to permit or enable the autonomous matching function. 4.3.3 Location Tracking Tempus Fugit supports tracking the location of its users. In general tracking the location of individuals is inherently invasive. However, Tempus Fugit adheres to the e-Social Contract design philosophy, and only tracks those users that have given their permission to be tracked. In addition, it provides two other mechanisms to allow a tracked user to disable tracking on an as desired basis. By not keeping a history of potentially sensitive location information, such as previous locations visited, paths taken, and proximity to other users, Tempus Fugit further safeguards its users as specified in the e-Social Contract tenet A.2-4.
An example of applying the e-Social Contract as a guiding philosophy in making design choices is the approach taken in the tracking of laptops participating in a wireless Ethernet. Two techniques were available. The first was to poll the wireless access points (WAPS) (i.e., “base stations”) that serve as the “connect points” for the laptops to the wired network. This technique gives the list of laptops that were “connected” to the access point. The second technique was to have the laptops run software that periodically beaconed the name of the access point to which they were connected. Both were equivalent in giving an estimated position of the laptop as somewhere in the proximity of the access point. Applying the tenet B.1 of the e-Social Contract that states that the collection of information about a user is under their control, Tempus Fugit uses only the second technique. This gives users complete control to turn off tracking and be confident that their position is not being recorded in any way. Tempus Fugit uses the tracked location information to predict the attendance and arrival times of individuals (Figure 6). Tempus Fugit also applies location information in support of its social automation. It is included in status displays, so a client can be aware of the location of people important to him/her. It is also used to automate serendipity, the fortuitous discovery of something not sought. In interpersonal relationships, this manifests itself when one discovers the presence of a friend/colleague without prior arrangement of a meeting. For instance, when two users are within proximity of each other, they can be informed of the presence of the other. Another example, involves static locations, rather than tracked dynamic locations. Tempus Fugit can examine the schedules (including meeting locations) of its users and discover the potentia l for future encounters, and inform each user accordingly. In keeping with the e-Social Contract, both users would have to permit or enable this informed “connection”. Before Tempus Fugit could move along the collaborative plane towards autonomous connections, both users would have to give additional permissions to enable autonomous connections. 4.3.4 Information Gathering Tempus Fugit provides background information to support a user's scheduled activities. For instance, when a user creates an event and invites participants, Tempus Fugit will automatically initiate a search to discover background information (if it exists) on the named participants. This information can include their position in the organizational hierarchy, inventions they patented, and publications they authored. In accordance with the e-Social Contract design philosophy, only public information is obtained, all private information is off-limits. Another example is when a user creates an event identifying a public company. Again, Tempus Fugit will initiate a search to discover background information on the named company. This information includes stock price, financial data, and company news. Figure 7 displays information about 2 companies identified in the creation of an event. Information gathering, though initiated autonomous, is strictly informative. Its impact on the user is dependent upon the value of the material gathered.
A storm is coming and we must prepare ourselves for what may
unfold. It is ironic that the liberties and freedom members of modern
societies take for granted could be tested by technologies born
under that freedom. Will a society that nurtures the freedom to
invent anything also create the tools of its own destruction[
We propose an e-Social Contract as a prototype philosophy for consideration. The contract is not, however, a hypothetical suggestion. It is the foundation of the design philosophy behind a working system, Tempus Fugit. This system combines features of autonomous agents with physical tracking of its users to create a kind of “super” personal information management system. This PIM system knows “everything” about a user’s activities, plans, goals, relationships, and current location. Tempus Fugit is the first system we are aware of that puts such an ominous combination of features into one powerful package. It uses those features to assist individuals and to help groups of users work together. A wealth of detailed information gives Tempus Fugit great power to fulfill its mission. It also makes it a potentially dangerous device if used to control people. Tempus Fugit is an ongoing experiment to test the idea that these dangers can be avoided with an appropriate design philosophy. Future systems, as is always the case, will be yet more powerful. We believe that some design philosophy must emerge by consensus, which will protect the rights of individuals, in the same spirit as our e-Social Contract.
We would like to acknowledge the people who contributed to Tempus Fugit: K. Cao, S. Edlund, M. Gerlach, J. Jackson, R. Kraft, S. Lassahn, P. Lazarus, D. McKenzie, M. Muhammad, J., Myllymaki, and the many subscribers to the system.
[1] AirLink Communications, www.airlinnk.com [2] Di Stefano, A., Santoro, C., “Agent Support for Personal
Mobility”, IEEE Computing, March/April 2000, p74-79. [3] Economist, October 9, 1999. [4] Ford D., Edlund S., Ruvolo J., Myllymaki, J., Kraft R., Lazarus P., Cao K., Lassahn S., McKenzie D., Jackson J., Muhammad M., Gerlach M., Kaufman J. “Tempus Fugit: A system for user automation and awareness”, SIGCHI’01, (submitted for publication).