Ubiquitous thinking means designing human-computer interaction in a fundamentally new way. The perceived distance between ubiquitous systems and their users decreases while the heterogeneity of modalities increases compared to classical human-computer interaction. The initiation of work phases becomes less formalized. Instead of explicitly declaring the start of an interaction by activating a computing device, the interaction starts gradually and sometimes implicitly based on an estimation of the user's needs. For bridging the gap of initiation, in this article we present the ubiquitous interaction concept “Ubiquitous Alignment”. It comprises of the three steps recognition, sparking interest and start of collaboration. The Ubiquitous Alignment concept is based on a comparison between traditional human-computer and human-human interaction. Finally, two examples show the applicability of the Ubiquitous Alignment concept.
LEAVE BLANK THE LAST 2.5 cm (1”) OF THE LEFT COLUMN ON THE FIRST PAGE FOR THE
COPYRIGHT NOTICE.
to be aware of them in order to interact [
After discussing related work, we will first analyze how interaction between humans and other humans (humanhuman interaction) as well as between humans and computers (human-computer interaction) usually starts, then highlight differences between the two situations. Subsequently, we will describe our concept of Ubiquitous Alignment, explain where it differs from human-computer interaction and identify parallels with human-human interaction. Finally, we will present two example scenarios in which the concept can be applied.
RELATED WORK
There has been a large amount of work to provide
computing devices with additional sensors to perceive an
arbitrary range of signals from the outside world. To name
only a few, sensor systems to detect human means of
expression such as pointing at something with the hand [
The behavior of user interfaces that sometimes act
proactively and sometimes leave the control to the user is
called mixed initiative. There has been much research on
this topic over the course of the past few decades. It focuses
on ways to achieve and employ mixed initiative [
TRADITIONAL INTERACTION STYLES This section describes two examples of starting an interaction between humans and other humans and between humans and today’s computers, respectively. Both examples are chosen in a way that the participants of the interaction do not have any prior knowledge about each other or are not yet collaborating. Thus, the situations are analogous to the interaction between a user and ubiquitous devices as described further below.
Human-Human Interaction As an example of human-human interaction, we have chosen a customer in a self-service store and a sales assistant. This scenario was selected because it closely resembles a situation where ubiquitous technology might come into play. Basically, the customer could manage well without any additional help. Generally, for a comfortable shopping experience, the sales assistant should not behave in a pushy way by insisting on helping the customer against his wish. The assistant may however indicate that she is available and ready to help if help is required, and the customer may decide on his own to start a more thorough interaction.
Initially, the customer is examining the items in a shelf, reading the information given on the labels and the price tags. The sales assistant is waiting nearby. In order to not appear obtrusive, she should not wait right next to the customer or in front of the shelf, as this might make the customer feel controlled. Still, it is important not to express disinterest or lack of attention. This can be achieved by displaying an initial sign of responsiveness, such as greeting the customer when he enters the store, or by explicitly offering help when the customer has been browsing the products for a while.
At the least, when the customer has picked up some items and placed them in his shopping cart, the sales assistant may carefully indicate that she is willing to start a conversation. This can be expressed by a single casual remark about one of the products or by pointing out an alternative. At this point, it is important to note that the information given is not among that which the customer has likely already seen. Instead, he may be pointed to a feature that is not evident from the labels or to an item that is not currently located on the same shelf. In this way, the customer will perceive the assistant as helpful rather than merely reiterating known facts.
If the customer desires to receive more information afterward, he will ask the assistant. The assistant can inform the customer about what information she is able to provide, while the customer gets a feeling of how reliable the information received from that assistant is.
Human-Computer Interaction Due to the lack of proactivity in most of today’s software, the gradual start of an interaction as seen in the example of human-human interaction cannot be customarily found in human-computer interaction. Assuming that the computer is already switched on and the user has logged into her account, she starts for the first time the new application she would like to use.
The application displays a default set of options and commands. Guides to the most important features can be provided. An example location would be a welcome screen. Nevertheless, the user has to start exploring the interface right away. After taking a few steps in the user interface, the application tries to estimate what the user is trying to do and displays hints accordingly. The application can only evaluate the user input and does not possess any additional sensors. Hence, it cannot take into consideration any contextual information about the user and her environment. The estimation of the user’s intentions is accordingly imprecise; therefore, the displayed hints occasionally fail to be of any help, which in turn makes the user dissatisfied with the application.
Once the user has gathered some experience with the application, she will actively customize the user interface and create templates and macros for repeating tasks. Due to bad experience with the automatic input analysis, she might eventually choose to completely disable the automatic adaptation of application behavior. Even though this means some additional effort for the user in that some settings have to be done manually, she values the absence of distracting hints that do not provide any helpful information higher than saving some time by allowing the software to automatically adapt itself.
Comparison Despite being basically equivalent scenarios of starting an interaction with a previously unknown partner, these two descriptions of human-human and human-computer interaction sport substantial differences. First of all, in human-computer interaction the user has to know and launch the application she wants to use. The application is not just there and ready by default, as it is the case with the sales assistant. By launching that application, the computer user also explicitly declares the start of the interaction, as opposed to the gradual process found in the interaction between the customer and the sales assistant.
As mentioned above, the lack of variety of input channels
available to the system results in a lack of knowledge about
the overall behavior and context of the user. Thus, any
estimation about the current intentions of the user can at
most be a rough guess. Accordingly, helpful clues can only
be given based on the experience with average users or by
trying to find repeating patterns in the behavior of the
current user. The same applies to input interfaces such as
menus: Even though some software manufacturers have
attempted to automatically restructure menus, a user study
suggests that any such change is likely to confuse the user
rather than support her [
UBIQUITOUS ALIGNMENT
In order to make human-computer interaction more like
human-human interaction, one can take advantage of the
special capabilities of ubiquitous computing technology.
The additional data gathered by the sensors in a ubiquitous
computing environment allows for a more natural initiation
of collaboration between users and systems [
The Ubiquitous Alignment concept assumes that a user is going to perform a particular task in an environment equipped with ubiquitous computing devices. The user does not yet have sufficient knowledge about those devices to explicitly trigger any operations. He may or may not be aware that his environment is equipped with ubiquitous computing systems at all. In order to achieve collaboration, the three steps recognition, sparking interest and start of Fig. 1. User and ubiquitous system gradually intensify their interaction in three steps of the Ubiquitous Alignment concept: At first, they barely know of each other, then they start to interact and intensify that interaction further on. collaboration are performed (cf. Fig. 1).
Step 1: Recognition The user recognizes the system in a pleasant rather than a pushy way. A pleasant ubiquitous system remains in the background until the user wishes to start an interaction. In this phase, the system is still largely ignorant of what the user intends to do. This matches the behavior of the assistant from the human-human interaction example, who remains passive. Any other behavior might annoy or scare away the other person or the user, respectively. The ubiquitous system visibly exhibits a certain level of proactivity only when it is absolutely certain that an intervention is desired by the user. Otherwise it remains largely invisible, except for some unobtrusive hints that it is there and ready.
Any other operations the ubiquitous system performs go unnoticed by the user, who gets the impression that he is just using everyday objects. Automatic locks that secure lids of boxes unless the user actually attempts to open them, the adaptation of fridge power to cool down newly inserted warm items or the temporary dimming of lights while the user is not in the room do not require any active input. That is also why no new interface concepts need to be considered at this point. The ubiquitous devices do not have any new input controls. They can be used just like their non-ubiquitous counterparts. Step 2: Sparking Interest User and system begin to communicate with each other. As the user finds out what the system is or is not capable of, the system output at this point must particularly strive for a high reliability. This concerns both the information provided and the estimations made. This step corresponds with the customer becoming acquainted with the sales assistant and vice-versa.
In order to not appear overzealous at communicating with the user where no communication is desired, a good strategy is to continue giving small hints of the presence and features of the system, just as the sales assistant will try to be supportive without flooding the customer with information. In particular, those hints should spark the interest of the user/customer and motivate him or her to find out what kind of support can be obtained. At the same time, the ubiquitous devices may be able to catch some clues as to how the user behaves or reacts and what kind of output inspires him to further interact with the system, just as the sales assistant will adapt his behavior to suit the customer’s preferences to a certain degree.
Step 3: Begin of Collaboration After the computer system has been recognized by the user and he has indicated that he is willing to collaborate with the system, the system can become more active. As the user has become interested in the system, he is likely to try and explore further capabilities of the ubiquitous devices. This behavior can be encouraged by facilitating the exploration process. Amongst others, options related to the current operation that have not yet been employed by the user can be recommended to him. Likewise, any means of discovering and learning about unknown system features must be easy to find. A human sales assistant will too, in a comparable situation, express what information he is able to provide to the customer.
While interacting with the user, a system that follows the Ubiquitous Alignment approach has gathered and is still gathering an increasing amount of information about the user and his behavior. This allows for better estimates of the current intentions of the user and thus provides the system with the means to support the user in an optimized way.
How Ubiquitous Alignment helps improve HCI Ubiquitous Alignment reflects the gradual process used to establish contact between two humans with the goal of collaboration. These parallels hold true both for situations where the actors do not have any prior knowledge about each other as well as for cases where they do. In the former case, the steps explained serve for the initial contact between two strangers, just as for the initial contact between a future user and a network of ubiquitous devices. In the latter case, the participating persons already know each other, so the objective is collaboration on a given task. The actors do not yet know whether the collaboration will actually turn out to be beneficial, which is why they use the same approach of gradually initiating their interaction. Likewise, a user might already know some parts of a ubiquitous system, but he is not sure yet whether the system is helpful with a new kind of task. At the same time, the system should not behave in a paternalistic way and insist on collaboration in this particular new task just because the user makes frequent use of the system on other occasions.
To sum up, the main advantages of ubiquitous computing systems over traditional computing systems at approximating human-human interaction are their greater variety of input channels and their integration into everyday objects. The additional input channels in the form of a variety of sensors allow for a more accurate and complete perception and evaluation of the user, his behavior and his context. By integrating system parts into appliances previously known to the user, the handling of the ubiquitous system does not have to be learned right from the start on; instead, some features can be used by manipulating appliances the usual way, so the prospective user can gradually extend his knowledge to encompass the additional system features that require any special input. POSSIBLE APPLICATIONS OF THE UBIQUITOUS ALIGNMENT CONCEPT To underline that the Ubiquitous Alignment concept can indeed be used in ubiquitous computing scenarios, we describe two example scenarios in which our Ubiquitous Alignment concept is applied.
One example of a ubiquitous system that uses mobile
computing devices is the ActiveClass system described by
Griswold et al. [
In the second example, we consider a table that is aware of what objects are placed on the tabletop (cf. Fig. 2). This awareness can be achieved through a variety of means, such as load detection, image analysis or tracking of object locations (assuming that each object is tagged in some way), or a combination thereof. In addition, some means of tracking what the user is doing is available. The knowledge about object positions can be used to guide a user by indicating where on the table to find a particular item. This can be used for workbenches or interactive cookbooks, to name only two examples. In the first step of the Ubiquitous Alignment approach, the user may be using the table just as a table, placing objects on top of it. The system performs its minimum default function, inserting positional indicators such as “on the left” or “next to ...” in the instructions for the user. Only when the user keeps searching for something for a longer time, does the system clarify its output, providing more information in an additional message. If the user responds by locating objects faster based on those hints, step two of the Ubiquitous Alignment concept has the system highlight any references to objects in the displayed instructions (or, in the case of voice output, make clear what is highlighted in text in some other way), pointing the user to the possibility of finding out more about the respective items. Eventually, in the third step the system may display additional information right away as the user requires it, and offer some options to modify how much and what kinds of information the user wants to be displayed about objects referred to in the work instructions. These examples show how the concept of Ubiquitous Alignment can be applied to scenarios where a user starts getting to know a ubiquitous system or one of its features. In all described scenarios and examples, the user had had a certain resistance to using the system, or at least he or she was not assumed to spend a lot of initial effort to learn how to use the system. This is where Ubiquitous Alignment is particularly beneficial. Users who take the time to read a manual first do not require the same degree of gradual initiation of interaction. Nonetheless, striving for a display of reliability towards that kind of users and not annoying them with frequent messages or other possibly undesired output retain their importance.
CONCLUSION AND FUTURE WORK In this work, we have examined some exemplary situations of human-human and human-computer interaction. In an effort to make human-computer interaction more alike to human-human interaction, we have described the Ubiquitous Alignment concept. It defines how collaboration between a human user and a computer system can be initiated in a way that closely resembles the interaction between humans, taking advantage of the possibilities found in ubiquitous computing devices. As seen in the comparisons of the Ubiquitous Alignment approach with the previous examples of human-human and human-computer interaction, our approach has a strong resemblance to the former. The main reasons for the differences were found to be the additional sensor input and, similarly, the additional input modalities which can totally match the normal manipulation of everyday objects, as opposed to handling specialized devices such as mice or keyboards to provide input to traditional computers. As this work presents a concept of how a ubiquitous system should behave, a future goal is the implementation of this concept. Thereby, we hope to show how the Ubiquitous Alignment concept works in practice and how it can be implemented in detail. Also, we expect this to be a starting point for defining processes for the development of ubiquitous software components and for gathering a better understanding of the user’s behavior. A model system will not need to incorporate all of the described attributes. With the incorporation of additional sensors, the system could gradually come closer to the ideal form of the Ubiquitous Alignment concept. ACKNOWLEDGMENTS This research was funded through the IP-KOM-ÖV project (German Ministry of Economy and Technology (BMWi) grant number 19P10003N). Also, we would like to thank our students Thomas Bach, Steffen Bold and David Kruzic.