Ambient interface design for a Mobile Browsing Experience (AMBE) Rosaleen Hegarty†, Tom Lunney†, Kevin Curran†, Maurice Mulvenna* †School of Computing and Intelligent Systems *School of Computing and Mathematics Faculty of Computing and Engineering Faculty of Computing and Engineering University of Ulster, Magee College University of Ulster, Jordanstown Campus Derry/Londonderry, BT48 7JL Antrim, BT 37 OQB (+44) 28 71375174 (+44) 28 71375388 {hegarty-r, tf.lunney, kj.curran}@ulster.ac.uk md.mulvenna@ulster.ac.uk ABSTRACT of ‘calm’ [20] remains the consummation in these Our preliminary research focuses on the development of phenomena and a reflection of the technology we seek. an Ambient Mobile Browsing Experience (AMBE) The principle of AIS, is captured in the following instance system. AMBE is a communication and synchronisation and subsequent descriptions; “When you look at a street framework that will provide integrated connectivity sign, for example, you absorb its information without across heterogeneous geographically distributed devices. consciously performing the act of reading. Computer The intention is to provide persistent location-independent scientist Herbert A. Simon calls this phenomenon and appliance-sensitive viewing for the user, thus “compiling”; philosopher Michael Polanyi calls it “the enabling Internet mobility. Human technology interface tacit dimension”; psychologist J.J. Gibson calls it “visual communication will be abstracted to a representation that invariants”; philosophers Hans Georg Gadamer and facilitates optimisation and customisation across a number Martin Heidegger call it “the horizon” and the “ready- of different displays. This will help to ensure seamless to-hand”, John Seely Brown at PARC calls it the continuity between components providing usability and “periphery”. All say, in essence, that only when things maximum user convenience. An emphasis of our work is disappear in this way are we freed to use them without the application of a human-centered design ethos. thinking and so to focus beyond them on new goals” [20]. Keywords Nature in her pureness has provided simple informative Ambient interfaces, peripheral displays, ubiquitous cues that act on the subconscious delivering subliminally computing, mobile digital communication, sensor technology, context awareness. INTRODUCTION Denning and Metcalfe affirm, “to become attuned to more information is to attend to it less” [5]. This cause is central to Ambient Information System (AIS) design within Ambient Intelligence (AmI). The ethos of which Figure 1. Ambient information existing in nature. lies in the classification of center (the user) and periphery and with critical effect. Take the occasion of a raindrop, a (computational devices) for intelligent knowledge signal and a suggestion requiring little cognitive effort, management, with the objective of controlling yet retrieving from memory past experiences stimulated information overload and unnecessary complexity. by similar impetus to deliver appropriate actions. Nature’s Carbonell reflects on ambient interface interactions as ambient sound and light further inform our everyday state having to be reconfigured for throughput to output of existence. With this in mind it is perhaps to nature we terminals of varying media and screen dimensions [4]. must return in order to re-balance the information Implementation of these constraints gives rise to congestion that exists combining nature’s intuition with ‘interface plasticity’ and ‘adaptive multimodality’ [3]. twenty-first century engineering. ‘DataFountain’ reflects However maintaining simplicity whilst asserting notions this concept. ‘DataFountain’ is an Internet enabled display of currency comparisons for the Yen, Euro and Dollar, and uses pressurised water levels from three points to Copyright © 2008 for the individual papers by the papers' provide particular information with calming aesthetics authors. Copying permitted for private and academic [14]. Another example is ‘PlantDisplay’ appealing to purposes. Re-publication of material from this volume human emotion through organic changes in the plant’s requires permission by the copyright owners. appearance [13]. Human-centered design is attempting to capture this optimisation between distributed devices such as Personal vision in computing technology by augmenting Digital Assistants (PDAs), Personal Computers (PCs), flat consciousness and accommodating human-machine co- screens and smart mobile phones by using appropriate operation. The emphasis is on efficient user-affable and sensor technology. Where appropriate the incorporation immersive interfaces with distributed virtual services that of abstract symbolism via an artefact may filter surround; empowering the user with control [12]. information to ambient displays in public space for This anthropomorphic model of interaction refocuses the personal user discernment. Customisation and user to the foreground and creates synergies between the synchronisation of multimedia input and output between user and the environment [15]. Operations are intended to the distributed devices are to ensure continuity of the user be omnipresent, non-intrusive and transparent. It is in the experience. application of ascribing human characteristics such as In pursuing the objectives outlined, several research sensory perception, and cognitive behavioural interactions questions will need to be addressed. For example: What to physical or hidden phenomena that the essence of sort of profiler will be required to track and perhaps ambient intelligence is encapsulated to provide enhanced interpret the user behaviour/movement, both within the user proficiency [12]. context of the virtual environment and the physical one Ambient information displays are about the analysis and (possibly intelligent algorithms coupled with infrared, or representation of information in public, semi-public and Radio Frequency Identification-RFID tags) to provide private space, incorporating subtle techniques and persistence and session continuity? communication methods through peripheral perception In addition how will the profiler store the user’s session, [19]. will it cache and co-ordinate seamlessly to a new device Ambient displays rely on our multimodal senses, from decentralised clients (subsequently passing from web operating subliminally and below the threshold of script to client – client-side facilities), or from a central consciousness requiring only subconscious recognition repository (heavy overload for concurrent sessions – [2]. The classic example is Jeremijenko’s Display server-side facilities)? Installation entitled ‘Live Wire’, which attracts either How will AMBE tailor context sensitive intelligent user aural or visual attention as the incitement requires [11]. interfaces with automatic profiling to optimise the mobile More recent ambient displays include ‘The Kandinsky user experience? system’, which generates aesthetic information collages What symbolic abstraction will be incorporated to release converting textual input to image output [8]. ‘IMPACT’ sensitive information in public space, for example what monitors daily physical activity and provides feedback indicators will alert the importance of an incoming through detailed and abstracted displays [9]. ‘Ambient electronic message in an operating theatre or surgical Orb’ presents ambient information through wireless room where disturbance of external influence is configurations to track personal portfolio interests such as unwelcome, yet may be critical information required by market shares [1]. ‘Hello.wall’ uses a large ambient the main operator within the given environment. Will this display coupled with a hand-held device exploiting our utility be incorporated through artefacts of two or three ability to perceive information via codes [19]. Real time dimensional composition? Or perhaps through colour data panoramas map to visual components such as ocean coding or contour configurations adapted from existing waves and sun strength reflecting stock market activity. works such as the Kandinsky paintings illustrated in Consistently the purpose is to refine knowledge to a Figure 2. Would this approach eliminate the beep and symbolic representation requiring little cognitive effort buzz of phones and pagers providing social and [10]. acceptable communication etiquette, and in a minimally intrusive manner? MOTIVATION The motivation for this research is to provide sensor- activated communication. This will enable contextualised content viewing to be available at the current terminal or display screen demonstrating visual peripheral information updates for the user. The primary objectives of this study are in the development of a framework intrinsically linked to the porting of browsing session information over the network. Figure 2. (i) Marcadores (ii) Squares with Concentric This will be demonstrated through the implementation of Rings by Kandinsky. a scalable solution distributing current context An ethical issue that arises is that the abstracted notation information to appropriate selected device displays. The for information is reliable and consistent for the initiated requirement is to ensure user interface continuity and users specified; otherwise it could lose all purposeful functionality. Privacy related data may need to be tagged as ‘sensitive’ and filtered away from any public form of orchestrate this profile to heterogeneous devices within display to enable security and dependability within the dynamic environments. In addition the server will also be design. responsible for carrying out routine authentication and authorisation and provide session state and mobility handling within the system. CHALLENGES Amongst the challenges for this system, there exists the requirement to work in real-time and to cope with varying CONTEXT AWARENESS levels of ambiguity, such as changes in user predilection, The design process of AMBE will rely on the concept of user idiosyncratic actions and weak sensor signals. context awareness. Adaptability to a user’s situation is Adaptability to new heterogeneous devices (Figure 3) and enabled by context awareness, “Context is any amended environments will result in readjustment to meet information that can be used to characterise a situation of user specification and compensate for device failure an entity” [6]. This entity can be a person, place or supporting integration and interoperability. Whilst computational device, alternatively has real existence and dynamically adapting to user requirements through can change dynamically. Schmidt et al., say “context can reconfiguration, ‘trust,’ ‘security’ and ‘safety’ standards give meaning to something else [17]. A challenge must also be adhered to, and integrated into the system highlighted in the development of deployable context- design. aware services is the aptitude to lever ambiguous contexts as both sensed and interpreted context is often unclear. In addition deployment of contextual information to mobile applications brings to light the trust and privacy features - critical with automated processing of sensitive information. The sources available to capture contextual information in this research include sensors in mobile devices, RFID tags, network servers and application servers among others. Contextual awareness between disparate entities seeks to facilitate interoperability between application platforms with some context immeasurable, but derived through inference [18]. Activity Theory Modeling Activity Theory Modeling is taken from the psychology and social science disciplines, working in the areas of consciousness and cognitive acts within phenomenology [16]. Activity theory applied in AMBE finds application in the areas of context awareness and situational Figure 3. High-level schematic of AMBE. descriptions because of its adaptability to socio-technical perspectives and centers on three key concepts namely; The core of the application architecture is to provide action, situation and presence to give context [16]. The natural interactions and abstraction of the underlying ability to capture the context of the user in state, technical communication infrastructure; hiding application and service requires interpretation of the ‘6 complexity, whilst enhancing experience and confidence. W’s; ‘Who?,’ ‘What?,’ ‘Why?,’ ‘Where?,’ ‘When?’ and Successful ubiquity however, requires transparency ‘hoW?’ and is central to the design and profile of the user. integrated into the ecology of ones environment Context is argued to be a feature of interaction in any facilitated through peripheral interfacing. human-computer symbiosis [7]. It is based on the premise that intelligence is action orientated and context can be The key components of AMBE work to ensure continuity used to bring order and clarity to unclear situations in of service between multiple parts and include a sensor order to deliver appropriate actions. Therefore context is network, web server, session server, and user session (to seen as a tool for action selection. Within AMBE enabling store user history, cookies, current web page state and device exchange whilst sustaining the capabilities and bookmarks amongst other user facilities) to different resources of the current session is part enabled by context displays. The server side can act as a coordinator to awareness. Location information is another form of manage the data, and facilitate screen resizing before context aware information. ‘Activity Theory Modeling’ exporting to a newly activated device. The client side may be incorporated further into the design process of component will have the necessary functionality to AMBE as a means to capture information concerning the manage session synchronisation as a feature. The server user. This modeling may encapsulate the user’s intentions must also maintain a user’s personal profile and towards a display terminal, the capabilities of their display equipment and their surrounding interfaces. that Contain Information Proceedings (UIST2001, Additionally this information could form a ‘migration New York), ACM Press, 141-150. theory’ between the user, their session and their display. 9. Forlizzi, J., Li, I., Dey, A. Ambient Interfaces that Motivate Changes in Human Behaviour, 2007. CONCLUSION Available at:http://ftp.informatik.rwth- Ambient Intelligence is a dynamic vision, one in which aachen.de/Publication/CEUR-WS/Vol- technology serves information filtering. AMBE seeks to 254/paper02.pdf. provide an information utility through seamless 10. Information Aesthetics, 2008. Available at: coalescence and switching of display devices activated by http://infosthetics.com/archives/ambient/. sensors. This is achieved by caching the associated 11. Interaction Design Thesis, 2007. Available at: objects and relaying them to another possibly central http://interactionthesis.wordpress.com/2007/02/14/dan repository, to facilitate viewing to commence on a gling-string/. different platform. Context-aware and context-dependent information will be captured to provide the dynamics in 12. Khedo, K. Short Notes on Ambient Intelligence, 2007. supporting this feature. The question that arises is how we Available at: deliver critical information via ambient displays to highly http://khedo.wordpress.com/2007/05/18/short-notes- intensive environments of people centered care through on-ambient-intelligence. human-centered design. In addition, by incorporating 13. Kuribayashi, S., Wakita. A. PlantDisplay: turning interface plasticity and multimodality how should we test houseplants into ambient display, in Proceedings of and with what recognised metrics, do we quantify, qualify the 2006 ACM SIGCHI international conference on and assess the standard since these systems are designed Advances in computer entertainment technology, not to occupy our full attention rather to augment it. In 2006. Available at: addition how interoperable are these systems and what http://portal.acm.org/citation.cfm?id=1178823.117887 future dependability and security features can be applied. 14. Mann, S., Georgas, M., Janzen, R. Water Jets as Pixels: Water Fountains as both Sensors and Displays, REFERENCES in Proceedings of the eighth IEEE International 1. Ambient Orb (2008). Available at: Symposium on Multimedia (ISM’06). Available at: http://www.ambientdevices.com/cat/orb/orborder.html http://www.eyetap.org/papers/docs/ieee_ism_funtain_ simon_game274600766.pdf. 2. Baars, B. J., Mc Govern, K. Cognitive views of consciousness, What are the facts? How can we 15. Remagnino, P., Foresti, G. L. Ambient Intelligence: A explain them?, in The Science of Consciousness New Multidisciplinary Paradigm, in IEEE (eds.)Veldmans, M., (1996), Routledge Press, 69. Transactions on Systems, Man and Cybernetics – Part A: Systems and Humans, Vol. 35, No 1, 2005. 3. Calvary, G., Thevenin, D., Coutaz, J. A Reference Framework for the Development of Plastic User 16. Riva, G., Vatalaro, F., Davide, F., Alcaniz, M. Interfaces, 2003. Available at: Ambient Intelligence, the Evolution of Technology http://iihm.imag.fr/publs/2003/MuiBook03.pdf. Communication and Cognition - Towards the Future of Human-Computer Interaction IOS Press, 2005, 21- 4. Carbonell, N. Ambient Multimodality: towards 23. Advancing Computer Accessibility and Assisted Living, International Journal on Universal Access 17. Schmidt, A., Beigl, M., Gellersen, H.W. There is more in the Information Society (UAIS, 2006), 18-26. to Context than Location Computers and Graphics Journal, Elsevier 23 (6), 1999, 893-902. 5. Denning, P. J., Metcalfe, R. M. Beyond Calculation: The Next Fifty Years of Computing, (Springer 1998), 18. van Sinderen, M. J., van Halteren, A.T., Wegdam, M., 83-84. Meeuwissen, H. B., Eertink, E. H. Supporting Context-Aware Mobile Applications: in IEEE 6. Dey, A.K., Abowd, G. D. Towards a Better Communications Magazine, Sept 2006, 96-104. Understanding of Context and Context Awareness in Proceedings-Conference Human Factors In 19. Vogel, D., Balakrishnan, R. Interactive Public Computing Systems , The Hague, The Netherlands, Ambient Displays: Transitioning from Implicit to April, 2000. Explicit, Public to Personal, Interaction with Multiple Users, 2004. Available at: 7. Dourish, P. What we talk about when we talk about http://www.dgp.toronto.edu/~ravin/papers/uist2004_a context, Personal and Ubiquitous Computing 8(1), mbient.pdf. 2004, 19-33. 20. Weiser, M., The Computer for the Twenty-First 8. Fogarty, J., Forlizzi, J., Hudson, S.E. Aesthetic Century, in Scientific American September 1991, 94- Information Collages: Generating Decorative Displays 104.