Ambient interface design for a Mobile Browsing Experience (AMBE)

Abstract

Our preliminary research focuses on the development of an Ambient Mobile Browsing Experience (AMBE) system. AMBE is a communication and synchronisation framework that will provide integrated connectivity across heterogeneous geographically distributed devices. The intention is to provide persistent location-independent and appliance-sensitive viewing for the user, thus enabling Internet mobility. Human technology interface communication will be abstracted to a representation that facilitates optimisation and customisation across a number of different displays. This will help to ensure seamless continuity between components providing usability and maximum user convenience. An emphasis of our work is the application of a human-centered design ethos.

Keywords

• eol>Ambient interfaces
• peripheral displays
• ubiquitous computing
• mobile digital communication
• sensor technology
• context awareness

-

INTRODUCTION Denning and Metcalfe affirm, “to become attuned to more information is to attend to it less” [ 5 ]. This cause is central to Ambient Information System (AIS) design within Ambient Intelligence (AmI). The ethos of which lies in the classification of center (the user) and periphery (computational devices) for intelligent knowledge management, with the objective of controlling information overload and unnecessary complexity. Carbonell reflects on ambient interface interactions as having to be reconfigured for throughput to output terminals of varying media and screen dimensions [ 4 ]. Implementation of these constraints gives rise to ‘interface plasticity’ and ‘adaptive multimodality’ [ 3 ]. However maintaining simplicity whilst asserting notions of ‘calm’ [ 20 ] remains the consummation in these phenomena and a reflection of the technology we seek. The principle of AIS, is captured in the following instance and subsequent descriptions; “When you look at a street sign, for example, you absorb its information without consciously performing the act of reading. Computer scientist Herbert A. Simon calls this phenomenon “compiling”; philosopher Michael Polanyi calls it “the tacit dimension”; psychologist J.J. Gibson calls it “visual invariants”; philosophers Hans Georg Gadamer and Martin Heidegger call it “the horizon” and the “readyto-hand”, John Seely Brown at PARC calls it the “periphery”. All say, in essence, that only when things disappear in this way are we freed to use them without thinking and so to focus beyond them on new goals” [ 20 ]. Nature in her pureness has provided simple informative cues that act on the subconscious delivering subliminally and with critical effect. Take the occasion of a raindrop, a signal and a suggestion requiring little cognitive effort, yet retrieving from memory past experiences stimulated by similar impetus to deliver appropriate actions. Nature’s ambient sound and light further inform our everyday state of existence. With this in mind it is perhaps to nature we must return in order to re-balance the information congestion that exists combining nature’s intuition with twenty-first century engineering. ‘DataFountain’ reflects this concept. ‘DataFountain’ is an Internet enabled display of currency comparisons for the Yen, Euro and Dollar, and uses pressurised water levels from three points to provide particular information with calming aesthetics [ 14 ]. Another example is ‘PlantDisplay’ appealing to human emotion through organic changes in the plant’s appearance [ 13 ].

Human-centered design is attempting to capture this vision in computing technology by augmenting consciousness and accommodating human-machine cooperation. The emphasis is on efficient user-affable and immersive interfaces with distributed virtual services that surround; empowering the user with control [ 12 ]. This anthropomorphic model of interaction refocuses the user to the foreground and creates synergies between the user and the environment [ 15 ]. Operations are intended to be omnipresent, non-intrusive and transparent. It is in the application of ascribing human characteristics such as sensory perception, and cognitive behavioural interactions to physical or hidden phenomena that the essence of ambient intelligence is encapsulated to provide enhanced user proficiency [ 12 ].

Ambient information displays are about the analysis and representation of information in public, semi-public and private space, incorporating subtle techniques and communication methods through peripheral perception [ 19 ].

Ambient displays rely on our multimodal senses, operating subliminally and below the threshold of consciousness requiring only subconscious recognition [ 2 ]. The classic example is Jeremijenko’s Display Installation entitled ‘Live Wire’, which attracts either aural or visual attention as the incitement requires [ 11 ]. More recent ambient displays include ‘The Kandinsky system’, which generates aesthetic information collages converting textual input to image output [ 8 ]. ‘IMPACT’ monitors daily physical activity and provides feedback through detailed and abstracted displays [ 9 ]. ‘Ambient Orb’ presents ambient information through wireless configurations to track personal portfolio interests such as market shares [ 1 ]. ‘Hello.wall’ uses a large ambient display coupled with a hand-held device exploiting our ability to perceive information via codes [ 19 ]. Real time data panoramas map to visual components such as ocean waves and sun strength reflecting stock market activity. Consistently the purpose is to refine knowledge to a symbolic representation requiring little cognitive effort [ 10 ].

MOTIVATION The motivation for this research is to provide sensoractivated communication. This will enable contextualised content viewing to be available at the current terminal or display screen demonstrating visual peripheral information updates for the user.

The primary objectives of this study are in the development of a framework intrinsically linked to the porting of browsing session information over the network. This will be demonstrated through the implementation of a scalable solution distributing current context information to appropriate selected device displays. The requirement is to ensure user interface continuity and optimisation between distributed devices such as Personal Digital Assistants (PDAs), Personal Computers (PCs), flat screens and smart mobile phones by using appropriate sensor technology. Where appropriate the incorporation of abstract symbolism via an artefact may filter information to ambient displays in public space for personal user discernment. Customisation and synchronisation of multimedia input and output between the distributed devices are to ensure continuity of the user experience.

In pursuing the objectives outlined, several research questions will need to be addressed. For example: What sort of profiler will be required to track and perhaps interpret the user behaviour/movement, both within the context of the virtual environment and the physical one (possibly intelligent algorithms coupled with infrared, or Radio Frequency Identification-RFID tags) to provide persistence and session continuity? In addition how will the profiler store the user’s session, will it cache and co-ordinate seamlessly to a new device from decentralised clients (subsequently passing from web script to client – client-side facilities), or from a central repository (heavy overload for concurrent sessions – server-side facilities)? How will AMBE tailor context sensitive intelligent user interfaces with automatic profiling to optimise the mobile user experience? What symbolic abstraction will be incorporated to release sensitive information in public space, for example what indicators will alert the importance of an incoming electronic message in an operating theatre or surgical room where disturbance of external influence is unwelcome, yet may be critical information required by the main operator within the given environment. Will this utility be incorporated through artefacts of two or three dimensional composition? Or perhaps through colour coding or contour configurations adapted from existing works such as the Kandinsky paintings illustrated in Figure 2. Would this approach eliminate the beep and buzz of phones and pagers providing social and acceptable communication etiquette, and in a minimally intrusive manner?

An ethical issue that arises is that the abstracted notation for information is reliable and consistent for the initiated users specified; otherwise it could lose all purposeful functionality. Privacy related data may need to be tagged as ‘sensitive’ and filtered away from any public form of display to enable security and dependability within the design.

CHALLENGES Amongst the challenges for this system, there exists the requirement to work in real-time and to cope with varying levels of ambiguity, such as changes in user predilection, user idiosyncratic actions and weak sensor signals. Adaptability to new heterogeneous devices (Figure 3) and amended environments will result in readjustment to meet user specification and compensate for device failure supporting integration and interoperability. Whilst dynamically adapting to user requirements through reconfiguration, ‘trust,’ ‘security’ and ‘safety’ standards must also be adhered to, and integrated into the system design.

The core of the application architecture is to provide natural interactions and abstraction of the underlying technical communication infrastructure; hiding complexity, whilst enhancing experience and confidence. Successful ubiquity however, requires transparency integrated into the ecology of ones environment facilitated through peripheral interfacing.

The key components of AMBE work to ensure continuity of service between multiple parts and include a sensor network, web server, session server, and user session (to store user history, cookies, current web page state and bookmarks amongst other user facilities) to different displays. The server side can act as a coordinator to manage the data, and facilitate screen resizing before exporting to a newly activated device. The client side component will have the necessary functionality to manage session synchronisation as a feature. The server must also maintain a user’s personal profile and orchestrate this profile to heterogeneous devices within dynamic environments. In addition the server will also be responsible for carrying out routine authentication and authorisation and provide session state and mobility handling within the system.

CONTEXT AWARENESS The design process of AMBE will rely on the concept of context awareness. Adaptability to a user’s situation is enabled by context awareness, “Context is any information that can be used to characterise a situation of an entity” [ 6 ]. This entity can be a person, place or computational device, alternatively has real existence and can change dynamically. Schmidt et al., say “context can give meaning to something else [ 17 ]. A challenge highlighted in the development of deployable contextaware services is the aptitude to lever ambiguous contexts as both sensed and interpreted context is often unclear. In addition deployment of contextual information to mobile applications brings to light the trust and privacy features critical with automated processing of sensitive information. The sources available to capture contextual information in this research include sensors in mobile devices, RFID tags, network servers and application servers among others. Contextual awareness between disparate entities seeks to facilitate interoperability between application platforms with some context immeasurable, but derived through inference [ 18 ]. Activity Theory Modeling Activity Theory Modeling is taken from the psychology and social science disciplines, working in the areas of consciousness and cognitive acts within phenomenology [ 16 ]. Activity theory applied in AMBE finds application in the areas of context awareness and situational descriptions because of its adaptability to socio-technical perspectives and centers on three key concepts namely; action, situation and presence to give context [ 16 ]. The ability to capture the context of the user in state, application and service requires interpretation of the ‘6 W’s; ‘Who?,’ ‘What?,’ ‘Why?,’ ‘Where?,’ ‘When?’ and ‘hoW?’ and is central to the design and profile of the user. Context is argued to be a feature of interaction in any human-computer symbiosis [ 7 ]. It is based on the premise that intelligence is action orientated and context can be used to bring order and clarity to unclear situations in order to deliver appropriate actions. Therefore context is seen as a tool for action selection. Within AMBE enabling device exchange whilst sustaining the capabilities and resources of the current session is part enabled by context awareness. Location information is another form of context aware information. ‘Activity Theory Modeling’ may be incorporated further into the design process of AMBE as a means to capture information concerning the user. This modeling may encapsulate the user’s intentions towards a display terminal, the capabilities of their display equipment and their surrounding interfaces. Additionally this information could form a ‘migration theory’ between the user, their session and their display. CONCLUSION Ambient Intelligence is a dynamic vision, one in which technology serves information filtering. AMBE seeks to provide an information utility through seamless coalescence and switching of display devices activated by sensors. This is achieved by caching the associated objects and relaying them to another possibly central repository, to facilitate viewing to commence on a different platform. Context-aware and context-dependent information will be captured to provide the dynamics in supporting this feature. The question that arises is how we deliver critical information via ambient displays to highly intensive environments of people centered care through human-centered design. In addition, by incorporating interface plasticity and multimodality how should we test and with what recognised metrics, do we quantify, qualify and assess the standard since these systems are designed not to occupy our full attention rather to augment it. In addition how interoperable are these systems and what future dependability and security features can be applied.

References

1. Ambient Orb ( 2008 ). Available at: http://www.ambientdevices.com/cat/orb/orborder.html

2. Baars , B. J. , Mc Govern , K. Cognitive views of consciousness, What are the facts? How can we explain them?, in The Science of Consciousness (eds .)Veldmans, M. , ( 1996 ), Routledge Press, 69 .

3. Calvary , G. , Thevenin , D. , Coutaz , J. A Reference Framework for the Development of Plastic User Interfaces , 2003 . Available at: http://iihm.imag.fr/publs/2003/MuiBook03.pdf.

4. Carbonell, N. Ambient Multimodality: towards Advancing Computer Accessibility and Assisted Living, International Journal on Universal Access in the Information Society (UAIS , 2006 ), 18 - 26 .

5. Denning , P. J. , Metcalfe , R. M. Beyond Calculation: The Next Fifty Years of Computing, ( Springer 1998 ), 83 - 84 .

6. Dey , A.K. , Abowd , G. D. Towards a Better Understanding of Context and Context Awareness in Proceedings-Conference Human Factors In Computing Systems , The Hague, The Netherlands , April, 2000 .

7. Dourish , P. What we talk about when we talk about context , Personal and Ubiquitous Computing 8 ( 1 ), 2004 , 19 - 33 .

8. Fogarty , J. , Forlizzi , J. , Hudson , S.E. Aesthetic Information Collages: Generating Decorative Displays that Contain Information Proceedings (UIST2001 , New York), ACM Press, 141 - 150 .

9. Forlizzi , J. , Li , I. , Dey , A . Ambient Interfaces that Motivate Changes in Human Behaviour , 2007 . Available at:http://ftp.informatik.rwthaachen.de/Publication/CEUR-WS/Vol254/paper02.pdf.

10. Information Aesthetics , 2008 . Available at: http://infosthetics.com/archives/ambient/.

11. Interaction Design Thesis , 2007 . Available at: http://interactionthesis.wordpress.com/ 2007 /02/14/dan gling-string/.

12. Khedo , K. Short Notes on Ambient Intelligence , 2007 . Available at: http://khedo.wordpress.com/ 2007 /05/18/short-noteson -ambient-intelligence.

13. Kuribayashi , S. , Wakita . A. PlantDisplay: turning houseplants into ambient display , in Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology , 2006 . Available at: http://portal.acm.org/citation.cfm?id= 1178823 . 117887

14. Mann , S. , Georgas , M. , Janzen , R. Water Jets as Pixels: Water Fountains as both Sensors and Displays , in Proceedings of the eighth IEEE International Symposium on Multimedia (ISM'06) . Available at: http://www.eyetap.org/papers/docs/ieee_ism_funtain_ simon_game274600766.pdf.

15. Remagnino , P. , Foresti , G. L. Ambient Intelligence: A New Multidisciplinary Paradigm , in IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans , Vol. 35 , No 1, 2005 .

16. Riva , G. , Vatalaro , F. , Davide , F. , Alcaniz , M. Ambient Intelligence, the Evolution of Technology Communication and Cognition - Towards the Future of Human-Computer Interaction IOS Press, 2005 , 21 - 23 .

17. Schmidt , A. , Beigl , M. , Gellersen , H.W. There is more to Context than Location Computers and Graphics Journal, Elsevier 23 ( 6 ), 1999 , 893 - 902 .

18. van Sinderen, M. J., van Halteren , A.T. , Wegdam , M. , Meeuwissen , H. B. , Eertink , E. H. Supporting Context-Aware Mobile Applications: in IEEE Communications Magazine, Sept 2006 , 96 - 104 .

19. Vogel , D. , Balakrishnan , R. Interactive Public Ambient Displays: Transitioning from Implicit to Explicit , Public to Personal, Interaction with Multiple Users , 2004 . Available at: http://www.dgp.toronto.edu/~ravin/papers/uist2004_a mbient. pdf.

20. Weiser , M. , The Computer for the Twenty-First Century , in Scientific American September 1991 , 94 - 104 .