Introduction The vision of Ambient Intelligence assumes a shift in computing towards a multiplicity of communicating devices disappearing into the background, providing an intelligent environment, where the emphasis is on the human factor. Realizing this vision requires the integration of expertise from a multitude of disciplines. Despite the rapid advancement of these fields, existing approaches have difficulty in meeting the real-world challenges imposed by developing ambient information systems.
Context in AmI Aim of AmI systems ► right information to the right users, at the right time, in the right place, and on the right device ► Requirement:
thorough knowledge and understanding of context Context in Ambient Intelligence ► “.. any information that can be used to characterize the situation of an entity. An entity is a person, place or object that is considered relevant to the interaction between a user and application, including the user and application themselves..” [Dey and Abowd, 1999]
Activities Creation of small-scale experimental AmI spaces ► AmI Sandbox ► Smart Office Building a new facility for R&D
in AmI technologies R&D through competitive funded projects at national and European level An experimental space within ICS-FORTH ► 6 rooms (~ 100m2) Installation, testing and integration of a large variety of technologies and applications Allows researchers from different domains to bring together and share their know-how and resources Main goals ► Experimentation in a creative,
flexible and informal setting ► Acquisition of hands-on
experience ► 1st step towards the AmI
Facility Installed Technologies ► Computer vision system, comprising 8 cameras ► Surround speaker system with 8 speakers ► Various computer-operated lights (neon, spot lights, floor and desk lamps) using both the
DMX and X10 protocols ► Computer-operated air-condition ► Various screens and high definition TVs,
including touch screens ► One large front projection screen created by 2
ceiling-mounted short-throw projectors ► One back projection screen ► Several sensors (distance, temperature, etc.)
and actuators ► Desktop and mobile RFID readers ► Interactive table ► Access control systems (IRIS Scanner, RFID, …) ► Positioning system through wireless access
points ► Various robotic systems The AmI Sandbox
Smart Office Augmenting an existing office space with AmI technologies ► Multiple interconnected displays Large screen e-Desktop e-Frame ► Smart table ► Controllable lights ► Computer vision camera ► e-pens ► Distance sensor ► Laser keyboard The Smart Office New building (~ 3.000m2) ► Basement, ground floor, 1st
floor Fully accessible by people
with disabilities Includes: ► Simulation spaces ► Laboratories for R&D in AmI
technologies ► Offices
Permanent research staff & visitors
Blueprints Ground floor
Basement
Garden
Home
Doctor’s office
AmI
Facility Office
Exhibition Class
Entertainment space Simulated home
environment 2 floors (staircase + elevator) ► living room ► Kitchen ► house office ► 2 bedrooms
adults & children ► 2 bathrooms Scenarios ► local, remote and automated home
control ► safety and security ► health monitoring ► independent living ► (tele)working ► entertainment Fully accessible by the elderly &
people with disabilities
Components under development AmI software and hardware architectures Middleware Context management and reasoning Environment sensing technologies & sensor fusion Access control, information and communications security Seamless and intuitive user-environment interaction Speech recognition and speaker localization Computer vision subsystem for multiple
user localization and gesture recognition Dynamic surround sound playing system Environmental control AmI @ FORTH Experience with Context Reasoning AI for AmI
Contextual Reasoning in
Ambient Intelligence Challenges ► Imperfect nature of the available context information Unknown, ambiguous, imprecise, erroneous ► Special characteristics of ambient environments Agents with different goals, computing and perceptive capabilities, and vocabularies Highly dynamic and open environments Distributed context knowledge Unreliable and restricted wireless communications Limitations of current AmI systems ► No formal model for reasoning with imperfect context ► Centralized architectures → No support for distributed reasoning Dr. Amber is located in the ‘RA201’ university classroom reading his e-mails on his laptop. It is Tuesday, the time is 7.50 p.m., and he has just finished with a lecture for course CS566. His context-aware mobile phone receives an incoming call, but it is not in silent mode.
Dr. Amber’s phone is configured to take decisions about whether it should ring in case of incoming calls based on its context and Dr. Amber’s preferences: – The phone should ring, unless it is in silent mode or Dr. Amber is busy with some important activity.
– A lecture at the university is one such important activity.
The mobile phone is not aware of Dr. Amber’s current activity. It attempts to infer the activity using two rules: – If there is a scheduled lecture for a course at this time, and Dr. Amber (actually his mobile phone) is currently located in a classroom, then Dr. Amber is possibly giving a lecture. – If Dr. Amber is located in a classroom, but there is no class activity taking place in the classroom, Dr. Amber is rather not giving a lecture. class
RA201 Information about scheduled events is imported from Dr. Amber’s laptop. According to his calendar, there is a scheduled class event for Tuesdays from 7.00 to 8.00 pm.
The localization service possesses knowledge about Dr. Amber's current position. In this case it 'knows' that Dr. Amber is currently located in 'RA201'.
Technologies Used Interconnection with other Devices can be supported easily ► GPS Receiver
Bluetooth enabled ► Various Sensors
Eg RFID ► Parse messages from these devices to extract knowledge such as coordinates, sensor measurements etc. ► Conversion into DR-Prolog Facts & loading in KB
► Circles are mobile devices ► Oval nodes are server computers, eg. Desktop versions of DR-Prolog ► Cylinder nodes are computers or websites with Knowledge Base Data, Theories, etc in
DR-Prolog syntax. ► Connections: Message exchange (Blue: queries & their results, Red: facts) Bluetooth, wifi, or through Internet using 3G, GPRS etc.
PDA h t o o t e u l B Mobile phone
Wi‐Fi Internet (3G,GPRS)
Internet
Social Scenario 1 User with Bluetooth enabled cellphone
passing by a classroom Bluetooth Server with lecture and lesson information for that classroom attempts to connect to cellphone Cellphone based on profile information either notifies the user or rejects the info ► E.g. based on course registration etc. Based on the above Cellphone should inform user for this announcement
Social Scenario 2 User with Bluetooth enabled cellphone sitting at ICS lobby; profile entry e.g.
hobby(‘tennis’).
hobby(X),tournament(X,…) => notifyUser(activity,X…) ► Lobby computer eg:
• tournament(‘tennis’,’location’,’date’)... ► Based on the above
• Cellphone should inform user for this activity ► Send the ‘event’ to closeFriends via SMS Based on profile entries of the reciever he will be notified or not! (reasoning on sms data received on certain port from the DR-Prolog mobile application)
Lessons Learnt A lot of non-logic related work required ► Infrastructure ► Middleware ► Need for Facility! Technical difficultues
► Scenario with SMS, not calls Performance no challenge for small applications ► Combination with large portions of knowledge from (semantic) Web will be the challenge AmI @ FORTH Experience with Context Reasoning AI for AmI
AI for AmI Context representation Context reasoning Privacy Planning Coordination
Context Representation Use semantic web languages to model ► User profiles ► Devices ► Rooms ► Activities Challenge: the usual one ► tradeoff between expressivity and efficiency
Context Reasoning Distributed Heterogeneous ► Different types of information call for different reasoning methods Inconsistency and imperfection tolerant Dynamic
► E.g. reasoning about stream input Efficient
A major concern! Lightweight access control languages Blend in other languages/operations ► E.g. in our context reasoning algorithm
Planning Reasoning about action is a well-formed subfield of AI. ► But the classical planning problem adopted a number of restrictive assumptions to delimit the domain. The AmI environment is open and highly dynamic. World knowledge in AmI is incomplete. Plan generation must preserve a level of uncertainty. Exogenous events occur in AmI environments.
Coordination A device-rich environment that places users in the center of attention. Devices need to coordinate their actions, cooperate in generating plans and collaborate during execution. A decentralized self-organizing infrastructure is a non-trivial challenge for the realization of the AmI vision. Leather desktop
► Extendable interface (when opened) Input ► RFID-augmented objects ► Vision-based left/right switches ► Distance sensor for up/down motion ► IR pens (using the Wiimote) ► Vision-based object position tracking Output ► Projection (on the desktop) ► 2 speakers ► Sounds + speech synthesis Used for: ► Logging in the system ► Running applications
Ambient presentation Uses standard PowerPoint slides Coordinated slide change Replicated drawing on the current slide Can adapt to the current state / size of
the smart desktop Can change the lighting conditions Output ► Multiple screens
desktop, large LCD, laptop Input ► Start/stop: RFID tag (ICS-FORTH leaflet) ► Next / previous: right/left gesture ► Drawing: IR pen (can change pen color using the pens’ RFID tag)
E-mail Shows the e-mails of the person who
logged in Can adapt to the current state / size of the smart desktop ► Standard size: mails’ list ► Extended: mails’ list + selected mail’s
content Output
► Smart desktop Input ► Start: RFID tag (envelope) ► Next / previous mail: right/left gesture ► Use slider: distance sensor ► Buttons press: IR pen (click) ► Send predefined e-mail:
RFID tag (photo or ID card)
Mobile photo Get photo using the mobile
phone and drag it on any display Output
► Smart desktop, LCD screen, Archie Input ► Mobile phone recognition: RFID tag
(envelope) ► Mobile phone position: Vision ► Photo drag: IR pen
display Monitor e-mail account and visualise email semantics ► Number of e-mails ► Number of e-mails from specific people Junk e-mail ► Virus-infected e-mails ► Unsent drafts ► Receipt of specific e-mail ► Urgent e-mail Output
► Archie Input ► E-mail account ► Touch
Video conference Video conference Button presses around the table result in the camera turning to face the button’s position Output ► Large screen ► Smart desktop Input ► Camera ► Multidirectional microphone ► Buttons 7-player game Gun fighting duel 1 moderator (optional) ► Undead ► 1 button (show up / fire) Output
► LCD screen Input ► Buttons around the table Turn on all devices Connect devices via
Bluetooth