SAMSUNG and SAIT Interaction Lab
Context Awareness R&D
Financial Services Electronics Industries Not-
Not-For-
For-Profit Other Biz. Areas
Organizations
at SAIT Interaction Lab Fire
Ho-am Foundation Cheil communications
Marine Samsung Economy R.I.
Shila Hotels and resorts
www.sait.samsung.co.kr Credit cards Samsung Welfare Center
Everland amusement park
Life insurances etc. etc.
etc. SAIT
Seoul
Gi-heung
Daejeon
Daegu
ubiPCMM 05: 1st International Workshop on Personalized Context Modeling and Management for UbiComp Application Busan
1 2
Mission : The Challenge of Innovation Technology-oriented next UI
Dominant Design by the Next UI Technology
Leading the Next UI Technology Conventional UI Next UI
Leading the Next Samsung Graphical Physical Auditory Gesture Voice Haptic Touch
UI UI UI UI UI UI UI UI
…
TN’s Next Post - Mobile Realization
Information Architecture * Intelligent Agent
Post - DTV
DM’s Next (Interactive & Media-rich )
Graphic Processing
Sensing
DA’s Next Post - Home Appliance Basic Tech. Audio/Sound Processing
Context Awareness
Physical Design
User Modeling
Next UI Technology: Intelligence & Mobility * Comprise inference rules, intelligent agent, proactive response etc.
* TN: Telecommunication Networks, DM: Digital Media, DA: Digital Appliance
3 4
1
Technology Group Employee Involvement
Maximizing communication and enhancing teamwork Increasing efficiency and strengthening our core division
Anthropologist
Human
Human Environment
Environment Machine
Machine Psychologist
Sociologist
etc.
Degree of involvement
Interaction Modeling Context Awareness
User
User Research
Research Context
Context Analysis
Analysis
Designer
• Reasoning & Inference Designing • Application Development
• Behavior Analysis • Haptic Synthesis
• Usability Engineering • Sensor Signal Processing
• Multimodal UI Prototyping Active feedback
• Context Understanding
Engineer
Usability
Usability Testing
Testing Optimization/Verify
Optimization/Verify
Conceptual design Implementation System Integration
Phase of R&D
Intelligent I/O Solution
5 6
3D- Gesture Recognition Haptic Display on a Mobile Phone
Haptics
Information
3G accelerometer CDMA processor to be
(Kionix KXM52) (Qualcomm MSM) presented
Perception/ Immersive
3-d motion 3-d voltages calculation
3
Emotion Mobile Device
Another way of presenting information.
Mobile Phone
• Electro-tactile actuation
Tactile Display • Real-time tactile information.
3-d movement recognition algorithm
Haptic rendering
Physical Noise Posture Motion
property change cancellation calculation detection
MP3 Player End point Feature
HMM Haptic firmware
detection Extraction Haptic device
7 8
2
SCURRYTM: : Wearable Input Device Interactive Remote Controller
- Wearable & Virtual Input Device
infrared rays
- User configurable HW/SW Sending data
to DTV
SCURRY
wearable input device
iMEMS Optical sensor
Immersive 3D game
Mobile device
Easy of use New experience
Wearable computer - Point detection using 4 LEDs & vision sensor
Core technology
- Absolute pointing (Usable for DCM)
ScurryTM
9 10
Direct Mapping UI Intelligent Content Creation
DTV Remote Controller
1
1 2 - Preference estimation
2
3 - Story creation
3
As-is To-Be
User Interaction Intelligent Agent User Interaction
Picture search
• ‘look-and-feel’ concept Picture selection User
User Preference
Preference Estimation
Estimation
Picture grouping Content
Content Event
Event Clustering
Clustering Contents selection
Music search Story composition
Dynamic
Dynamic Layout
Layout Generation
Generation
Music selection
Multi-path
Multi-path Story
Story Creation
Creation
Picture layout
Special effect selection
Special effect layout
Zoom out
Content
Content Interaction
Interaction Design
Design
View Contents search
Interaction
Interaction Device
Device Design
Design
User feedback
11 12
3
Home Robot : Context-aware Technology Home Robot: Application Areas
Context-Aware Technology
Recognizing position related context
DTV Air conditioner Home theater Cleaner Air purifier Security system
Intelligent Chip Solution Movable
Home Appliance Home Appliance
Robot Position Speaker Position User Position
Home Robot
Self-
Call & Come User-Following
Localization
Education Information Delivery Silver Care
13 14
Electronic Compass UC Service Scenario (1/4)
f Make-up
Hyen-Hyi, a university student, requests information necessary for Hyen-Hyi asks Make-up agent which make-up is suitable sitting in front
make-up from Make-up agent in front the u-mirror of before going to of the u-toilet stand.
class in the morning.
Level. 1
Make-up agent informs her information about her schedule, today’s
Make-up agent informs her the schedule, the spare time, the weather, weather, the time required for make-up, and her state of skin.
and the her state of skin on the u-mirror.
She decides to do the basic make-up, sun-protection, and the color-
Microprocessor & UART Interface Type She decides to go to class fast after doing her make-up. tone make-up for going-out.
(6.5 X 6.5 X 1.2 mm3)
Hyen-Hyi, a university student, requests information necessary for Hyen-Hyi asks Make-up agent which make-up is suitable sitting in front
make-up from Make-up agent in front the u-mirror of before going to of the u-toilet stand
Digital Output & SPI Interface Type class in the morning.
Level. 2
Make-up agent suggests two ways; First, doing the basic and sun-
(4 X 4 X 1.2 mm3) Make-up agent tells that she have the spare time of 30 minute and protection make-up because it has the strong direct ray of light today,
Map Heading-up suggests doing the make-up because it has the strong direct ray of and she has the sensitive skin. Second, doing both the color-tone
light today. make-up and the basic and sun-protection make-up because she is
scheduled to go out.
She decides to do the make-up fast.
She takes the second suggestion.
Hyen-Hyi, a university student, requests information necessary for Make-up agent concludes that she needs the sun-protection make-up
Level. 3
make-up from Make-up agent in front the u-mirror of before going to as well as the basic make-up, moreover she is required to do the color-
class in the morning. tone make-up because she is scheduled to go out.
Make-up agent advises her to finish the make-up within 20 minutes,
considering the spare time before class, today’s weather, and her state
of skin.
15 16
4
UC Service Scenario (2/4) UC Service Scenario (3/4)
Environment
Cosmetics
Preference
f Make-up
Cosmetic
Refrigera
Schedule
Financial
Manager
Manager
Manager
Manager
Manager
U-Toilet
Manager
Manager
Manager
Make-up
MIRROR
Method
History
Object
Health
Agent
stand
&Sets
tor
U-
U-
U-
Request for time, schedule,
time required, data
Time, schedule,
time required, data
Request for data of mood, state of skin
Decide yes or no for the make-up
Request for data of mood, state of skin
Data of mood, state of skin
On requesting the method for make-up, Hyen-Hyi takes from Make-up On taking request for make-up method, Make-up agent shows Data of mood, state of skin
agent information about today’s schedule and weather, her state of information about the state and the location of cosmetics necessary for
skin, various ways for make-up, and her usual history of make-up, and the selected make-up.
Level. 1
the make-ups fitting to her style.
She finds the cosmetics and gets ready to do the make-up. Request for data of weather
She chooses the make-up to reduce the time required and fit to the
color of her cloth Data weather
Request for message of suggestion of the make-up
On taking request for make-up method, Make-up agent suggests two When Hyun-Hyi asks the state of cosmetics, Make-up agent tells that all
make-up methods; type A and type B, considering the little spare time the cosmetics are fine and some of them are located in u-cosmetics Message display of suggestion
and her cloth. refrigerator, and suggests she take some brushes from her sister’s room. Confirm the message display of suggestion of the of the make-up
Level. 2
make-up, decide yes or no for the make-up
She chooses the type A. She take them off u-cosmetics refrigerator and bring the brushes
Make-up agent concludes the type A for the make-up, considering the Make-up agent checks the state of cosmetics set and sends the request
Level. 3
little spare time and her cloth. message to bring brushes to her sister. Cosmetics necessary in the u-
toilet stand and the u-cosmetics refrigerator flash, so she easily finds
them.
17 18
UC Service Scenario (1/4)
Environment
Cosmetics
Preference
Cosmetic
Refrigera
Schedule
Financial
Manager
Manager
Manager
Manager
Manager
Manager
U-Toilet
Manager
Manager
Make-up
MIRROR
Method
History
Object
Health
Agent
stand
&Sets
tor
U-
U-
U-
Some men see things as they are and say “Why?”
Request for time, schedule,
time required, data I dream things that never were and say “Why not?”.
Time, schedule,
time required, data – Robert F. Kennedy -
Request for data of mood, state of skin
Request for data of mood, state of skin
Data of mood, state of skin
Decide the type for the make-up
Data of mood, state of skin
Request for data of weather
THANK YOU !
Data weather
Request for message of suggestion of the make-up type
Message display of
Confirm the message display of suggestion of the suggestion of the make-up
make-up type, decide the type for the make-up type
Sang-
Sang-Goog Lee, SAIT Master / Ph.D
sglee@samsung.com
19 20
5