• Centre & my role • Selection of multimedia applications and their presentation design issues • Some observations - Different applications, different design decisions - Applying general design principles
Overview Centre for Digital Video Processing at Dublin City University • Developing automatic indexing/retrieval tools for managing large amount of image/video information – Object/Face Detection & Tracking in Video – Audio & Video Event Detection – Video Delivery on Mobile Devices – Large-scale Distributed Web Image Search – Search Engine Design for Collaborative Video Retrieval – Hardware Accelerator Design for MPEG-4 Mobile Platform – Personalisation & Recommendation for Video – Synergy between automatic & manual indexing – Fusion of multi-modal query results
My Role: Usability & User Issues • Understand the research & development of Image/Video indexing/retrieval tools within the Centre • Think how these could be exploited – Envision the use: scenarios & future system use – Prototyping user-interfaces – Deploy (if possible) – User testing: monitor usage & guide future development Mobile Físchlár-News
Movie Browser CCTV Search System BBC Rushes Search System
shot boundary detection Camera shot
Keyframe Extraction
Físchlár-News Archive • Online archive of daily RTE1 9pm TV news • Automatic video indexing:
News Story Segmentation, based on: – Anchorperson detection (by shot clustering) – Face detection – Advertisement detection – Shot length – Activity measure
Story Segmentation - SVM (Support Vector Machine) with: • Speech vs. music discrimination • Anchorperson shot clustering • Face detection • Shot length cue • Activity measure Shot segmented, advert detected program
Advertisement Detection
browsing, searching, streamed playback and… …recommendation
Video Server
Web application
profile User Evaluation of Físchlár-News: An Automatic Broadcast News Delivery System. Lee H, Smeaton A.F, O'Connor N and Smyth B. TOIS - ACM Transactions on Information Systems, 24(2), 2006.
• Application specific...
• General design principles, guidelines, graphic design, web design, etc. – knowledge & experience I have in general – E.g. Overview first, details on demand
• General design principles, guidelines, graphic design, web design, etc. – knowledge & experience I have in general – E.g. Overview first, details on demand – E.g. Visual consistency Whenever list of stories appears... ... to make obvious what a piece of presentation on the screen represents and doesn’t require interpretation effort Applications Development (interaction design + software engineering)
Físchlár-Nursing
Físchlár-News CCTV Search System BBC Rushes Search System Físchlár-TRECVid2004: Combined Text- and Image-Based Searching of Video Archives. O'Connor N, Lee H, Smeaton A.F, Jones G, Cooke E, Le Borgne H and Gurrin C. ISCAS 2006 - IEEE International Symposium on Circuits and Systems, Kos, Greece, 21-24 May 2006. Applications Development (interaction design + software engineering)
Físchlár-Nursing
Físchlár-News Technology Development for automatic extraction of syntactic & semantic features in image/video
Físchlár-TV Shot Boundary Detection Keyframe Extraction
Mobile Físchlár-News
Movie Browser TRECVid02 Interactive Search System
TRECVid03 Interactive Search System
TRECVid04 Interactive Search
System Face Detection Building Detection Indoor/Outdoor Cityscape/Landscape Advert Detection
News Story
Segmentation Video Recommendation
Sports Summarisation
Image-Image Similarity RF Potential screen complexity – use of main plain vs. background plain, round edges, and corresponding buttons MediAssist (Personal
Photo Manager) CCTV Search System BBC Rushes Search System - all the detected Objects & Events and link possibility are indicated [unit] Link to… … and here is the ASR or Closed Caption text that
Link to…
Link to… [unit]s with linked
OBJECT 3 … and here is the ASR or Closed Caption text that … and here is the ASR or Closed Caption text that … and here is the ASR or Closed Caption text that
Link to…
[unit]s with similar text/image content Selecting
Search result +
Search result
buttons that represent objects), then develop it further in an integrated interface Detected objects have no labelling (meaning) but only as spatial region or blob on the keyframe
User-Interface to a CCTV Video Search System. Lee H, Smeaton A.F, O'Connor N and Murphy N. ICDP 2005 - IEE International Symposium on Imaging for Crime Detection and Prevention, London, U.K., 7-8 June 2005.
list of objects but also geographical map summarising and highlighting the matched object’s route (note the application’s main purpose: chase a suspect)
Observations • Considerable amount of time & effort in manually hard-designing UIs – For example...
refinements: starting with pen-and-paper sketches, then Photoshop sketches, discussing with technical team then me resketching accordingly
Observations • Huge amount of time & effort in manually hard-designing – For example... • Different situations require different specific design decisions – For example, presentation of keyframes
– For example... • Different situations require different specific design decisions – For example, presentation of keyframes