The World Wide Web (WWW) is one of the main entry points to access and consume Internet content in various forms. In particular, the Web browser is used to access different types of media (i.e., text, image, audio, and video) and on some platforms is the only way to access the vast amount of information on the Web. Recently, it has been proposed to stimulate also other senses than vision or audition while consuming multimedia content through socalled sensory effects, with the aim to increase the user's Quality of Experience (QoE). The effects are represented as Sensory Effects Metadata (SEM) which is associated to traditional multimedia content and is rendered (synchronized with the media) on sensory devices like fans, vibration chairs, lamps, etc. In this paper we provide a principal investigation of whether the sensory effects are ready for the WWW and, in anticipation of the result, we propose how to embed sensory effect metadata within Web content and the synchronized rendering thereof.
In recent publications, it was proposed [
In [
Proposed Approach for Integrating Sensory Effects within Web Content The preferred way to link resources such as a SEM description to a Web document – herein after referred to as document – is the link element. An example usage of the link element to reference a SEM description is shown in Listing 1.
Listing 1. Example usage of the link element to reference a SEM description.
<link rel="alternate" type="text/xml" title="SEM description"
href="http://server.org/sem.xml" media="light, wind, vibration"/>
The attributes type, title, and href are used as specified by HTML. The rel
attribute indicates that the SEM description provides an alternate representation of
the current document which, in our view, conforms with the concept of sensory
effects as depicted above. In particular, if used in conjunction with the media
attribute, the ref attribute indicates that the referenced SEM description is intended
for use with the media specified. Please note that the media attribute must support
some version of the media query language [
Based on the information in the link element, the SEM description could be
parsed either directly via JavaScript or within a browser plug-in utilizing an MXM
engine as proposed in [
However, the main issue is the synchronization with possible multimedia resources
included within the document such as video or audio content. Therefore, it is required
to access the current playback position of the actual multimedia resources which is
provided thanks to HTML5’s video and audio elements and corresponding DOM
attributes [
Listing 2. Usage of timeupdate and currentTime for synchronization of video content with sensory effects. <script> ... function time_update() { var vid = document.querySelector('video'); var now = vid.currentTime; // note: sem holds sensory effects information previously parsed for (var i=0; i<sem.length; i++ { if (now >= sem[i].start && now <= sem[i].end) { update_sensory_devices(sem[i]); break; } } } // end function time_update() ... </script> ... <video src="http://server.org/video.mp4" ontimeupdate="time_update()" autobuffer controls>
Finally, in [
Acknowledgments. This work was supported in part by the European Commission in the context of the NoE INTERMEDIA (NoE 038419), the P2P-Next project (FP7-ICT-216217), and the ALICANTE project (FP7-ICT-248652).