In this paper, we propose a Multimedia Scripting System (MSS) to support endangered language learning. With the use of advanced multimedia techniques and web 2.0 technology, MSS provides a user friendly environment to ensure that general users (without strong technical backgrounds) in an on-line community can e®ectively and collectively create multimedia stories and game scripts for a common cause. It is an essential component of our on-going endangered language revitalization project.
As stated in [
In this paper, we present some on-going work and ideas on revitalizing endangered languages using a multimedia scripting system (MSS). MSS promotes the use of advanced multimedia techniques and web 2.0 technology to ensure that new evolving multimedia presentations (i.e., multimedia stories or game scripts) can be dynamically created, edited, aggregated, syndicated and shared via web by both experts and general audiences. The game scripts and multimedia contents collected and created in MSS can later be used for the development of a gaming environment.
The remainder of the paper is organized as follows. Section 2 gives a brief background introduction and literature review. Section 3 details the MSS system. Finally, Section 4 concludes the paper and discusses some future works.
Languages are an essential part of the living heritage of humanity but more
than half of the 6000 world languages will disappear in this century [
It is commonly agreed that game development is an interdisciplinary ¯eld [
Our project is envisioned to accommodate various language communities.
Currently, we are working on revitalizing Blackfoot, a Native American language
with very few documents and only about 5000 speakers in the world [
A group of activities are supported in MSS, which can be roughly
categorized into Web-based and multimedia-based functions. The ¯rst category
includes some common Web functions, such as Upload (i.e., adding data to the
system server), Post (posting a subject to the webpage), Describe (adding
descriptions, which can be used for searching, about a subject), Search (¯nding
interested data), Recommend (sending the link of a subject to others via emails),
Comment (posting comments about a subject), Share (de¯ning the read/write
permission about a subject among the community), etc. Currently, an interface
to YouTube [
The multimedia-based functions are the focus and main contributions of MSS, which are used to generate and update multimedia presentations. In MSS, a multimedia presentation is broadly de¯ned to be either a single media object (e.g., a text document, image, audio track, video clip, etc.) or a composition of multiple media objects that are related temporally and/or spatially. Therefore, a multimedia story or a game script is essentially a multimedia presentation. Two functions, Multimedia Presentation Authoring and Layered Augmenting, have been developed to enable the collaborative creation and manipulation of multimedia presentations by general users without strong technical backgrounds. 3.1
Multimedia presentation usually contains various media objects with spatial
and/or temporal relationships. In our current prototype, we have adopted the
Multimedia Augmented Transition Network (MATN) [
As shown in Fig. 1(a), to add a media object into a presentation, users can ¯rst issue a query using the Search function (de¯ned earlier) and see the results in the upper-left window. A preview of the selected object will be given in the bottom-left window. By using the \Add an arc" function, a window pops up where the selected ¯le can be added. The users can continue adding more ¯les if they are to be synchronized. By clicking the \Enter" button, a corresponding arc will be generated. The main window in Fig. 1(a) shows a modeled presentation, where the labels on the arcs represent di®erent media objects to be included in the presentation and the circles represent their starting and ending points (reading from left to right). A multimedia presentation can be easily created, edited, saved, and opened through this MATN-based authoring environment. We have also implemented an interpreter to translate the MATN structure automatically to an HTML+SMIL document for Web-based presentation rendering (see Fig. 1(b) for a snapshot of the resulting presentation). 3.2
To facilitate collaborations among the community, a system should have the
capability to support media manipulation and to enable shared but di®erentiated
social viewing among users [
As shown in Fig. 2, it has four main sections, a main window on the top, an editable information panel on the bottom-left, a toolbar on the bottom-right (with \Add Text," \Add Image," and \Add Audio" buttons and a \Submit" command) and a layer manager box (above the toolbar). After selecting a video (from YouTube in this example), users can apply enrichments by click on \Add Text," \Add Image," or \Add Audio" button. A layer is created and its name is placed in the layer manager (\Layer 1," \Layer 2," etc.). Editable information panel shows its related parameters, where users can enter texts or upload image/audio depending on the type of the enrichment. Such enrichment is automatically placed at the upper-left corner of the main window but can be dragged around using the mouse (the X and Y coordinates in the editable information ¯eld will be changed accordingly). Users can also de¯ne when to show such annotations by specifying the On and O® time. More layers can be added following the same process. Each layer can be edited by selecting its name in the layer manager and changing values in the editable information panel. In the example, three layers (one text box and two images) are placed on the black margin of the video. If an audio layer is added, it will replace the original sound track of the video during the speci¯ed On and O® time. These are dynamically created and inserted to the page through Javascript, utilizing the DOM structure.
Once the user has ¯nished layer creation, the video can be saved and uploaded back to the MSS server via AJAX to be used in multimedia presentation. The advantage of layer augmentation is that users are allowed to create alternative views by selecting or unselecting the layers in the layer manager box. For example, by unselecting layers 2 and 3, the video becomes the one showing inside the top right box of Fig. 1(b). 4
In this paper, a multimedia scripting system (MSS) is presented for endangered language learning. In the system, various web-based functions and multimedia functions are proposed to ensure an on-line participating environment where general users can share their opinions and actively contribute to the creation and improvements of multimedia stories and game scripts. The system can be further improved in many aspects. For instance, the Multimedia Presentation Authoring function can be extended to model the spatial relationships among media objects and the Layered Augmenting function needs to support the enrichment on other media objects besides videos. The game scripts and multimedia contents collected and created in MSS can be used for the game development.