Model-based Generation of Interactive Digital TV Applications Fabio Paternò, Sandro Sansone ISTI-CNR Via G.Moruzzi 1 56124 Pisa +390503153066 {fabio.paterno, sandro.sansone}@isti.cnr.it ABSTRACT • The abstract user interface, which provides a modality- In this paper, we present a solution for the model-based independent description of the user interface; generation of interactive software applications for the digital TV • The concrete user interface, which provides a modality- platform, which is soon likely to become one of the most dependent but implementation language-independent widespread interactive platforms. We show how this solution can description of the user interface; be integrated in an authoring environment for model-based design of interactive multi-platform applications and used at run-time for • The final implementation, in an implementation language supporting migratory interfaces, which allow users to freely move for user interfaces. about, change device and continue task performance from the The advantage of this type of approach is that it allows point they left off in the previous device. designers to focus on logical aspects and take into account the user view right from the earliest stages of the design process. In 1. INTRODUCTION the case of interfaces that can be accessed through different types Model-based approaches to interactive applications have often of devices the approach has additional advantages. First of all, the been criticised because they require time and effort to develop the task and the abstract level can be described through the same models and then the control on the resulting user interface is languages for whatever platform we aim to address. Then, in our sometimes limited. However, the increasing availability of approach we have a concrete interface language for each target various types of interactive devices (desktop, PDAs, cell phones, platform. By platform we mean a set of interaction resources that wide screens, …) has brought renewed interest in such model- share similar capabilities (for example the graphical desktop, the based approaches: the idea is to have device-independent vocal one, the cellphone, the graphical and vocal desktop). Thus, languages able to highlight the main design concepts and then a given platform identifies the type of interaction environment some transformations that generate the implementation adapted to available for the user, and this clearly depends on the modalities the current target platform. This type of approach has been used supported by the platform itself. to support access through various platforms: desktop, mobile, combined used of graphical and vocal interface, but has not been At the abstract level we introduce a number of basic applied to an emerging platform such as the digital TV. In this elements able to support different activities in a platform- paper, we present a solution that is able to generate user interfaces independent manner, and we also describe how to compose such for the digital TV platform. To this end, we had to capture the basic elements through some composition operators. In specific aspects of interactions with such platform, consider the particular, the composition operators have been defined taking design criteria to obtain usable results and incorporate them into a into account the type of communication effects that designers aim transformation able to transform logical user interface to achieve when they create a presentation (grouping, relation, descriptions into digital TV application implementations. ordering, hierarchy). The concrete level is a refinement of the abstract interface: depending on the type of platform considered In the paper, we first provide some background information there are different ways to render a specific object of the user regarding the model-based approach framework that we apply. interface and a specific operator of the abstract user interface. For Next, we discuss the specific aspects of digital TV and user example, in a graphical desktop system a navigator can be interface design criteria for this platform. Then, we describe our implemented either through a textlink, or an imagelink or a simple transformation from logical description to implementation. Lastly, button. The same holds for the composition operators: indeed, the we illustrate how it is exploited at both design and run-time. desktop environment allows using tables, so the grouping operator can be implemented by a number of techniques including both 2. BACKGROUND unordered lists by row and unordered list by column (apart from In the research community in model-based design of user classical grouping techniques such as fieldsets, bullets, and interfaces there is a general consensus regarding the useful logical colours). However, in a mobile phone platform the limited descriptions [2][5]: capabilities do not allow implementing the grouping operator by using an unordered list of elements by row, thus this technique is • The task and object level, which reflects the user view of not available on this platform. In a vocal device, on the other the interactive system in terms of logical activities and hand, grouping can be achieved by inserting specific sounds or objects that should be manipulated to accomplish them: pauses or using a specific volume or keywords. 3. USER INTERFACES for DIGITAL TV • define graphical or textual help functions; As already stated, herein we focus on how to support devices, There is no standard so far, even if there are several constraints in such as the digital TV, which are not traditionally connected with the TV platform. Thus, it is important to: the office environment. In the case of the digital TV, the objective • use appropriate fonts, such as Tiresias, and appropriate is to understand the type of issues that inclusion of such a text sizes from 20 (for notes) up to 36 (for titles) points; platform would raise in comparison with traditional desktop systems. Indeed, although this platform is similar in many ways to • reduce flickering by avoiding drawing lines with a size a graphical desktop, we must nevertheless take into account that of 1 or 2 pixel; users generally have no mouse or keyword for interacting, but just • avoid strongly contrasting colours in contiguous areas a TV remote control. Therefore, as for comparing the concrete in order to avoid flickering; user interface for digital TV with that for the graphical desktop, • choose appropriate colours taking into account that the we noticed some interesting differences. One of these was DVB-MHP standard supports a limited number of colours deciding to use a specific kind of font –Tiresias– due to its high (palette with 188 colours). suitability for display on TV screens. In addition, in order to guarantee the best readability of the text, quite high font Some applications require the user to enter some data. The lack of dimensions were selected (range between 20-36 points), avoiding a keyboard requires using the TV control for this purpose. Thus, it the smaller ones, which do not guarantee sufficient readability. is important to: Usability is a fundamental consideration for interactive services • limit the number of fields to fill in; delivered through technologies, such as the digital TV, which can • define mechanisms to enter textual data; be accessed by people with any background, often with limited computer skills. Since this is a new area there is still a lack of • use the TV control to enter numerical data; standards, but some guidelines have started to emerge [3] based • inform the user when the return channel is used; on a number of user tests. If we analyse the main characteristics • indicate the result of the transactions through of digital TV interfaces, we can find that they involve many appropriate feedback. aspects, as discussed in the following. The nature of the TV is different from many other communication In digital TVs the availability of an application should be clearly media providing textual information. It is thus important to pay indicated and the access should be simple, fast and clear. In attention in order to avoid making the reading process tedious. To particular, it is important to: this end the interactive applications should: • indicate the availability of the application; • present the text in such a way that the important • set a key to access the interactive application; information is at the beginning; • set a key to exit; • use simple verbal forms; • set a key to support zooming; • use titles, subtitles and abstracts to structure the text; • indicate when the application is closing; • divide text on multiple pages or insert a scrolling bar; • indicate the loading state. • use the colours with semantic purposes. The application structure should be clear and intuitive. In particular, it is important to: 4. LOGICAL DESCRIPTION for DIGITAL • separate content and commands; TV USER INTERFACES In order to support transformations from a platform-independent • locate the most important elements in the top-left area; language to an implementation (in our case for the digital TV), we • locate the audio/video flow in the top-right area; use an intermediate language, the concrete description, which is a • dedicate a portion of the screen to the commands; refinement of the abstract interface but platform-dependent. It is important to consider that there is no mouse to support Table 1 provides a description of how each abstract interface navigation but TV controls. Thus, designers should: element is implemented in the digital TV platform. Three types of • structure the content hierarchically; elements are considered: output-only, interaction and the composition elements (indicating how to put the other elements • assign a consistent and unique function to the keys; together). Thus, for example, a navigator element (which allows • keep the order and representation of the control keys users to move to another point of the application) can be consistent on the screen; supported through textual or graphical links or buttons. • allow users to access menus through directional, In our case the generation of the user interface implementation coloured or numerical keys associated with the functionalities. involves the generation of a file in a Java version for digital TVs It is important that the user receive feedback of the activated representing a Xlet, which is an application that is immediately process before it terminates. Thus, the interactive applications compiled and can be interpreted and executed by the interactive should: TV decoders. Xlets bear a strong resemblance with common java applets, with the difference that, instead of the Web browser • promptly communicate through messages what is (which executes the applets) there is the MHP layer of the digital happening; receiver (Set-Top-Box) which interprets them. Only output Digital TV Implementation A specific font for the TV is used (Tiresias). The text size is usually larger Textual than in computer output. It usually varies from 20/22 for the notes to 36 for the titles. Images in their real dimensions or scaled if Object they are too large. It is possible their zooming or scrolling. The annotation of the images can be Description presented as text under the image Interaction Digital TV Implementation Operations are associated with the selection Navigator of textual links, graphical links, and buttons The TV Reset Button resets the fields of a form as well. Activator Button and functions: associate the button selection with a TV function execution Text fields are supported. The text editing can be performed by selecting the field to Figure 1. The authoring environment for Digital-TV. Text Edit edit and entering the data through a virtual keyboard. Our logical descriptions and transformations have been included It is as the text edit but there is no need for in an authoring environment. Figure 1 shows such an Numerical Edit the virtual keyboard because it is sufficient environment in the example of a museum application to use the numbers of the TV control. corresponding to the interface in Figure 2. The interface is divided It is implemented with radio button (with into four main panels: one (top-left) listing the presentations of Single Selection up to 6 choices) and a list box (with more the application; the abstract interactors and composition operators than 6 choices) of the currently selected presentation are shown in the top-right part; the concrete attributes of the currently selected abstract It is implemented with check box (with up element are indicated in the bottom-right part (with the possibility Multiple to 6 choices) and a list box (with more than of editing them); and lastly the connections supporting the Selection 6 choices) navigation are listed in the bottom-left part. Composition Digital TV Implementation The next figure shows a part of a museum application in which it Groups elements lining them vertically or is possible to fill in the information requested from the user and Grouping horizontally, or with the same colour or proceed with making reservation, and such editing activity is with bullets performed by using a virtual keyboard. Ordering Order elements through numbered list Most important elements are located in the Hierarchy top area with lager graphical attributes It is implemented through a container of interactors and their composition operators, Relation in which some activators trigger functionalities involving all the elements. Table 1 Logical elements implementation in Digital TV. In real settings, the file generated with our tool (the Xlet) can be downloaded on the Set-Top-Box (via broadcasting or http). If this is not possible, an emulator can be used to execute the interactive Xlet. For our examples, we used the XletView emulator (http://xletview.sourceforge.net), we will show an example of its use in the next sections. Figure 2. Digital TV interface for Booking a Ticket. 5. MIGRATION The transformations that we have discussed aiming at obtaining The example application considered in this case is a shopping implementations adapted to the features of the target platform are application. This allows users returning home from work to start also useful at run-time. An interesting application is in migratory to prepare the shopping list through a mobile device (see Figure interfaces: interface able to follow the user across different 3) while they are on the bus or train. When they arrive home they devices allowing task continuity and user interface adaptation. may realise that something is still missing and thus decide to This type of interface is very useful for forthcoming ubiquitous migrate to the digital TV so that they can continue the editing of environments in which users want to freely move and change the shopping list through a larger screen (see Figure 4) and lastly interaction device with the possibility of continuing the task from send the request. the point they left off in the source device. In order to support migration involving Digital TV, we have extended a previously developed infrastructure for migratory interfaces [1]. In practise, the environment supports a number of reverse and forward transformations that allow taking an existing PC desktop implementation of an application, building the corresponding concrete and abstract description and using it as a starting point for creating the concrete description and the implementation adapted for the device accessing it. In addition to interface adaptation, the environment also supports task continuity. To this end, when a request of migration to another device is triggered the infrastructure detects the state of the application modified by the user input (elements selected, data entered, …) and identifies the last element accessed in the source device. Then, a version of the interface for the target device is generated, the state detected in the source device version is associated with the target device version, which is then activated at the point supporting the last Figure 4. The Digital-TV application interface after migration. basic task performed in the source version. In order to perform such transformations the logical descriptions play a key role. For example, when deciding how to associate the 6. CONCLUSIONS state of the source version to the target version, our migration We have presented a solution for the model-based design and infrastructure looks for the abstract elements corresponding to the generation of interactive applications for digital TVs. It is based implementation elements modified in the source version, then on the use of specific models able to highlight the important identifies what elements in the target version are used for their aspects related to the user interface and provide a description at implementation and then associate the entered values to such various abstraction levels. Such models and the associated elements. Likewise, the point in which the target interface is transformations have been integrated in an authoring environment activated is identified: the environment identifies the last input and a migration infrastructure. We have also shown example entered and the corresponding abstract element and then looks for applications. The authoring tool is available for external its implementation in the target version. The presentation download and use (http://giove.isti.cnr.it/teresa.html). including it will be activated in the target device because it is assumed that the users want to continue from the point in which they left off in the source device. 7. REFERENCES [1] Bandelloni, R., Mori, G., Paternò, F., Dynamic Generation of Migratory Interfaces, Proceedings Mobile HCI 2005, ACM Press, pp.83-90, Salzburg, September 2005. [2] Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J. A Unifying Reference Framework for Multi-Target User Interfaces. Interacting with Computers. Vol. 15, No. 3, June 2003, pp. 289-308. [3] Fondazione Ugo Bordoni – “Raccomandazioni per le interfacce dei servizi interattivi della televisione digitale”, http://www.fub.it/ambientedigitale/repository/Generale/Racc omandazioni.pdf [4] Mori, G. Paternò, F. Santoro C., Design and Development of Multi-Device User Interfaces through Multiple Logical Descriptions, IEEE Transactions on Software Engineering, August 2004, Vol.30, N.8, pp.507-520, IEEE Press. [5] Paternò, F., “Model-Based Design and Evaluation of Figure 3. The interface for the mobile device Interactive Application”. Springer Verlag, ISBN 1-85233- 155-0, 1999.