An augmented reality (AR) system supplements the real world with virtual objects that appear to coexist with the real ones [2]. Augmented reality has been applied to different fields, such as medicine, military training, entertainment, or engineering design, but one of the domains where it has the potential to be more useful is cultural heritage.
The application that we are addressing is that of providing the user with information while they are visiting an archaeological site or a monument, creating a personal "touring guide machine". For many people visiting a monument or an archaeological place can be somewhat disappointing, even when accompanied by a tour guide. Archaeological evidence for a building may consist of little more than a few fragments of wall foundation. For the untrained eye it can be difficult to imagine the shape of a building reconstructing it from these few fragments. Also, an augmented reality system provides the user with additional data such as architectural information or historical facts that are not available just looking at the environment.
Though information on these elements can be obtained through different media, such as Web pages or TV programs, most people likes to visit historic and archaeological places in situ, combining entertainment with cultural enlightenment. One of the main advantages of the augmented reality paradigm for the cultural heritage domain comes from the facility to allow the mobility of the user while she is walking around the historical place.
On the other hand, if we compare the information given to a visitor of an historical place by a human guide with that provided by the new technologies, we can see that the last is more complete, and also with a more attractive format (graphics, sounds, three dimensional images).
There already exist prototypes of systems in which the visitor receives this information through a head-mounted display or a tablet [19] [23].
The need to integrate virtual and real information implies a change in the design methodology and design tools with respect to other types of interactive systems that has been little explored. In this paper we will present a methodology for designing AR interfaces for cultural heritage that is supported in its different phases by the use of multimedia sketches.
The organization of the paper is as follows: In the first section we will state the requisites of such a methodology and will review the related work. Next, we will explain why to use sketches in supporting the design. In the third section, we will outline the main steps of the methodology we propose and how are multimedia sketches used in each one of them. Finally, we will present our conclusions and our plans for the future.
The design of AR interfaces for guided tours is a complex process because consists of two kind of designs: 1) the design of the visit: selecting the information to show, developing its historical interpretation, elaborating a journey, and building a narrative script, and 2) the design of the user interface itself (how to show the information, what multimedia means to use, and how the user interacts with the application) Currently, there are methodologies to design some of these elements separately but none that integrates all of them. Mitchell and Economou [18], discuss some methodological approaches. Cohen [8] and McDonnel [17], while reviewing the role of the tour guide, discuss also the content of the visit. In [5] Balaguer et al present the user centered design of a visit to an archaeological site by using scenarios, storyboards, and a video prototype. In the Archeoguide project [21], the design was based in use cases [1]. Granollers et al [9] present how the location attribute is taken into account in the design process and the dialogue model.
We have been working on such an integrated methodology and have stated in [16] the requirements we think it should meet. Among them, we would emphasize that this methodology must take into account the multiple dimensions of the information to show (spatial, timedependent, social and cultural aspects, etc.), the most adequate approach to link them to elaborate a narration, and the method employed to design the appearance of this information using the means provided by the multimedia and the augmented reality paradigms.
Such a methodology must we applied in close collaboration with specialist in multiple domains, such as architects, archaeologists, or experts in cultural tourism and with end users. For communication among them and the designers, we began using paper storyboards, but quickly we started to use multimedia sketches.
Actually, the case of guided tours is a special case of multimedia narration. According to [3] and [13], designers of multimedia applications prefer the development of sketches over formal notations, authoring tool or other prototyping application early in the design process. Sketches allow us to be imprecise, which is needed in the early design when not all the user requirements have been completely gathered. It is also a fast method to have something to check and refine [22].
With respect to augmented reality interfaces, there is a lack of tools for design, particularly for early design. The tool presented in [7] covers early design, but it requires programming, which makes it inappropriate for multidisciplinary design. Other tools, such as [10] and [11], are more oriented to development.
In our case we use sketches not only for storyboarding the presentation, but also in the first steps of the design, to specify requirements, compile and organize the information, and selecting and interpreting it.
Multimedia sketches also let us see the overlapping of computer-generated objects on real ones without treating, in the first stages of the design, the problem of position detection, which is a very desirable characteristic for the design of any augmented reality application, and a particularly difficult point in the case of AR interfaces for guided visits.
Of the various available tools for sketching multimedia applications, we selected DEMAIS [4], mainly because of its capability for expressing dynamic behavior through designer's ink strokes and textual annotations. Originally intended for sketching user interaction elements and synchronization of time-based information display, we use them also to help designers to navigate among the different parts of the information.
Apart from these advantages, DEMAIS helps us to map the multi-dimensionality of our domain (time, 3-space, cultural aspects, historical features) into the 4-dimensional world (time, 3-space) of the historical place visited by the user.
The methodology took as starting point the design of film movies [14], which are a kind of non-interactive multimedia narration. Later, we also took into account the interactivity of these narration for the case of augmented reality and the case of the cultural heritage domain.
The methodology is divided in several steps, that are outlined below.
The target of the application In this case the objective is more than presenting information, but instead to transmit a comprehension of the culture of the visited place.
The typical users will be cultural tourists, but they could also be students. They can also be divided in subgroups depending on age, previous knowledge, distance between their culture and that of the visited place, and so on. There should be variations on the content, structure and navigation for each one of these subgroups.
The selection consists in determining which points are going to be visited and what information is going to be transmitted. As it implies an interpretation of the information, which usually means to make hypotheses from the known facts, it must be done in close collaboration with domain experts.
Though in this type of AR interface the user is allowed to divert from the proposed path, most people prefer to be told an itinerary (or a small set of them) and a basically lineal narration [12].
In this phase, we use multimedia sketches mainly as a mean of communication between the different persons intervening in the design, such as experts in cultural heritage or cultural tourism, multimedia designers, etc. and as a mean to organize the information obtained. Figure 1 shows how an itinerary is designed, using the plane of a building as a base to lay out the paths and to mark elements of interest. DEMAIS layers are used to organize the multiple dimensions of information, and DEMAIS dynamic behaviour elements, such as buttons, to relate elements, acting somewhat as hyperlinks (Figure 2).
When the place has suffered changes along the history, multidimensional diagrams (figure 3) can be used to show these changes. First, the hardware where the application will run must be selected: the computer platform, the type of display, the method and devices to detect user position, because many aspects of the design of the interface are dependent on them.
Next, the type of media to be used (sound, graphics, images) for each information element is selected and these information elements are designed in detail.
As the multimedia contents have to be thoroughly integrated with the physical objects that are seen by the user, we use multimedia sketches to design the overlapping between real and generated elements. In this case, the base for the sketches is photographs and videos of the place (figure 4). Moreover, they can be used to see the degree of visibility of the real elements from each position. Videos also help to design the temporal aspects of the visit, pacing the oral contents with the time it takes the user to cover a stretch of the itinerary (figure 5).
As the objective is not only information transmission, but to make the user enjoy the experience of the visit, designing through sketches allow to experiment easily and with a low cost with different design options and even artistic expression forms. There is not much experience with this type of application, so it is particularly important to evaluate early the different aspects of the design, with the participation of end users.
Finally, elements for explicit interaction with the user can also be designed by using the dynamic capabilities of Demais.
Designing the application software.
It consists on selecting the software platform and designing the software architecture, the mechanism for information personalization, etc.
Though with sketches we can experiment and evaluate the main aspects of the design, there are other factors, such as synchronization and integration [20], that affect usability and whose evaluation requires a working prototype and a visit to the place.
The design of augmented reality interfaces for visiting cultural heritage places is a complex process, because it involves both the design of the visit and the design of the multimedia information presentation. It is also a multidisciplinary process, because it needs the intervention in the design of persons with different backgrounds, such as multimedia designers, usability specialists, archaeologists or architects.
In this paper we have presented the steps of a methodology for solving these problems, and the use of dynamic multimedia sketches in supporting the different phases of the design, from the selection and interpretation of cultural information to the layout of the presentation.
In the future, we plan to use DEMAIS also for early prototyping, in order to experiment and evaluate different interaction techniques and forms of presentation.
We also want to refine the steps of our methodology, and particularly the transition from multimedia storyboards to production, that is not trivial in this case because presentation must be adapted in function of the position and point of view of the user in each moment.