Authoring tools for augmented reality (AR) aim to enable users with no programming skills to create AR experiences. The dominant approach uses mobile-based AR technology. However, some authors have proposed VR-based authoring tools allowing the remote creation of AR experiences in a virtual replica of the space. In this paper we describe an exploratory study that compared these two approaches with 10 participants. No significant diferences in efort and user satisfaction were found. VR authoring was even potentially faster. This suggests VR authoring could be a complementary alternative to mobile-based AR under some conditions, particularly for rapid prototyping and when accessing the physical site to augment is challenging.
CEUR ceur-ws.org
This research advocates for enhancing Augmented Reality (AR) prototyping processes to
promote its adoption by supporting alternative edition scenarios. Most authoring AR tools
implement an approach in which the user superimposes virtual content onto the image of the
real environment captured by a device, often their own mobile phone. This method requires
the user’s physical presence in the space to augment. However this is not always feasible and
sustainable, especially if the augmentation follows an iterative process that needs to assess
several prototypes in realistic scenarios to improve the user experience. Some authors have
proposed an alternative authoring method utilizing immersive Virtual Reality (VR) technology
[
Apart from some successful use cases, to the best of our knowledge, there are no comparative studies between VR and AR authoring for AR experiences. These studies are essential to demonstrate whether VR authoring matches the eficiency of AR and does not add more complexity. To address this gap, we implemented a system that permits authoring AR experiences using both VR and mobile-based AR. We conducted a comparative study between the two authoring approaches. Our findings suggest that users do not perceive significant diferences between both approaches in terms of mental and physical efort, satisfaction, and efectiveness. Moreover, users were slightly faster in completing the AR authoring task when using VR. Despite their preliminary nature, these results suggest that the VR authoring can serve as a complementary alternative to AR authoring approach. This is especially useful when employing an iterative process, wherein interacting with the real site each time a change is required could be dificult, costly and less sustainable.
The paper proceeds as follows: Section 2 reviews VR authoring for AR, Section 3 outlines system design, Section 4 presents empirical study results, and the paper concludes with discussion and future research directions.
One of the earliest proposals of immersive VR authoring for AR is CAVE-AR [
To support the study, we developed an AR authoring tool with two diferent modules (Fig. 1): SimulAR, which implements the VR authoring approach, and InSituAR, which implements the AR authoring approach.
Both tools implement a similar editing process, provide the same set of menus for the operations, access to the same virtual content repository and present the same look and feel. However, SimulAR uses immersive VR technology to create and edit the AR virtual replica that was generated through photogrammetry, 104 images of the room were shot with a reflex camera and then were processed with the Agisoft Metashape software. The user interacts with the replic using the Oculus Rift and its controllers. On the contrary, in InSituAR the editing process is done with a mobile phone, and the interaction is mediated by the device screen. In both cases the AR experience is displayed using a mobile phone with TANGO technology, which is used to recognize the environment to augment and display the virtual content at specific positions.
Firstly, the user browses a Virtual Object Gallery to select the object to be placed in the real or simulated environment. Once the object is positioned within the scene, the user can adjust its properties using the Action Menu. This menu ofers options to modify the size, orientation, and position, as well as assign predefined behaviors or remove the object from the scene. In SimulAR (Fig. 2 left), users perform this process while immersed in a virtual replica of the environment using a VR headset and controllers. They navigate the environment by walking or using joystick controls, and interact with menus and objects using a ray pointer. To adjust object orientation and size, users select the desired operation from the editing menu and manipulate the controllers accordingly by rotating or moving them closer or further away. In InSituAR (Fig. 2 right), the authoring process takes place directly at the augmentation site using a mobile phone. Users interact with menus and options by tapping on the device’s screen. To add virtual content to the scene, users focus the device’s camera at the desired location and use their fingers to place it by dragging and dropping elements from the Virtual Object Gallery Menu. Rotation and scaling of objects are performed by swiping and pinching/spreading gestures, respectively. Once the user finishes editing the scene, the design is saved in the server-broker, and it is ready to be displayed in the AR player.
The AR Player module retrieves the AR scene design from the server and displays it on a mobile device with TANGO technology. Prior to this, environment scanning is necessary using TANGO’s area learning technology. The system matches the generated ADT file with the one on the server, containing environment description and virtual content information. The motion tracking system determines the device’s position and orientation in the real world, establishing the 3D coordinate origin and quaternion identity. Using this data, it calculates the size and position for displaying augmented objects on the device screen, creating the illusion of real-world placement.
To evaluate and compare the outcomes of the VR and AR authoring approaches, we conducted a comparative empirical study. Each participant in the experiment developed an AR scene using the two approaches. We aimed at gaining insights into diferences in (1) the amount of efort required, (2) the user satisfaction and (3) the time required. 10 volunteers (3 females) with ages ranging from 20 to 37 years (24.8±5.4) were recruited. The group included people with little or no previous experience with VR (6 participants) and AR (5 participants) technologies, and some experts (2 participants). Participants utilized an Oculus Rift CV1 Head-Mounted Display (HMD) and Oculus Touch Controllers for VR authoring tasks. For AR authoring, an Asus Zenfone AR with Google Tango was employed.
The authoring activity involved two tasks: Free Design (T1) and Reproduce Design (T2). In the Free Design task participants were tasked with augmenting a room by placing nine virtual objects from a gallery of objects. The objects included home appliances, furniture, and panels such as a chair, a fire alarm button, a television, and an emergency exit panel. The objects were deliberately chosen to challenge participants to manipulate objects of varying sizes, ranging from too small to too large. Participants could determine the final position and size of each object within the room. In the Reproduce Design task the participant was instructed to place a virtual chair in the room. The goal was to align the chair’s location, size, and orientation as closely as possible to a reference picture of the chair that was previously shown.
The augmentation location was a meeting room in the university of approximately 45 square meters. The virtual replica of the room for VR authoring tasks was generated via photogrammetry, using 104 images captured with a reflex camera. The Agisoft Metashape software processed these images to create a 3D model of the room.
The workload assessment in the study employed the NASA Task Load Index (NASA TLX) questionnaire. Additionally, participants were given a questionnaire to evaluate their experience. They rated their satisfaction on a Likert scale while performing basic editing operations on virtual objects such as selecting, moving, rotating, and resizing. They also assessed the dificulty, accuracy, and ease of learning. Furthermore, an open-ended question allowed participants to provide justifications and suggestions. The completion time for the second authoring task (T2) was measured, while the time for the first task (T1) was not considered, as participants had the freedom to create their designs using existing virtual objects.
The preliminary results obtained suggests that immersive VR authoring can efectively and satisfactorily support the creation of AR experiences similar to the AR approach. The level of physical and mental efort required (Table 1) was similar for the two approaches: the diferences between the mean scores were very small for all the factors, being the highest for Physical Demand (8.5 points out of 100). The responses to the questionnarie on the experience show that the level of satisfaction was between 5 and 6.3 for the the four basic interaction tasks (Q1 to Q4), for both authoring approaches. The ratings for Dificulty to Use (Q5), Accuracy (Q6), and Ease of Learning to Use the tool (Q7) were also very similar, being the highest diference of 0.5 points for Dificulty to use. Finally, the time taken to complete the second task was smaller for the VR authoring (54.5±30.2 seconds vs. 75.1±39.6 seconds), showing a statistically significant diference between both approaches (p=0.03).
Even though the number of participants is not high enough to provide sound conclusions, these results suggest that VR technology might be a viable and complementary alternative to AR authoring tools. The results related to mental and physical efort and overall user satisfaction are particularly noteworthy because most users had limited or no prior experience with VR technology, while they extensively use mobile phones in their daily activities. It is worth mentioning here that after a short training session, participants could use the VR technology as satisfactorily as the mobile AR. The smaller time taken for designing the AR scene in VR mode might be influenced by diferent issues. Firstly, the joystick on the VR controller enables faster movement within the environment compared to physically walking in AR authoring mode. Additionally, the wider field of view provided by devices like the Oculus Rift reduces the need for users to check multiple viewpoints. Moreover, the VR pointer allows manipulation of objects from a greater distance compared to mobile AR authoring, where users may need to move closer due to the limited scope of the mobile depth camera. However, AR still provides a more realistic perspective of the final experience, for which doing part of the prototyping tasks using AR would be still required to adjust the scene to the real conditions.
It is important to note here that we do not imply that VR authoring is a better approach. Indeed, one major drawback when compared to AR authoring is the requirement of creating a virtual model of the environment to augment, which is unnecessary in AR authoring. Additionally, VR and AR authoring are not mutually exclusive. Both approaches can be used together, for instance, the bulk of the AR scene can be created in VR mode to save time when placing and manipulating several objects, followed by refining the final experience in AR mode. As far as the physical space can be simulated realistically, this mixed approach could make AR prototyping more sustainable as a great part of the work can be done in the design physical space, supporting rapid prototyping cycles and postponing the last adjustments to the real space till the end of the process. Moreover, new immersive devices like the Meta Quest 3 might simplify this approach by enabling interaction in mixed environments, and facilitating the creation of the virtual replica mesh through their scanning features.
The main contribution of this study is a comparison between two approaches for creating AR experiences: VR and mobile-based AR authoring. The initial findings indicate that VR authoring could be a viable and complementary alternative option to mobile AR authoring, as users did not perceive it as more demanding in terms of efort and reported similar levels of satisfaction. When prototyping complex AR experiences in non-accessible physical sites, recreating the physical space in VR might be a sustainable and valid way for rapid prototyping. Our current work aims to define a hybrid method for prototyping AR experiences. This method will leverage the best features of both authoring approaches at diferent stages of the authoring process. Additionally, it will support the combination of these techniques to facilitate the collaborative creation of AR scenes, both synchronously and asynchronously.
This work is supported by the Spanish State Research Agency (AEI) under grant Sense2MakeSense (PID2019-109388GB-I00).