One current challenge in enterprise modeling is to establish it as a common practice in everyday work instead of its traditional role as an expert discipline. In this paper we present rst steps in this direction through augmented and virtual reality-based conceptual modeling. For this purpose we developed a novel meta-metamodeling framework for augmented and virtual reality-based conceptual modeling and implemented it in a prototypical tool. This permits us to derive further requirements for the representation and processing of enterprise models in such environments.
One vision that has recently been formulated for enterprise modeling states, that
within some years from now on, modeling shall be embedded in our daily work
practices [
In the following, rst research results in augmented and virtual reality (AR/
VR)-based conceptual modeling towards realizing this vision are presented. We
focus mainly on the presentation and representation of models, as well as on
the scope of models. This includes the analysis of everyday work practices, the
identi cation of adequate situations for model creation and use, as well as the
selection of appropriate content in particular contexts. Thereby we build upon
previous work where we derived constituents of AR-based applications [
As a sample scenario, let us imagine a domain expert working on a task using a machine in some business process. Suppose that the person would like to know about the possible next steps in the process. In a traditional setting, this person would have to revert to a classical modeling tool and be familiar with the used modeling notation. Consider now that the person wears a head-mounted display ? Copyright © 2021 for this paper by its author. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). (HMD) that automatically displays the relevant information about the process and embeds the visualization into the real world at the speci c location in the form of augmented reality. This would mean that the model is embedded into the current work practice.
When analyzing this scenario, there are many aspects that must be
considered for combining modeling and AR. In particular, we can revert to a previously
described conceptual framework for AR and denote the di erent steps of the
process as content and the working environment of the domain expert as
context [
The remainder of the paper is structured as follows: In Section 2 we brie y discuss related work. In Section 3, we will introduce the framework we developed for integrating AR/VR concepts in metamodeling. In Section 4 we present the additionally derived requirements for such an approach. The paper will end with a conclusion and an outlook to future work. 2
The representation of models in three-dimensional space has been investigated by
several authors. As summarized in [
Abdul et al. presented an approach for visualizing BPMN collaboration
models extracted from a standard le format in VR [
Ruiz-Rube et al. presented a tool that focuses on a metamodeling approach
for the creation of AR editors for domain-speci c languages (DSL) [
Metzger et al. designed and implemented a system for interacting with virtual
process models by using smart glasses [
In summary, there are several previous approaches that target the use of augmented and virtual reality in conceptual modeling. However, to the best of for and
our in a knowledge no generic way. publications so far address this topic on the meta-metalevel 3
A
For developing a meta-metamodeling framework for AR and VR, we followed
an exploratory and experimental research approach. We rst investigated
existing meta-metamodels as described for example in [
metaobject - uuid - name - description - geometry - coordinates2D - relativeCoordinates3D - absoluteCoordinates3D has 1..1 1..1 1..1
class relationclass has 0..* role_port_reference is_instance_of_relationclass is_instance_of _class 0..* 0..*
0..* 0..* class_instance 0..1
has_ports 0..* 0..* 0..* has at ribute contains_classes 0..* 0...* role_from role_to 0..1 assigned_to 0..1class_instance_ref 0..*
0..* attribute_instance relationclass_instance 0..1 has_relationclass_ reference is_instance_of_scene 1..1 0..*
0..* has_attribute attribute 0..* 0..1 0..* has_type 1..1 attribute_type 1..1 0..*
Port 1..1 0..* has_ports 1..1 1..1 has role 1..1
The innovative aspect of this meta-metamodel is, ously used for 2D and 3D modeling. Unlike previous proaches, it is however natively based on 3D space. It that it can be simultanemeta-metamodeling apmust be noted that the meta-metamodel shown in Figure 1 only contains an excerpt of the actual constructs due to limitations of space. It is composed of a meta layer and an instance layer. This is to show the relation between the de nition of a modeling language and the instantiation of the speci c objects when de ning a model. The main classes in the meta-metamodel inherit the general properties from the superclass metaobject. The core classes inheriting from metaobject are class, role, scene type, attribute and attribute type.
The core part comprises classes and relationclasses that are contained in one or multiple scene types. A scene type represents the closed 3D space of a model. Classes, relationclasses and scene types have attributes that are further detailed with exactly one attribute type. Classes, relationclasses and scene types can be set in relation to each other by relationclasses. Each relationclass has exactly two roles assigned. A from role and a to role. Further, each role has at least one Initial Concepts for AR and VR-based Enterprise Modeling reference to a class, relationclass or scene type, that de nes, to what this role can connect. Further classes and scene types can have ports. Also to these ports, roles can be assigned.
All constructs inheriting from metaobject have a visual representation. This representation is de ned with a domain-speci c language called VizRep, that de nes the 3D representation and behavior of an object. This information is stored in the geometry attribute in metaobject. Further, each visual object has 2D coordinates for the positioning in a 2D modeling environment, as well as relative 3D coordinates (relativeCoordinates3D) for positioning objects in AR and VR environments relative to the user position. These positions may di er from the coordinates used for the 2D screen representation. Further, each metaobject may have absolute 3D coordinates (absoluteCoordinates3D) for the positioning of objects using real world coordinates like GPS coordinates or indoor positioning information.
In the lower half of Figure 1 the instance layer of the meta-metamodel is depicted. It shows the constructs for holding information of the instances of the metamodel when instantiating a model, for example instances for class, attribute and scene instances.
For evaluating the technical feasibility of the meta-metamodel, a
prototypical implementation has been created using JavaScript and WebXR1 via the
ThreeJS2 library. The resulting modeling tool works entirely in a 3D
environment, and not like most other modeling tools on a 2D canvas. This has the
advantage, that the models can be used in a traditional 2D environment by
holding the depth coordinates constant, or without changes in a 3D mode for
AR and VR. An example of the browser-based modeling tool is shown in
Figure 2. Further, we conducted tests by specifying subsets of BPMN3 and ERD
[
With the insights gained above, we can formulate the following requirements
for enterprise modeling in AR/VR. First, the technology stack underlying such
modeling tools must support AR and VR, including according hardware
devices. Further, the graphical representation and positioning of objects must be
accomplished in 3D space. For the representation of 3D geometries, the already
mentioned VizRep language can be used. This is a new and generic JavaScript
function to de ne the visual representation of the di erent components, the
according labels, the used attributes for the labels etc. This enables the de
nition of 3D objects and the speci cation of according labels. This has direct
consequences for the interaction with models where novel types of user-machine
interaction need to be used, as for example discussed in detail in [
Concerning the positioning of objects, an AR/VR-enabled modeling environment permits to place objects in virtual 3D space as well as attach them to real-world coordinates, e.g. to attach a task in a process or an entity type in an ER diagram to a physical machine. Thus, this information needs to be provided in addition to the traditional 2D coordinates. This leads to new types of enterprise modeling scenarios, e.g. for using enterprise models as a guidance in the style of a map in the real-world.
Further, one of the strengths of AR devices is to analyze the environment and recognize the situation of the user through di erent sensors. For integrating this in enterprise modeling, the properties of the context need to be inferred so that the models can be adapted to a speci c situation. Again, this information about the context has to be made available for the objects in an enterprise model. As existing modeling languages do not consider such aspects, they will have to be adapted for this purpose, e.g. by a context attribute for a BPMN task. 5
In this paper, a rst design of an AR and VR enabled meta-metamodeling framework as well as a prototype were shown. With rst tests we could verify that the use of the framework and the implementation of some basic modeling languages for AR and VR is feasible. In further work we will extend the framework and the implementation. In particular, we will address interaction techniques and the positioning of models and their elements using real world coordinates, as well as the integration of situational context in AR.