=Paper=
{{Paper
|id=Vol-3618/pd_paper_5
|storemode=property
|title=Towards augmented reality applications for it maintenance tasks based on ArchiMate models (short paper)
|pdfUrl=https://ceur-ws.org/Vol-3618/pd_paper_5.pdf
|volume=Vol-3618
|authors=Sophie Crevoiserat,Fabian Muff,Hans-Georg Fill
|dblpUrl=https://dblp.org/rec/conf/er/CrevoiseratMF23
}}
==Towards augmented reality applications for it maintenance tasks based on ArchiMate models (short paper)==
https://ceur-ws.org/Vol-3618/pd_paper_5.pdf
Towards augmented reality applications for it
maintenance tasks based on ArchiMate models
Sophie Crevoiserat1 , Fabian Muff1,* and Hans-Georg Fill1
1
University of Fribourg, Digitalization and Information Systems Group, Boulevard de Pérolles 90, CH-1700 Fribourg
Abstract
Augmented reality permits to embed virtual objects in the real environment to enhance the perception of
users. In this paper, we describe an approach for embedding conceptual models using augmented reality
in IT maintenance scenarios. It is based on the ArchiMate modeling language that has been extended
with the goal of bridging the gap to models for physical environments. This allows, for example, to guide
users in IT maintenance tasks by displaying necessary information originating from the models in the
real world. The approach has been implemented on a novel metamodeling platform, which natively
supports augmented reality scenarios.
Keywords
Augmented Reality, Physical Modeling, ArchiMate, Maintenance, IT Infrastructure
1. Motivation
Augmented reality (AR) is used today in many domains, ranging from medicine, video games,
education, and many others [1, 2]. Recent use cases include personal information systems,
industrial, military and medical applications, AR for entertainment, or AR for personal workplace
areas [3]. In AR, electronically generated information is superimposed onto objects in the real
world using devices such as head-mounted displays, smartphones, or tablets. The use of
conceptual modeling for augmented reality has so far been investigated in various domains [4].
This includes for example maintenance tasks or training, e.g., to create augmented reality
applications using model-driven engineering, or to fuel knowledge from conceptual models into
AR environments, e.g., by using ontologies for reasoning about the perceived environment [5].
Although it has also been explored how AR can support physical IT operation and maintenance
tasks – such as removing cables or other hardware components [6, 7] – a mapping to the
domain of enterprise architecture (EA) is missing. Thereby, enterprise architecture stands for
the model-based representation of the interplay of an enterprise’s organizational structure,
processes, information systems, and infrastructure [8]. EA is widely used today in organizations
to support decision makers in business-IT alignment projects. Typical use cases include the
design and management of an organization’s IT landscape, for business continuity planning to
ensure resilience in the event of outages or failures in IT systems, or for migration planning for
introducing new IT applications or components in an enterprise.
According to a survey by Roth et al. [9], information about enterprise architectures has
ER2023: Companion Proceedings of the 42nd International Conference on Conceptual Modeling: ER Forum, 7th SCME,
Project Exhibitions, Posters and Demos, and Doctoral Consortium, November 06-09, 2023, Lisbon, Portugal
*
Corresponding author.
$ sophie.crevoiserat@unifr.ch (S. Crevoiserat); fabian.muff@unifr.ch (F. Muff); hans-georg.fill@unifr.ch (H. Fill)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
been represented in the past primarily using text, graphs, charts, as well as 2D and 3D models.
This includes also the representation using augmented or virtual reality devices for displaying
conceptual enterprise architecture models in 3D space, e.g., to support decision making [10].
However, what is missing so far is a link between the high-level view of an enterprise
architecture for supporting business operations via IT, i.e., the conceptual models, and the
low-level view of physical IT hardware components. Integrating these two perspectives would
enable maintenance tasks at the physical layer to be supported by knowledge from higher layers.
This could be useful, for example, to trace problems in IT applications back to the physical
environment, such as an incorrectly connected cable. Such a link could be beneficial not only
for professional IT operations, but also for consumer use cases, for example, when having to
connect devices in smart home scenarios.
In the following, we propose a first approach to bridge enterprise architecture models on the
technology layer using ArchiMate notation with models of physical IT hardware components,
thereby adding details to the enterprise architecture information to fuel it into real-time aug-
mented reality supported work scenarios. This is achieved through an intermediary layer that
acts as a facilitator for transitioning from the high-level view used in enterprise architectures to
the level of physical components.
2. Model-based AR for it maintenance tasks
As shown in Figure 1, our approach is based on three layers. At the top layer, traditional
enterprise architecture models in ArchiMate notation are used. These may be connected to
other ArchiMate models, e.g., for specifying IT services on the application layer, which in turn
support particular business processes on the business layer. On the intermediate model layer, a
first transition towards physical models is described.
The information from the enterprise architecture needs to be mapped to a conceptual repre-
Enterprise Architecture Model Layer (Technology)
Intermediate Model Layer (Wiring)
Physical Model Layer (Hardware)
Figure 1: Layers for transitioning from Enterprise Architecture models using ArchiMate technology
concepts to Physical models via an Intermediate layer
sentation that focuses on abstract physical devices and their connections. Thus, for example,
the concept of a communication network on the EA level needs to be translated into network
components and cables for connecting them. Finally, at the bottom layer, a model of the actual
physical devices is shown. Here, the concrete devices with the detailed positioning of their
ports are shown. Depending on the goals of the intended AR application, further details on
coordinates or markers may be added on this layer to permit the positioning of virtual objects
in the AR environment.
Figure 2 shows an excerpt of the integrated metamodel for our approach. It features three
different types of diagrams corresponding to the three model layers, cf. Figure 1, classes, and
relationclasses, as well as references between the layers. Due to limitations of space only a
subset of concepts is shown here. As an example of references between layers, traces between
the Device class on the enterprise architecture model layer, on the intermediate model layer
and the concrete manifestations on the physical model layer are depicted. This highlights how
high-level concepts from the EA perspective need to be mapped to concrete physical devices on
the bottom layer.
In the example, the device needs to be specialized into either a PC, Switch, Modem, or Wall
concept on the physical layer. Thereby, the Wall concept has been introduced as an additional
concept only in the intermediate model layer, as it is necessary for the physical operation, but
has not been part of the models at the EA level.
3. Implementation and preliminary evaluation
The approach has been implemented using a new metamodeling platform based on [11], which
is currently under development, and aims to natively embed model information into augmented
C C R R
Enterprise Architecture
Model Layer
(Technology)
R
C
C
C
C
Intermediate Model
R Layer (Wiring)
C R
C C C C C
C C C
C Class
R Relationclass R R Physical Model
Layer (Hardware)
Figure 2: Excerpt of the Integrated Metamodel showing Classes and Relationclasses on the Enterprise
Architecture model, the Intermediate model, and the Physical model layer and an exemplary inter-layer
reference for Devices. The three Metamodels correspond to the three Layers introduced in Figure 1
AR-Marker
Figure 3: Resulting Augmented Reality application showing the embedding of the Physical model,
recognized AR markers for detecting the current scene and hints for the next steps to be performed.
reality environments using the W3C WebXR proposal [12]. In particular, three scene types
have been created that contain the concepts from the metamodels shown above. All classes and
relationclasses have been complemented by a corresponding graphical specification. As shown
in the exemplary prototype application, this permits to visualize for example, the information
from the physical model layer directly in an AR environment, as well information from connected
models – see Figure 3. In addition, mechanisms have been added to guide the user through
different steps by checking the properties of the models. This is depicted, e.g., in Figure 3 by the
red visualization of a connection between two devices.
Based on the implementation of the metamodels and a separate rudimentary prototypical
application1 , it was possible to evaluate the approach in four usage scenarios. These included
scenarios such as fixing a broken Ethernet cable, the interruption of a network connection
to a network attached storage (NAS) due to a malfunctioning Ethernet port, an unplugged
coaxial cable that caused a missing internet connection, and a broken LAN port on a desktop PC.
All scenarios were first modeled in ArchiMate notation to represent the general architecture
components and their conceptual linkages. Subsequently, the intermediate and physical models
were added. In addition, a BPMN diagram was elaborated to specify possible resolutions of
malfunctions that could be applied to all scenarios. With the help of the prototypical AR
application, the models could be displayed on a WebXR compatible tablet and embedded directly
in the user’s view based on recognized markers, which acted as surrogates for a full object
detection.
4. Conclusion and outlook
In this paper, we could briefly describe our vision towards model-based augmented reality
applications using information from enterprise architecture models. In particular, we highlighted
how an intermediate layer may help to transition to actual physical models that depict how the
information from high-level enterprise architectures maps to the real world. In the current state
of the modeling platform, the processing of multiple models is not yet supported. Future work
will include the integration of the processing and visualization of the created models directly in
1
Available on: https://zenodo.org/record/7889218
the modeling platform, as well as the addition of functionalities for reasoning about the current
state of the environment for displaying the necessary model information [5].
Acknowledgments
Financial support for this paper is gratefully acknowledged by the Smart Living Lab funded by
the University of Fribourg, EPFL, and HEIA-FR.
References
[1] R. T. Azuma, A survey of augmented reality, Presence: teleoperators & virtual environ-
ments 6 (1997) 355–385.
[2] J. Carmigniani, B. Furht, Augmented Reality: An Overview, 2011, pp. 3–46. doi:10.1007/
978-1-4614-0064-6_1.
[3] L. Xue, C. J. Parker, H. McCormick, A Virtual Reality and Retailing Literature Review: Cur-
rent Focus, Underlying Themes and Future Directions, Springer International Publishing,
Cham, 2019, pp. 27–41. doi:10.1007/978-3-030-06246-0_3.
[4] F. Muff, H. Fill, Past achievements and future opportunities in combining conceptual
modeling with VR/AR: A systematic derivation, in: B. Shishkov (Ed.), BMSD 2023, Utrecht,
volume 483, Springer, 2023, pp. 129–144. doi:10.1007/978-3-031-36757-1\_8.
[5] F. Muff, H. Fill, A framework for context-dependent augmented reality applications using
machine learning and ontological reasoning, in: AAAI Spring Symposium on Machine
Learning and Knowledge Engineering for Hybrid Intelligence, volume 3121, CEUR-WS,
2022. URL: https://ceur-ws.org/Vol-3121/paper11.pdf.
[6] S. , K. S. Sudeep, Management of it operations and it infrastructure with virtual reality, in:
2022 OPJU International Technology Conference on Emerging Technologies for Sustainable
Development (OTCON), 2023, pp. 1–4. doi:10.1109/OTCON56053.2023.10114008.
[7] R. Palmarini, J. A. Erkoyuncu, R. Roy, H. Torabmostaedi, A systematic review of augmented
reality applications in maintenance, Robotics and Computer-Integrated Manufacturing 49
(2018) 215–228. doi:10.1016/j.rcim.2017.06.002.
[8] M. M. Lankhorst (Ed.), Enterprise Architecture at Work - Modelling, Communication and
Analysis, Fourth Editiontion, Springer, 2017. doi:10.1007/978-3-662-53933-0.
[9] S. Roth, M. Zec, F. Matthes, Enterprise architecture visualization tool survey 2014, Technical
Report (2014). ISBN: 3844289380.
[10] K. Rehring, M. Greulich, L. Bredenfeld, F. Ahlemann, Let’s get in touch - decision making
about enterprise architecture using 3d visualization in augmented reality, Proceedings of
the 52nd Hawaii International Conference on System Sciences (2019) 1769–1778. doi:10.
24251/HICSS.2019.215.
[11] F. Muff, H.-G. Fill, Initial concepts for augmented and virtual reality-based enterprise
modeling, Proceedings of the ER Demos and Posters 2021 co-located with 40th International
Conference on Conceptual Modeling (ER 2021) 2958 (2021).
[12] B. Jones, M. Goregaokar, R. Cabanier, WebXR Device API, W3C Candidate Recommendation
Draft, work in progress, World Wide Web Consortium, 2023. URL: https://www.w3.org/
TR/2023/CRD-webxr-20230303/.