=Paper=
{{Paper
|id=Vol-1963/paper473
|storemode=property
|title=HoloMiracle: Intuitive In-Situ Querying for Industrial Environments
|pdfUrl=https://ceur-ws.org/Vol-1963/paper473.pdf
|volume=Vol-1963
|authors=Simon Mayer,Jack Hodges,Dan Yu,Konrad Diwold
|dblpUrl=https://dblp.org/rec/conf/semweb/MayerHYD17
}}
==HoloMiracle: Intuitive In-Situ Querying for Industrial Environments==
https://ceur-ws.org/Vol-1963/paper473.pdf
HoloMiracle: Intuitive In-Situ Querying
for Industrial Environments
Simon Mayer, Dan Yu, Jack Hodges, and Konrad Diwold
Siemens Corporate Technology
simonmayer@siemens.com
Abstract. We present HoloMiracle, a system that enables operators of
industrial equipment and beyond to pose queries about physical, vir-
tual, regulatory, and functional relationships between components of the
equipment and that visualizes the responses to their queries in-situ, as a
holographic overlay. We report on HoloMiracle’s system architecture and
discuss a concrete use case in the automotive manufacturing domain.
Keywords: Semantics, Visualization, Hologram, Situational, Industry
1 Introduction and Background
Devices and systems across industrial domains – from manufacturing lines and
turbines to transformers and buildings – have virtual and physical components
that interact with each other in a plethora of different ways: physically, compo-
nents might be connected to one another structurally which might imply that
phenomena such as vibrations are transmitted across components of a device;
virtually, different components might be associated to a common process – con-
sequently, reconfigurations of one station in a manufacturing line might affect
others; there might also be regulatory connections between components, e.g.,
between all zones in a building that must be covered by smoke detectors, and
components might be functionally similar, meaning that one might be used in
place of another. Currently, operators access such information about industrial
equipment through various software applications, typically on a desktop or lap-
top computer, or in the form of physical documentation. The information re-
trieval process from these sources can take a considerable amount of time and
is prone to errors, as their information back ends are often not integrated.
2 HoloMiracle: System Architecture and Demonstrator
We present HoloMiracle, a solution that relies on semantic technologies and
mixed-reality visualization to enable operators to pose non-trivial queries about
relationships between components of a device or system and visualize query re-
sponses in-situ. HoloMiracle combines information inside knowledge models that
it accesses through the Open Semantic Framework [1] with the ability to query
these models verbally and visualizes the query responses on a Microsoft HoloLens
(a) ”Proximity Sensors” (b) ”Connected Components”
Fig. 1. Interacting with a model of a manufacturing line using HoloMiracle.
device. This enables the system to render responses to complex queries directly
on top of a physical device or system and makes the wealth of information that
is stored in complex, potentially cross-domain knowledge models, accessible to
average users in an intuitive and efficient way.
HoloMiracle was deployed in the context of interacting with a model of an au-
tomotive manufacturing line that mirrors a line deployed by Miracle Automation
Engineering Co., Ltd., a leading manufacturer of logistics automation technol-
ogy and equipment. We use this specific example to illustrate the functionality
of our system, although the approach described is in principle applicable to any
complex system of any granularity. In this demonstration, HoloMiracle produces
a visual rendering of the components of the manufacturing line from a semantic
model that includes the line’s components (e.g., skid rail, skid, motors, wheels,
sensors, etc.), their spatial dimensions, behavioral and functional properties, and
connectivity. An operator uses a cursor and a speech interface to interact with
the assembly line and pose queries that are answered by the underlying seman-
tic model and can be as complex as the model is detailed. Examples of such
queries are ”Show all proximity sensors” – the system highlights all compo-
nents of type ProximitySensor (yellow), see Fig. 1(a) – and ”Show all directly
connected components” – the system highlights all components (yellow) that are
directly connected to a component selected using the cursor (blue), see Fig. 1(b).
The system can also answer deeper queries such as ”Show all sensors that can
measure vibration of the selected component” by highlighting those acceleration
sensors that are suitable for measuring the vibration of a specific component.
We expect that HoloMiracle should contribute to making the interaction
with and exploration of a system more natural and efficient for humans, as it
enables the in-situ querying of equipment and rendering of information for ”on-
the-spot” decisions. In addition to shortening the time spent on asset surveys,
assessment, and analytics, and enabling site engineers to gain more insight in
the target system, it should also prove valuable for recording service processes
to identify potential for future improvements.
References
1. Mayer, S., Hodges, J., Yu, D., Kritzler, M., Michahelles, F.: An Open Semantic
Framework for the Industrial Internet of Things. IEEE Intelligent Systems 32(1)
(2017) 96–101