In the Industry 4.0 environments, machines are becoming increasingly more complex, and tasks like machine configuration and maintenance, demand higher human expertise, thus operators have to be able to deal with such complex systems. In this paper we introduce an AR nature inspired metaphor - BioIoT- to communicate information about complex, real-time processes in an engaging, interactive and apparent manner. In addition, we present an Industry 4.0 use case scenario where the BioIoT concept can be applied. We emphasize the contributions of such concept in the future manufacturing industrial workplaces.
• Human computer interaction → Interaction paradigms; Mixed / augmented reality • Interaction design → Interaction design process and methods 1
The Industry 4.0 concept describes the factory of the future, where information and communication technology and automation technology are fully integrated. Technological challenges that arise upon the implementation of this concept are being steadily addressed, and many industrial organizations are exceptional examples of the factory of the future.
Industry 4.0 also creates challenges related to the workforce and the human-machine interaction paradigm. In this paper we identify some of these challenges (e.g., large amounts of data, automatic decision systems, training, education and qualification) and propose an innovative, nature-inspired metaphor BioIoT to communicate information about complex, real-time processes in an engaging, interactive and easily comprehensible manner. The BioIoT representation is aesthetically pleasing and increases the overall well-being of the worker.
In this paper, we present a use case scenario showing the BioIoT metaphor as a part of daily life of a maintenance technician. We later discuss the contributions of this concept to the future manufacturing industrial workplaces. 2
Industry 4.0 marks the fourth industrial revolution by focusing on automation and data exchange in manufacturing technologies. Technological advances are driving dramatic increases in industrial productivity, enabling more interoperability and flexible industrial processes.
Examples of technologies that transform the industrial productions are: cyber-physical systems and advanced robotics, the Internet of Things (IoT), Big Data and analytics, artificial intelligence and machine learning, cloud computing, simulation, and augmented reality (AR). 2.1
The above-mentioned technological advancements have a profound impact on the manufacturing workforce. The qualifications and skills required for working in manufacturing and production are changing, and companies need to address these new challenges.
To that end, three approaches can be used: 1) increase training
efficiency, e.g., include virtual and augmented reality; 2)
increasing the intrinsic motivation and promoting creativity, e.g.,
implementing gamification concepts; and 3) increasing the
extrinsic motivation through individual incentive systems, e.g.,
providing individual feedback mechanisms [
There is a shift from a blue-collar workforce to a highly
qualified white-collar workforce. The physical work component
of the manufacturing jobs is decreasing over time due to the
introduction of automation systems. These type of jobs will be
displaced in the near future. Industry 4.0 workers are becoming
knowledge workers [
Industry 4.0 workers monitor the production and facility equipment, are integrated in decision-making process, participate in engineering activities, analyze problems, and find solutions quickly, plan for efficiency and reliability, and use a variety of systems.
To perform these duties and especially to monitor and control
the manufacturing system, the worker depends on easily
comprehensible visualizations of real-time data from a multitude
of data sources [
The worker can be stationed in a control room or can work on the plant floor in close connection with the equipment. Typically, the workers face long shifts (e.g., 12 hours), and have to deal with fatigue, distractions, and stressful work situations (e.g., alarm management). 2.2
Humans and machines are working side by side in the new
factories, complementing each other’s actions or even working
collaboratively. The interaction between workers and machines in
industrial environments creates special challenges in terms of
effectiveness (i.e., the worker is required to perform his work
effectively in the industrial environment) and acceptability (i.e.,
only a good user experience leads to the acceptance of the systems
in the workplace) [
Interaction with industrial production systems mostly occurs through complex graphical user interfaces (e.g., touchscreens integrated in the machines and desktop/mobile systems in control rooms) that over the years replaced gauges, buttons, and valves.
When designing an industrial user interface, the following
challenges have to be considered [
C1. Industry 4.0 systems produce and collect large amounts of data with various levels of abstraction, which needs to be delivered in such a way that the worker can understand it and use it to perform his tasks.
C2. Industry 4.0 systems support decision making by analyzing data, providing solutions and explaining reasoning.
C3. Adopting advance interface technologies creates constrains in the industrial environment particularities (e.g., voice command systems are not adequate for noise environments, touchscreens cannot be used when the worker is required to wear heavy gloves). In addition, user expectations and diversity have to be considered (e.g., older workers may not accept AR head mounted devices).
C4. Collaborative systems that team up workers and machines.
C5. Workplace and labor conditions changes.
C6. Since training and continuous professional development of the workforce is time consuming and economically demanding, learnability factors must be considered.
C7. Other usability issues and more expansive concept of user experience (e.g., the users’ emotional response).
These challenges prompt the development of more complex,
flexible and personalized systems. However, a key challenge in
designing user interfaces for industrial environments is to reduce
the perceived system complexity from the user perspective [
We propose an AR nature-inspired metaphor, BioIoT, to communicate, engage, and improve the industrial workers’ wellbeing.
The BioIoT concept connects each machine is connected with a virtual tree, creating a virtual forest on the factory floor, as depicted in Figure 1.
Each virtual tree serves as a living proxy of a specific machine. The machine’s general status is encoded in the features of the virtual tree (e.g., the temperature is encoded in the leaves’ color, which will turn red in case of abnormal values). Figure 2 shows an example of how data coming from five sensors is encoded in the color of the foliage, the density of the foliage, the presence and size of flowers, the highlight, and absence of leaves.
The BioIoT concept is based on the premise that if machines are perceived to be more like living beings, workers will maintain the machines’ condition better. Our assumption is supported by the biophilia hypothesis, which states that humans feel attracted to
nature and all living processes [
Steve is a production maintenance technician at a big manufacturing company. He is responsible for maintaining all production and facility equipment (e.g., production machines, conveyors, robotic automation). In the course of his work, Steve performs diverse duties: installing and (re)configuring equipment; performing preventive maintenance; and diagnosing and solving problems (e.g., repairs, replace parts).
Steve executes the company’s maintenance program daily, performing the scheduled maintenance activities. The maintenance program was designed to guarantee that maintenance is completed in an effective and efficient manner. Nevertheless, Steve has to constantly monitor the equipment to identify nonscheduled/emergency activities.
All the production maintenance technicians wear a headmounted display (e.g., Microsoft Hololens, Daqri Smart Helmet), running an AR performance support system that augments the real-world plant floor.
The performance support system provides many useful features, such as guided work instructions while servicing a machine; easy work history log registry; and opportunity to consult experts to solve a specific problem.
Using this system, Steve can access in-context information about the production and facility equipment, acquiring relevant data for his duties. Up to this point, the information is shown in dashboards.
However, in the new version of system the information has been encoded using the nature metaphor BioIoT, which conveys the general state of each equipment via a virtual tree. When Steve walks through the plant floor, he also enjoys a walk in a forest.
Initially, Steve was asked to personalize the BioIoT encoding and define the equipment or clusters of equipment connected with each tree. Steve was very happy to customize the entire metaphor.
Just now, Steve detects that one of the trees is losing all the leaves and looks distressed. He immediately thinks “Poor tree! Something is wrong with the fluids levels! Let me get the tool kit...” and decides to check the equipment. Steve stops next to the equipment and the system shows the detailed information dashboard. Steve confirms that the cooling fluid level is decreasing rapidly, and begins to work.
Soon Steve solves the problem and the tree returns to normal “Great! The tree looks wonderful now!” 4
The use case scenario describes Steve’s improved work environment, which not only facilitates his job by communicating real-time sensory information, but also can serve as a motivation factor to improve performance. In addition, the wellbeing of the worker is taken in account.
We believe that the BioIoT concept can be used to redesign industrial workplaces and address several HCI challenges in Industry 4.0.
The BioIoT concept address challenges C1 and C2 by communicate large amounts of real-time sensory information with a nature inspired metaphor. Clustering large amount of data in a single visualization (e.g. one machine or set of machines are represented with a virtual tree) makes it possible for the worker to infer the general status of the machine and only assess the detailed information when necessary.
We follow Weiser’s vision for calm technology [
Two studies explored communicating sensor information via
the BioIoT nature-inspired metaphor. The first study showed that
the participants correctly interpreted the general status of an IoT
coffee machine from the visual encoding [
Challenges C3, C4, and C6 are addressed in the BioIoT
concept through AR technology and a possible customization of
the nature metaphor. AR technology proved to be very successful
in industrial environments with regard to communicating
information and supporting and enhancing on-the-job training
(e.g., facilitating task comprehension and execution). AR can also
be used to provide cognitive support to complex or critical tasks
(e.g., machine (re)configuration and
providing the appropriate information in the given context.
and economics to design and develop new technologies that foster
psychological wellbeing and human potential [
Positive computing proposes to use psychological principles of
motivation, engagement, relatedness, autonomy, competence, and
Cognitive studies show that AR benefit manufacturing and
compassion to design technology that enhances the human life.
maintenance tasks in the areas of: information access, reduced
error likelihood, enhanced motivation, and concurrent training and
performance [
Challenge C4, C5, and C6 concern the relationship of the worker with the equipment and the work environment.
Research has been conducted to understand how people interact with machines (e.g., robots).
The perception of a
machine’s
intelligence
and
consciousness
combined
with
anthropomorphic factors (appearance, gestures and emotions) can
change the dynamics of human-machine interactions. Since
machines tend to be seen as social actors [
By introducing this nature-inspired metaphor, we hope that the workers perceive a machine as a living being and, by doing so, learn to take better care of it. The act of caring for the machine rewards the users with an aesthetical and pleasant tree, which enriches their environment.
perceived as pleasant.
Compared
to
other
metaphor
classic
was
state
representations, most users reported to feel more inclined to
perform machine maintenance or take care of the machine [
Challenge C7 refers to the user experience. With that regard, the BioIoT concept contributes largely to improving the user experience and wellbeing.
Research confirms human preference for natural environments
and shows that our wellbeing, productivity, and creativity improve
greatly by being in direct contact with nature, e.g., gardens, parks,
nature window view [
The positive effects of biophilic design extend beyond the
workers’ wellbeing and job satisfaction and can be translated into
economic advantages for organizations across sectors. The
workers’ productivity increases upon contact with nature. Direct
productivity metrics, such as the number of pieces produced and
time to task completion, can easily be measured and accounted.
The indirect productivity metrics (e.g., like illness, absenteeism,
staff retention, learning rates, stress levels and fatigue) show
remarkable gains, upon which organizations can capitalize [
We surmise that the BioIoT metaphor positively reinforces the
target behavior (e.g. perform maintenance) via a digital reward
(e.g. beautiful tree), which motivates the user to establish or
strengthen the desired behavior. The first impression of study
participants confirms this assumption [
Technology should be designed to positively affect the humans’ wellbeing and not only to impact productivity. Positive computing is a research area and practice that uses wellestablished methods in such fields as psychology, neuroscience,
This work is funded by the LiTech K-project, and by KnowCenter GmbH. Both funded within the Austrian COMET Program – Competence Centers for Excellent Technologies – under the auspices of the Austrian Federal Ministry of Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth and by the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.