What If Factories Looked Like Forests? Redesigning the Manufacturing Industry 4.0 Workplaces with an Augmented Reality Inspired Nature Metaphor Carla Barreiros Viktoria Pammer Eduardo Veas Graz University of Technology, Know Center, GmbH, Know Center, GmbH, Interactive Systems and Data Science Ubiquitous Personal Computing Knowledge Visualization Austria Austria Austria cbarreiros@know-center.at vpammer@know-center.at eveas@know-center.at Ulf Oberbichler Alphagate GmbH Austria ulf.oberbichler@alphagate.eu ABSTRACT In the Industry 4.0 environments, machines are becoming 1 INTRODUCTION increasingly more complex, and tasks like machine configuration The Industry 4.0 concept describes the factory of the future, and maintenance, demand higher human expertise, thus operators where information and communication technology and have to be able to deal with such complex systems. In this paper automation technology are fully integrated. Technological we introduce an AR nature inspired metaphor - BioIoT- to challenges that arise upon the implementation of this concept are communicate information about complex, real-time processes in being steadily addressed, and many industrial organizations are an engaging, interactive and apparent manner. In addition, we exceptional examples of the factory of the future. present an Industry 4.0 use case scenario where the BioIoT Industry 4.0 also creates challenges related to the workforce concept can be applied. We emphasize the contributions of such and the human-machine interaction paradigm. In this paper we concept in the future manufacturing industrial workplaces. identify some of these challenges (e.g., large amounts of data, automatic decision systems, training, education and qualification) and propose an innovative, nature-inspired metaphor BioIoT to communicate information about complex, real-time processes in an engaging, interactive and easily comprehensible manner. The CCS CONCEPTS BioIoT representation is aesthetically pleasing and increases the • Human computer interaction → Interaction paradigms; overall well-being of the worker. Mixed / augmented reality • Interaction design → Interaction In this paper, we present a use case scenario showing the design process and methods BioIoT metaphor as a part of daily life of a maintenance technician. We later discuss the contributions of this concept to the future manufacturing industrial workplaces. KEYWORDS Human Computer Interaction, Industry 4.0, Biophilic Design, Augmented Reality, BioIoT Figure 1: Trees representing machine states are visualized through an augmented reality display on a factory floor. IKNOW, October 2017, Graz, Austria C. Barreiros et al. 2 INDUSTRY 4.0 CHALLENGES effectively in the industrial environment) and acceptability (i.e., only a good user experience leads to the acceptance of the systems Industry 4.0 marks the fourth industrial revolution by focusing in the workplace) [11]. on automation and data exchange in manufacturing technologies. Interaction with industrial production systems mostly occurs Technological advances are driving dramatic increases in through complex graphical user interfaces (e.g., touchscreens industrial productivity, enabling more interoperability and flexible integrated in the machines and desktop/mobile systems in control industrial processes. rooms) that over the years replaced gauges, buttons, and valves. Examples of technologies that transform the industrial When designing an industrial user interface, the following productions are: cyber-physical systems and advanced robotics, challenges have to be considered [15][11]: the Internet of Things (IoT), Big Data and analytics, artificial C1. Industry 4.0 systems produce and collect large amounts of intelligence and machine learning, cloud computing, simulation, data with various levels of abstraction, which needs to be and augmented reality (AR). delivered in such a way that the worker can understand it and use it to perform his tasks. 2.1 Challenges for the Workforce C2. Industry 4.0 systems support decision making by The above-mentioned technological advancements have a analyzing data, providing solutions and explaining profound impact on the manufacturing workforce. The reasoning. qualifications and skills required for working in manufacturing C3. Adopting advance interface technologies creates and production are changing, and companies need to address these constrains in the industrial environment particularities new challenges. (e.g., voice command systems are not adequate for noise To that end, three approaches can be used: 1) increase training environments, touchscreens cannot be used when the efficiency, e.g., include virtual and augmented reality; 2) worker is required to wear heavy gloves). In addition, increasing the intrinsic motivation and promoting creativity, e.g., user expectations and diversity have to be considered implementing gamification concepts; and 3) increasing the (e.g., older workers may not accept AR head mounted extrinsic motivation through individual incentive systems, e.g., devices). providing individual feedback mechanisms [10]. C4. Collaborative systems that team up workers and There is a shift from a blue-collar workforce to a highly machines. qualified white-collar workforce. The physical work component C5. Workplace and labor conditions changes. of the manufacturing jobs is decreasing over time due to the C6. Since training and continuous professional development introduction of automation systems. These type of jobs will be of the workforce is time consuming and economically displaced in the near future. Industry 4.0 workers are becoming demanding, learnability factors must be considered. knowledge workers [10]. Knowledge work is understood as a C7. Other usability issues and more expansive concept of user dimension of the actual work, which comprises the creation, experience (e.g., the users’ emotional response). application, transmission, and acquisition of knowledge [9]. The These challenges prompt the development of more complex, responsibility of industrial workers has increased dramatically. flexible and personalized systems. However, a key challenge in Industry 4.0 workers monitor the production and facility designing user interfaces for industrial environments is to reduce equipment, are integrated in decision-making process, participate the perceived system complexity from the user perspective [14]. in engineering activities, analyze problems, and find solutions quickly, plan for efficiency and reliability, and use a variety of 3 BioIoT - A NATURE INSPIRED METAPHOR systems. To perform these duties and especially to monitor and control We propose an AR nature-inspired metaphor, BioIoT, to the manufacturing system, the worker depends on easily communicate, engage, and improve the industrial workers’ comprehensible visualizations of real-time data from a multitude wellbeing. of data sources [16]. The BioIoT concept connects each machine is connected with The worker can be stationed in a control room or can work on a virtual tree, creating a virtual forest on the factory floor, as the plant floor in close connection with the equipment. Typically, depicted in Figure 1. the workers face long shifts (e.g., 12 hours), and have to deal with Each virtual tree serves as a living proxy of a specific machine. fatigue, distractions, and stressful work situations (e.g., alarm The machine’s general status is encoded in the features of the management). virtual tree (e.g., the temperature is encoded in the leaves’ color, which will turn red in case of abnormal values). Figure 2 shows 2.2 Challenges for Human-Machine Interaction an example of how data coming from five sensors is encoded in the color of the foliage, the density of the foliage, the presence Humans and machines are working side by side in the new and size of flowers, the highlight, and absence of leaves. factories, complementing each other’s actions or even working The BioIoT concept is based on the premise that if machines collaboratively. The interaction between workers and machines in are perceived to be more like living beings, workers will maintain industrial environments creates special challenges in terms of the machines’ condition better. Our assumption is supported by effectiveness (i.e., the worker is required to perform his work the biophilia hypothesis, which states that humans feel attracted to 2 What If Factories Looked Like Forest? IKNOW, October 2017, Graz, Austria nature and all living processes [1]. Moreover, the biophilic design Using this system, Steve can access in-context information [2] is a method to design work and living spaces in such a way about the production and facility equipment, acquiring relevant that the workers’ productivity and well-being improve from data for his duties. Up to this point, the information is shown in contact with nature (or nature-like) elements [3]. dashboards. However, in the new version of system the information has been encoded using the nature metaphor BioIoT, which conveys the general state of each equipment via a virtual tree. When Steve walks through the plant floor, he also enjoys a walk in a forest. Initially, Steve was asked to personalize the BioIoT encoding and define the equipment or clusters of equipment connected with each tree. Steve was very happy to customize the entire metaphor. Just now, Steve detects that one of the trees is losing all the leaves and looks distressed. He immediately thinks “Poor tree! Something is wrong with the fluids levels! Let me get the tool kit...” and decides to check the equipment. Steve stops next to the equipment and the system shows the detailed information dashboard. Steve confirms that the cooling fluid level is decreasing rapidly, and begins to work. Soon Steve solves the problem and the tree returns to normal “Great! The tree looks wonderful now!” 4 DISCUSSION The use case scenario describes Steve’s improved work environment, which not only facilitates his job by communicating Figure 2: BioIoT visual language example: A) Foliage hue real-time sensory information, but also can serve as a motivation encodes the temperature; B) Foliage density encodes the water factor to improve performance. In addition, the wellbeing of the level; C) Absence of flowers indicates maintenance is immediately required. The size of flowers prompts when the worker is taken in account. next maintenance should be; D) Highlight animation shows We believe that the BioIoT concept can be used to redesign the presence of residues; E) Dead tree indicates a machine industrial workplaces and address several HCI challenges in critical error. Industry 4.0. The BioIoT concept address challenges C1 and C2 by communicate large amounts of real-time sensory information with 3 USE CASE SCENARIO – A DAY IN STEVE’S a nature inspired metaphor. Clustering large amount of data in a LIFE single visualization (e.g. one machine or set of machines are Steve is a production maintenance technician at a big represented with a virtual tree) makes it possible for the worker to manufacturing company. He is responsible for maintaining all infer the general status of the machine and only assess the detailed production and facility equipment (e.g., production machines, information when necessary. conveyors, robotic automation). In the course of his work, Steve We follow Weiser’s vision for calm technology [12][13], performs diverse duties: installing and (re)configuring equipment; which emphasizes calm and suggests that humans need to be performing preventive maintenance; and diagnosing and solving informed but not overloaded with information. Therefore, problems (e.g., repairs, replace parts). technology should be transparent and only request our attention Steve executes the company’s maintenance program daily, when needed. performing the scheduled maintenance activities. The Two studies explored communicating sensor information via maintenance program was designed to guarantee that maintenance the BioIoT nature-inspired metaphor. The first study showed that is completed in an effective and efficient manner. Nevertheless, the participants correctly interpreted the general status of an IoT Steve has to constantly monitor the equipment to identify non- coffee machine from the visual encoding [18], even with minimal scheduled/emergency activities. training. The second study verified that the BioIoT concept can be All the production maintenance technicians wear a head- scaled to a large number of machines (forty nine model machines) mounted display (e.g., Microsoft Hololens, Daqri Smart Helmet), and that fast-changing states encoded with the metaphor elicit running an AR performance support system that augments the preattentive response [19]. real-world plant floor. Challenges C3, C4, and C6 are addressed in the BioIoT The performance support system provides many useful concept through AR technology and a possible customization of features, such as guided work instructions while servicing a the nature metaphor. AR technology proved to be very successful machine; easy work history log registry; and opportunity to in industrial environments with regard to communicating consult experts to solve a specific problem. information and supporting and enhancing on-the-job training 3 IKNOW, October 2017, Graz, Austria C. Barreiros et al. (e.g., facilitating task comprehension and execution). AR can also and economics to design and develop new technologies that foster be used to provide cognitive support to complex or critical tasks psychological wellbeing and human potential [17]. (e.g., machine (re)configuration and maintenance) [6][7] by Positive computing proposes to use psychological principles of providing the appropriate information in the given context. motivation, engagement, relatedness, autonomy, competence, and Cognitive studies show that AR benefit manufacturing and compassion to design technology that enhances the human life. maintenance tasks in the areas of: information access, reduced error likelihood, enhanced motivation, and concurrent training and ACKNOWLEDGMENTS performance [8]. This work is funded by the LiTech K-project, and by Know- Challenge C4, C5, and C6 concern the relationship of the Center GmbH. Both funded within the Austrian COMET Program worker with the equipment and the work environment. – Competence Centers for Excellent Technologies – under the Research has been conducted to understand how people auspices of the Austrian Federal Ministry of Transport, Innovation interact with machines (e.g., robots). The perception of a and Technology, the Austrian Federal Ministry of Economy, machine’s intelligence and consciousness combined with Family and Youth and by the State of Styria. COMET is managed anthropomorphic factors (appearance, gestures and emotions) can by the Austrian Research Promotion Agency FFG. change the dynamics of human-machine interactions. Since machines tend to be seen as social actors [6], social rules and REFERENCES dynamics should be applied to designing systems that can change [1] E. Wilson. Biophilia, 12th edition. Harvard University Press. 1984. the user’s behavior [20] [6]. [2] The global impact of biophilic design in the workplace global report. Technical Report, Human Spaces, 2015. By introducing this nature-inspired metaphor, we hope that the [3] E. Largo-Wight, W. Chen, V. Dodd, and R. Weiler. Heathy Workplaces: The workers perceive a machine as a living being and, by doing so, effects of Nature contact at work on employee stress and health. Public Health learn to take better care of it. The act of caring for the machine Report 126 (Suppl 1), US, 124–130, 2011. [4] The Economics of Biophilia. Technical Report, Terrapin Bright Green, 2012. rewards the users with an aesthetical and pleasant tree, which [5] N. Miller, D. Pogue, Q. D. Gough, and S. M. Davis. Green Buildings and enriches their environment. Productivity, Journal of Sustainable Real State. Vol.1. No. 1. 2009. [6] C. Nass, and C. Yen. The man who lied to his laptop – What machines teach us The interaction with the BioIoT nature metaphor was about human relationships, 2010. perceived as pleasant. Compared to other classic state [7] H. Regenbrecht, G. Baratoff, and W. Wilke. Augmented Reality projects in the representations, most users reported to feel more inclined to automotive and aerospace industries. IEEE Computer Graphics and Applications. Vol. 25 – 6, Nov-Dec 2005. perform machine maintenance or take care of the machine [18]. [8] U. Neumann, and A. Majoros. Cognitive, performance, and systems issues for Challenge C7 refers to the user experience. With that regard, augmented reality applications in manufacturing and maintenance. Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998 the BioIoT concept contributes largely to improving the user [9] E. K. Kelloway and J. Barling. Knowledge work as organizational behavior. experience and wellbeing. International Journal of Management Reviews, Vol 2 – 3, pp. 287-304. 2000. Research confirms human preference for natural environments [10] T. Stock and G. Seliger. Opportunities of Sustainable Manufacturing in Industry 4.0. Procedia CIRP 40, pp. 536 – 54, 2016. and shows that our wellbeing, productivity, and creativity improve [11] Paelke V., Röcker C. (2015) User Interfaces for Cyber-Physical Systems: greatly by being in direct contact with nature, e.g., gardens, parks, Challenges and Possible Approaches. In: Marcus A. (eds) Design, User Experience, and Usability: Design Discourse. DUXU 2015. Lecture Notes in nature window view [2][3]. Contact with nature analogues (e.g., Computer Science, vol 9186. Springer, Cham nature-resembling colors, patterns and materials, art, natural-like [12] M. Weiser, J. Brown. 1996. The coming age of calm technology. Xerox PARC. [13] M. Weiser. 1991. The computer for the 21st century. Scientific American, 94- light, and sounds) is less effective but proved to be beneficial. 104. The positive effects of biophilic design extend beyond the [14] Ziefle, M., Röcker, C. (eds.): Human-Centered Design of E-Health workers’ wellbeing and job satisfaction and can be translated into Technologies: Concepts, Methods and Applications. IGI Publishing, Niagara Falls (2011) economic advantages for organizations across sectors. The [15] T. Pfeiffer, J. Hellmers,E. Schön, and J.Thomaschewski. Empowering User workers’ productivity increases upon contact with nature. Direct Interfaces for Industrie 4.0. Proceedings of the IEEE ( Vol. 104, Issue: 5, 2016 ) [16] D. Gorecky, M. Schmitt, M. Loskyll, and D. Zühlke. Human-machine- productivity metrics, such as the number of pieces produced and interaction in the industry 4.0 era. In Proceedings 12th IEEE International time to task completion, can easily be measured and accounted. Conference on Industrial Informatics (INDIN), 2014. The indirect productivity metrics (e.g., like illness, absenteeism, [17] R. A. Calvo, and D. PetersPositive. Computing – Technology for Wellbeing and Human Potential. MIT Press staff retention, learning rates, stress levels and fatigue) show [18] C. Barreiros, E. Veas, and V. Pammer. BioIoT: Communicating Sensroy remarkable gains, upon which organizations can capitalize [4][5]. Information of a Coffee Machine Using a Nature Metaphor. CHI EA’17 Proceeding CHI Conference Extended Abstracts on Human Factors in We surmise that the BioIoT metaphor positively reinforces the Computing Systems, pp. 2388-2394. 2017. target behavior (e.g. perform maintenance) via a digital reward [19] C. Barreiros, E. Veas, and V. Pammer. Pre-attentive Features in Natural Augmented Reality Visualizations. IEEE International Symposium on Mixed (e.g. beautiful tree), which motivates the user to establish or and Augmented Reality (ISMAR-Adjunct), 2016. strengthen the desired behavior. The first impression of study [20] BJ Fogg, C. Nass. How users reciprocate to computers: An experiment that participants confirms this assumption [18]. demonstrates behavior change. CHI 97 Electronic Publications, 1997. Technology should be designed to positively affect the humans’ wellbeing and not only to impact productivity. Positive computing is a research area and practice that uses well- established methods in such fields as psychology, neuroscience, 4