AVISAR - Adaptive Visual Assistance System using Spatial Augmented Reality for Manual Workplaces in Smart Factories

Abstract

Smart factories still require manual work by humans to remedy defects in automated production lines. Given the shortage of highly skilled workers and the demand to include people with language barriers or cognitive challenges in the workforce, novel assistance systems must be introduced to the factories. We propose an Adaptive Visual Assistance systems using Spatial Augmented Reality (AVISAR), which not only adapts to diferent repair tasks and the layout of a manual workplace in the factory, but also to the individual human skills and needs. This should result in a more inclusive work environment and the well-being of a more diverse workforce. Spatial AR promises to be a more accessible approach than head-mounted displays. In this contribution, we present the conceptual framework and a first working prototype of the AVISAR system.

Keywords

• eol>spatial augmented reality
• mixed reality
• adaptivity
• human-centered computing

1. Introduction

Automation is a key driver for smart factories of the future. However, there still remain tasks for human workers when defects are detected in the automated production lines. Sustainability goals demand that defects are manually repaired in order to reduce scrap in production. This issue becomes even more severe when considering individual production processes with a batch size of 1. Given the demographic shift and the shortage of skilled workers, fewer expert workers remain for these manual tasks. It is also desirable to include people with cognitive challenges or language barriers in such processes. For quick on-boarding and ongoing guidance of such personnel, human-computer interaction can provide appropriate solutions [ 1 ]. The aim is to not only ensure operational eficacy, but also cultivate the sustainable well-being of the workforce by ofering them tailored support in smart factories of the future.

We contribute conceptual considerations for an adaptive assistance system in industrial manufacturing based on Spatial Augmented Reality (SAR) as well as a partial implementation of the system for future studies. AVISAR (Adaptive Visual Assistance Sytem using Spatial Augmented Reality) is geared towards manual workstations in smart factories and features adaptivity on three distinct levels: human needs and skills, tasks, and the layout of the workstation. SAR is a branch of Augmented Reality (AR) that does not require users to wear specific glasses or handle additional devices such as tablet computers. The physical space is augmented with virtual information, e.g. using projectors, which makes it easily accessible for a diverse range of users. In contrast to wearable devices such as head-mounted displays, SAR is less obtrusive and tiresome to use over an extended period of time.

The next section reviews adaptive assistance systems and SAR solutions from the literature. Section 3 presents our concept and implementation of AVISAR. Consequently, we report an informal evaluation and discussion of the system in Section 4.

2. Related Work

Adaptive Assistance Systems. Assistance systems in manufacturing support workers in their assembly activities on the shop floor without replacing them, overruling them, or exposing them to risks [ 2 ]. Their main purpose is to compensate deficits (e.g. due to old age, lack of skills, or disabilities) and to expand existing skills. There are sensory, physical, and cognitive assistance systems [ 3 ], using visual tools, pictograms, or projection systems (such as AVISAR). Individual use of assistance systems gains importance in human-centered work of Industry 5.0. While current systems normally ofer standardized views and functionalities, newer systems can be adapted to the experiences, cognitive abilities, and needs of groups or individuals [ 4 ].

Based on the conviction that “work should adapt to people and not people to work”, the field of human factors/ergonomics in human-computer interaction (HCI) has long investigated how adaptive user interfaces (AUI) can improve usability [ 5 ]. The field of intelligent user interfaces (IUI) leverages algorithms and machine learning to realize adaptations [ 6 ]. Personalized support and user experience of AUI are based on individual user profiles, analysis of the user style or state, work context, mental workload prediction [ 7 ], or integration of user traits from psychological theory [ 8 ]. Technological innovations such as artificial intelligence and improved sensing technologies spur this development at software and hardware level.

Adaptive interfaces are usually mentioned coincidentally with adaptable interfaces. While adaptive interfaces automatically adapt to changes in environmental conditions or user characteristics [ 9 ], adaptable interfaces require the initiative of the human user. The transition between the two concepts is fluid and intermediate stages are referred to as “levels of automation”, e.g. by [ 10, 11 ]. In the following, we present frameworks and implementations of adaptive assistance systems on the shop floor. Historically, they appear after machine-centered, flexible, and reconifgurable systems from the age of automation and mass production [ 12 ]. These new systems reflect a human-centered notion of adaptivity, where technology is used to adapt production processes to humans or consumer behavior and market dynamics on the macro level [ 13 ].

According to Peruzzini at al., adaptivity is the ability of the entire production system to adjust to new conditions, including machines, material flow, control systems, and personnel [ 14 ]. Applying human-centered and contextual adaptation rules positively afects system usability and process performance for elderly workers. Schlund and Kostolani [ 15 ] present a framework for personalized work systems that integrates cognitive features for dynamic adaptivity across the dimensions operator (human), work equipment (machine), work process, workplace, and work environment. The authors cite implementations such as an adaptive projection of work instructions and other information to illustrate their concept. The INCLUSIVE System [ 16 ] is a general framework for industrial interaction systems that adapts to the skills and capacities of human operators. The framework relies on three modules: Measure (human capabilities), Adapt (the interaction), and Teach (unskilled workers) to tailor interaction with complex manufacturing systems to individual user requirements and particularly provide guidance and training. Objective measurements and user feedback obtained in industrial use cases showed the efectiveness of the approach. Similarly, Oestreich et al. present a conceptual approach for designing adaptive assistance systems in a human-centered way and create personalized user experiences, e.g. support learning, performance, motivation, or ergonomics [ 17 ].

Petzoldt et al. [ 18 ] examine human-centered assistance systems for manual assembly, including implementation approaches. Functionalities include both product- and process-related (assembly instruction, automatic configuration and calibration, progress recognition, quality control) as well as human-centered support dimensions (individualization, qualification, motivation, ergonomics). Bertram et al. provide an overview of assistance systems supporting manual work by visualizing and supervising tasks [ 19 ]. The authors propose mandatory aspects for future intelligent manual working stations such as assisted work instructions, recognition of products and working context, and autonomous learning ability, finding that adaptable integration in production is the least considered. Finally, Mark et al. identify nine potential user groups for assistance systems in production [ 20 ]. According to the authors, projection-based cognitive assistance systems with step-by step picture or video-based task instructions and integrated quality control are especially suited for unskilled, inexperienced, mentally handicapped, migrant, or content-flexible users.

Research shows that human-centeredness in manufacturing includes both adaptations to operators and products as well as the working environment. Adaptations are achieved by means of an adaptation strategy that is guided by worker and process states, user information and interactions, and digital system information. Feedback loops may be integrated to continuously improve adaptations. A need for group-specific or personalized adaptations in order to compensate for cognitive deficits or missing skills is evident.

Spatial Augmented Reality. According to the reality-virtuality continuum [ 21 ], the area between the completely real and the completely virtual is referred to as Mixed Reality (MR). MR encompasses Augmented Virtuality (AV) and Augmented Reality (AR). SAR is a specific type of AR, which blends the real and virtual worlds in real time. The key characteristic of SAR is that no additional devices such as head-mounted-displays, data glasses, tablets, or wearables are required, since digital information is projected directly into the surroundings and onto objects [ 22 ]. This enables hands-free interaction coupled with digital augmentation, making it a promising technology for industrial assembly. Hence, all kinds of users can easily access the augmented workstation without any preparation or personal setup of additional devices. Next, we present SAR assistance systems designed for industrial use.

Zigart and Schlund tested the industrial readiness of a SAR system in the field of assembling electronic control panels with skilled workers and students [ 22 ]. Skilled workers suggested to OPERATOR static physical and mental prerequisites, personality traits, formal qualification, experience situational/dynamic cognitive load, physical strain, performance, emotional state, motivation, interaction TASK static product specfications (color, location of parts) situational/dynamic product state (error and progress), location of products WORKPLACE static workplace layout, surface characteristics situational/dynamic environmental information e.g. pre-recorded digital user profiles, retrievable via RFID tags e.g. sensor-based measurements: eye tracking, brain activity, heart rate variability, electrodermal activity, motion/gesture detection e.g. instances of digital product twins, digital work instructions e.g. smart products, camera-based image recognition, tracking e.g. digital twin of the factory layout e.g. sensory detection of lighting conditions, noise, air quality etc.

ADAPTATION TRIGGERS

EVALUATION ADAPTATION STRATEGY

ADAPTATION software-based analysis • calibration module • interaction module • content module • cognitive module • motivation module • health module • notification module (A-)Synchronous feedback loop Assessment of objective (e.g. stress, fatigue, error rate) and subjective measures of performance and well-being (e.g. usability, user experience, motivation, and health) projection overall size and layout, distortion, brightness, content, colors, shapes, size of texts and graphics, language and wording, level of detail, speed of animation, style of graphics (realistic vs. abstract icons) projector rotate, tilt, adjust height etc. use the current system for new activities and training but not for series production. According to the authors, improvements could include automatic detection for switching to the next step and adapting the instructions to the qualification level. Hornacek et al. present a SAR system that enables and simplifies the efect of a keystone correction with an adjustable mirror and a downward-facing camera [ 23 ]. Their perspective correction has been tested for planar surfaces only, but with adjustments to warping and scene geometry recovery, it can also be applied to non-planar surfaces. The dynamic projection system by Rupprecht et al. detects human poses and gestures via the YOLOv3 algorithm and projects instructions relative to the worker using pictograms, simplified images, or elementary contours [ 24 ]. A comparison of static in-view (display information somewhere in the operator’s field of view) and guided in-situ (display of information at the place of action) instructions reveals that in-situ instructions have a shorter average total task time and perform slightly better in terms of usability and cognitive workload. Similarly, Uva et al. compared SAR to paper instructions, proving a reduction of completion time and error rates at manual workstations when using SAR [ 25 ].

Zhou et al. describe a SAR system which aims to improve the accuracy and eficiency of automotive spot welding inspection through dynamic in-situ visualization [ 26 ]. Specific geometric shapes indicate the inspection method. According to the authors, industrial applications of SAR include quality assurance, material handling, maintenance and repair, lowering of skill-level requirement, and training. The collaborative welding system by Tavares et al. uses projection to convey information to a human operator while orchestrating tasks between the operator and a welding robot [ 27 ]. SAR improves operator productivity and drastically reduces manufacturing errors. Mengoni et al. use SAR to support manual work in future smart factories and detect risks to human safety and the musculoskeletal system [ 28 ]. Usability, performance, and efectiveness proved to be higher than with pick by light system using LEDs, especially for dificult tasks.

3. Concept

Based on previous research on SAR and adaptivity, we describe an adaptivity framework for manual workstations on the shop floor in manufacturing. We then present a first prototypical implementation.

3.1. Adaptivity framework

Our framework addresses adaptivity in the dimensions of machines and factory layout, workers, and collaborative tasks, emphasizing human-centeredness and the workers’ well-being. The framework consists of four consecutive phases that form a closed loop (see Figure 1). Adaptation triggers from the operator, task, and workplace dimensions initiate an adaptation loop. The digitized information is analyzed software-side in an adaptation strategy and a decision is made about which adaptation to perform in the assistance system. Since our scenario is a SAR application, adaptations are limited to the projection and the projector. Finally, the adaptations are evaluated whether they are beneficial so that readjustments can be made accordingly. Each of the four phases is explained in detail below.

Adaptation Triggers. Potential triggers for adaptations in the operator, task, and workplace dimensions may be static or situational/dynamic. Static triggers relate to long-term characteristics such as individual prerequisites and skills of operators, general product dimensions, work instructions, or the layout of the workplace. Situational triggers change dynamically during work, and include the mental and physical condition, emotions or interactions of operators, faulty product assembly or changing environmental conditions in the factory such as lighting conditions or temperature. Depending on the type of trigger, diferent methods are suitable for monitoring and forwarding them to the adaptation strategy. Static triggers could be provided by user profiles or digital twins, while sensors are particularly suitable for capturing situational conditions.

Adaptation Strategy. An adaptation strategy decides on adaptations depending on the triggers. Various algorithmic implementations from simple, rule-based systems to machine learning algorithms are conceivable. However, producing human-centered decisions is of central importance. We propose a modular software architecture with components that analyze triggers and initiate adaptations. The calibration module adapts the projection to the workplace and surfaces. The interaction module detects movements and navigation gestures, whereupon the projection changes dynamically. The content module adapts the content projected, e.g. to the qualification of operators or incorrect assembly. The cognitive module plays a key role by adapting the system, e.g. to physical or mental limitations, experience, or situational conditions of workers. The motivation module ensures engaging adjustments such as short interruptions or the use of gameful design in case of fatigue or declining production speed, as proposed by [ 29, 30 ]. The health module processes health-threatening triggers such as extreme temperatures, poor lighting conditions, insuficient breaks, or bad poses. Finally, the notification module warns of critical situations.

Adaptation. Potential adaptations of the SAR system include the projection and the projector. Changes in the projection are possible for content (e.g. displayed information, language), style (e.g. shapes, colors, sizes, animation), or tone (e.g. level of detail, wording), and the projector could be rotated, turned or height-adjusted.

Evaluation. The purpose of the evaluation phase is to ensure that the adaptations are beneficial for operator engagement and well-being, as well as for human-machine collaboration, in order to adjust the adaptation strategy if necessary. We propose a mixture of subjective and objective measures that can either be incorporated into the work process in real time or used for long-term evaluation. Both within and between subjects study designs are conceivable. Between subjects is suitable to compare user groups that have diferent barriers. Within subjects can be used to investigate how the interface performs in comparison to previous versions of the interface and if significant individual diferences appear when adapting the novel interface. Objective measures include changes in stress, fatigue, accuracy, and error rates after preceding adaptations, and can be determined relatively well on a situational basis. Acute stress triggers changes in emotions, cognition, behavior, and physiological responses that can be continuously and unobtrusively recorded with sensors. Physiological measurements include heart rate, heart rate variability, skin conductivity, and thermal images. The coding of behavior may encompass location, gestures and interaction, gaze direction, facial expression, or speech. During acute stress, for example, the heart rate typically increases, whereby a greater variability of the heart rate signifies higher adaptability of the organism. These changes can be recorded using wearables such as a smartwatches or fitness trackers. In order to properly interpret measurements, stressful states should be compared with a no-stress baseline or restful states [ 31 ]. Alternatively, subjective self-report scales can be a viable option for long-term recording stress responses [ 32, 33 ]. Fatigue results from mental or physical exertion and is a multidimensional phenomenon with mental, physiological, and cognitive components lacking clear biological markers [ 34 ]. Measurement is usually based on uni- or multidimensional scales that capture the severity or type of fatigue. Specific performance aspects such as reduced reaction time, alertness, and short-term memory can indicate work fatigue. Measures of engagement and well-being, which are expressed, for example, in usability, user experience and on a psychological, physical, social and emotional level [ 29 ], need to be collected asynchronously to work. It is possible that these long-term surveys will result in the need to undertake general changes to the design and functions of the assistance system.

3.2. Use Case and Setup of the Prototype

AVISAR is located at the manual workstation of an Industry 4.0 model factory in which simplified smartphones consisting of a circuit board and two fuses are produced. Incorrectly assembled products are sent to manual repair. Our technical setup consists of a full HD ViewSonic M2e LED pico projector, a Leap Motion Controller for gesture recognition, and a 500mm x 100mm white, non-reflective projection surface mounted to the aluminium frame of the workstation (see Figure 2). The projector and the Leap Motion Controller are facing the product and the white projection surface from above, last of which is positioned at the operator’s waist height. There are two projection areas: On the product itself, in-situ projection highlights the faulty area, e.g. a missing fuse on the circuit board. Repair instructions are shown on the white surface.

Several adaptation triggers are implemented in the prototype, not all of which work automatically yet. Gesture recognition can be used to confirm repair steps so that the next step is highlighted. We implemented static poses over the instruction area, which allows confirmation when holding a hand over one of the repair steps for a certain amount of this. This dwell time is visualized by a circular loading animation. Moreover, gesturing over the repair area mutes the projection in order to avoid misleading visual cues on the user’s hands. In addition, three user profiles have been integrated for operators with barriers in terms of qualifications, language, or digital skills. In the future, RFID tags will be integrated to automatically recognize the profiles. For simulating various situational tasks, six typical assembly errors were implemented, which should be automatically detected via image recognition in the future. Marker-based calibration of the projection to the layout of the workstation is also in preparation.

4. Discussion

AVISAR has already gone through several development stages. An earlier version, which did not yet include barrier profiles and gesture recognition, was evaluated in an informal user survey by visitors to an Industry 4.0 model factory at university. 14 people in total took part in our survey (with an average age of 39). The participants were professionals with backgrounds in engineering, chemistry, process technology, and digitalization. Students and department heads took part as well. AVISAR was shown as part of the complete tour of the factory and the functionality and ideas behind the concept of the system were demonstrated. Afterwards, the participants answered 8 questions on a 6 point Likert scale about the general recognisability and arrangement of the texts and projections, icons and animations, as well as confidence to work independently using the assistance. Overall, the feedback was positive. Projections were easily recognizable, instructions were clear and users felt able to carry out repairs independently using the projection. Based on the feedback, interactivity was integrated and icons were revised so that they would look more realistic. Suggestions for improvement included repositioning the projection surface to take the strain of the user’s neck. Overall, the results give us confidence that AVISAR is a valuable tool to support manual work in manufacturing.

Regarding the practical implementation and interoperability with existing systems in smart factories, there are some important considerations. AVISAR consists of a TypeScipt/JavaScript web application and integrates the Leap Motion Controller for gesture recognition. It thus requires a PC with Windows, MacOS, or Linux, but it is possible to realize the architecture with other software and hardware. A more recent version of the Leap Motion Controller supports deployment on a Raspberry Pi, which makes the setup more afordable, smaller and scalable. For gesture recognition, there are various other approaches for camera-based recognition using computer vision and machine learning [ 35, 36 ] or sensing methods such as WiFi signal sensing or capacitive sensing exist [ 37 ]. To ensure that the adaptive assistance system blends seamlessly into existing industrial production systems, common industry standards such as the platformindependent machine-to-machine communication standard OPC UA should be used. Unless no other solution is already provided, real objects or humans, such as products delivered to a manual workstation or operators interacting with the assistance system, may be identified using RFID transponders. To ensure secure mounting and prevent mechanical damage to the projector and further components (e.g. gesture sensor and RGB camera), extra device protection may be required.

5. Conclusion and Outlook

In this paper, we proposed an Adaptive Visual Assistance system using Spatial Augmented Reality (AVISAR), which adapts to tasks, the layout of a manual workplace, and individual human skills and needs to enable a more inclusive work environment and the well-being of the workforce. We presented a conceptual adaptivity framework and a first working prototype of the AVISAR system.

Our next step will be to finalize the automatic adaptation of the system to user profiles and errors as well as the marker-based calibration of the projection to suitable projection surfaces. After that, further adaptation triggers will be incorporated into the user profiles for improved individualized support, training, and on-boarding for people with cognitive challenges or other special prerequisites. Moreover, sensors for recording the worker’s state will be integrated, starting with the measurement of electrodermal activity (EDA) to determine emotional arousal and stress levels. However, sensors are to be integrated with care, as it also touches on ethical issues and the fundamental rights of operators. The EU Artificial Intelligence Act (AI Act) [ 38 ] explicitly classifies AI-based emotion recognition in the workplace as an unacceptable risk and prohibits it unless for safety or medical reasons. Finally, we will draw attention to the evaluation phase of the AVISAR application by assessing performance, usability, and user experience. We are currently planning to conduct a between subjects laboratory study in which 60 participants with illiteracy, language barriers, and lack of familiarity with digital technologies carry out six diferent manual repair tasks with the help of AVISAR. Repair instructions in AVISAR are displayed in a diferent language or only with icons. By surveying stress levels, accuracy, task completion times, behavior patterns, and usability, we hope to gain insights into how adaptations foster an inclusive working environment for user groups with diferent needs. In follow-up studies, we intend to explore further and personalized adaptations and recruit real workers from industry as participants. It is essential that our assumptions about user groups and their individual skills and needs undergo a thorough investigation to ensure that AVISAR ofers true benefits for workers in industrial production. Our aim is to create an adaptive assistance system that can also be used in similar contexts in industrial production in order to promote the well-being of workers in the long term.

Acknowledgments

On behalf of Elena Stoll and Karishma Jagdish Gavali, this research was funded as part of the project “Produktionssysteme mit Menschen und Technik als Team” (ProMenTaT, application number: 100649455) with funds from the European Social Fund Plus (ESF Plus) and from tax revenues based on the budget passed by the Saxon State Parliament.

References

[1] E. Geurts , G. Rovelo Ruiz , K. Luyten , S. Houben , B. Weyers , A. Jacobs , P. Palanque, HCI and worker well-being in manufacturing industry , in: Proceedings of the 2022 International Conference on Advanced Visual Interfaces , AVI 2022 , Association for Computing Machinery , New York, NY, USA, 2022 , pp. 1 - 2 . doi: 10 .1145/3531073.3535257.

[2] B. G. Mark , E. Rauch , D. T. Matt, Worker assistance systems in manufacturing: A review of the state of the art and future directions , Journal of Manufacturing Systems 59 ( 2021 ) 228 - 250 . doi: 10 .1016/j.jmsy. 2021 . 02 .017.

[3] D. Romero , P. Bernus , O. Noran , J. Stahre, Å. Fasth, Fast-Berglund , The Operator 4.0: Human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems , in: Advances in Production Management Systems , 2016 , pp. 677 -- 686 . doi: 10 .1007/978-3- 319 -51133-7_ 80 .

[4] J. Ulmer , S. Braun , C.-T. Cheng, S. Dowey, J. Wollert , A human factors-aware assistance system in manufacturing based on gamification and hardware modularisation , International Journal of Production Research 61 ( 2023 ) 7760 - 7775 . doi: 10 .1080/00207543. 2023 . 2166140 .

[5] F. Gullà , S. Ceccacci , M. Germani , L. Cavalieri , Design adaptable and adaptive user interfaces: A method to manage the information , Springer, Cham, 2015 , pp. 47 - 58 . isbn: 978 -3- 319 -18374-9.

[6] V. A. Cortes , V. H. Zirate , J. A. R. Uresti , B. E. Zayas , Current challenges and applications for adaptive user interfaces, Human-Computer Interaction ( 2009 ) 49 - 68 . doi: 10 .5772/7745.

[7] M. Dimitrova , G. Johannsen, H. Nour Eldin , J. Zaprianov , M. Hubert , Adaptivity in humancomputer interface systems: Identifying user profiles for personalised support , IFAC Proceedings Volumes 31 ( 1998 ) 407 - 412 . doi: 10 .1016/S1474- 6670 ( 17 ) 40127 - 3 , 7th IFAC Symposium on Analysis, Design and Evaluation of Man Machine Systems (MMS'98) , Kyoto, Japan, 16 - 18 September 1998 .

[8] M. P. Graus , B. Ferwerda , 1. Theory-grounded user modeling for personalized HCI , De Gruyter Oldenbourg , Berlin, Boston, 2019 , pp. 1 - 30 . doi: 10 .1515/ 9783110552485 - 001 .

[9] F. Loch , J. Czerniak , V. Villani , L. Sabattini , C. Fantuzzi , A. Mertens , B. Vogel-Heuser , An adaptive speech interface for assistance in maintenance and changeover procedures , in: M. Kurosu (Ed.), Human-Computer Interaction . Interaction Technologies , volume 10903 , Springer International Publishing, Cham, 2018 , pp. 152 - 163 . doi: 10 .1007/ 978-3- 319 -91250-9_12, series title: Lecture Notes in Computer Science.

[10] P. Burggräf , M. Dannapfel , T. Adlon , M. Kasalo , Adaptivity and adaptability as design parameters of cognitive worker assistance for enabling agile assembly systems , Procedia CIRP 97 ( 2021 ) 224 - 229 . doi: 10 .1016/j.procir. 2020 . 05 .229.

[11] R. Oppermann , R. Rashev , S. Birlinghoven , Adaptability and adaptivity in learning systems , Knowledge transfer 2 ( 1997 ) 173 - 179 . isbn: 900427 - 015 -X.

[12] Y. Koren , X. Gu , W. Guo, Reconfigurable manufacturing systems: Principles, design, and future trends , Frontiers of Mechanical Engineering 13 ( 2018 ) 121 - 136 . doi: 10 .1007/ s11465-018-0483-0.

[13] M. Ghobakhloo , M. Iranmanesh , M.-L. Tseng , A. Grybauskas , A. Stefanini , A. Amran , Behind the definition of Industry 5.0: A systematic review of technologies, principles, components, and values , Journal of Industrial and Production Engineering 40 ( 2023 ) 432 - 447 . doi: 10 .1080/21681015. 2023 . 2216701 .

[14] M. Peruzzini , M. Pellicciari , A framework to design a human-centred adaptive manufacturing system for aging workers , Advanced Engineering Informatics 33 ( 2017 ) 330 - 349 . doi: 10 .1016/j.aei. 2017 . 02 .003.

[15] S. Schlund , D. Kostolani , Towards designing adaptive and personalized work systems in manufacturing , in: P. Plapper (Ed.), Digitization of the work environment for sustainable production , GITO Verlag, 2022 , pp. 81 - 96 . doi: 10 .30844/WGAB_ 2022 _ 5 .

[16] V. Villani , et al., The INCLUSIVE system: A general framework for adaptive industrial automation , IEEE Transactions on Automation Science and Engineering 18 ( 2021 ) 1969 - 1982 . doi: 10 .1109/TASE. 2020 . 3027876 .

[17] H. Oestreich , M. Heinz-Jakobs , P. Sehr , S. Wrede , Human-centered adaptive assistance systems for the shop floor , in: C. Röcker , S. Büttner (Eds.), Human-Technology Interaction , Springer International Publishing, Cham, 2023 , pp. 83 - 125 . doi: 10 .1007/ 978-3- 030 -99235- 4 _ 4 .

[18] C. Petzoldt , D. Keiser , T. Beinke , M. Freitag , Functionalities and implementation of future informational assistance systems for manual assembly: Towards individualized, incentivebased assistance and support of ergonomics , Springer International Publishing, Cham, 2020 , pp. 88 - 109 . doi: 10 .1007/978-3- 030 -64351- 5 _ 7 .

[19] P. Bertram , M. Birtel , F. Quint , M. Ruskowski , Intelligent manual working station through assistive systems , IFAC-PapersOnLine 51 ( 2018 ) 170 - 175 . doi: 10 .1016/j.ifacol. 2018 . 08 .253.

[20] B. Mark , L. Gualtieri , E. Rauch, R. Rojas , D. Buakum , D. Matt , Analysis of user groups for assistance systems in Production 4.0 , in: 2019 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), 2019 , pp. 1260 - 1264 . doi: 10 .1109/ IEEM44572. 2019 . 8978907 .

[21] P. Milgram , F. Kishino , A taxonomy of mixed reality visual displays , IEICE Trans. Information Systems vol. E77-D, no. 12 ( 1994 ) 1321 - 1329 . URL: https://api.semanticscholar.org/ CorpusID:17783728.

[22] T. Zigart , S. Schlund , Ready for industrial use? A user study of spatial augmented reality in industrial assembly , in: Proceedings 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct , 2022 , pp. 60 - 65 . doi: 10 .1109/ ISMAR-Adjunct57072. 2022 . 00022 .

[23] M. Hornacek , H. Küfner-McCauley , M. Trimmel , P. Rupprecht , S. Schlund , A spatial AR system for wide-area axis-aligned metric augmentation of planar scenes , CIRP Journal of Manufacturing Science and Technology 37 ( 2022 ) 219 - 226 . doi: 10 .1016/j.cirpj. 2022 . 01 .011.

[24] P. Rupprecht , H. Kuefner-McCauleya , M. Trimmela , S. Schlund , Adaptive spatial augmented reality for industrial site assembly , Procedia CIRP 104 ( 2021 ) 405 - 410 . doi: 10 .1016/j.procir. 2021 . 11 .068.

[25] A. E. Uva , M. Gattullo , V. M. Manghisi , D. Spagnulo , G. L. Cascella , M. Fiorentino, Evaluating the efectiveness of spatial augmented reality in smart manufacturing: A solution for manual working stations , The International Journal of Advanced Manufacturing Technology 94 ( 2018 ) 509 - 521 . doi: 10 .1007/s00170-017-0846-4.

[26] J. Zhou , I. Lee , B. Thomas , R. Menassa , A. Farrant , A. Sansome , Applying spatial augmented reality to facilitate in-situ support for automotive spot welding inspection , in: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, ACM , 2011 , pp. 195 - 200 . doi: 10 .1145/2087756.2087784.

[27] P. Tavares , C. M. Costa , L. Rocha , P. Malaca , P. Costa , A. P. Moreira , A. Sousa , G. Veiga, Collaborative welding system using BIM for robotic reprogramming and spatial augmented reality , Automation in Construction 106 ( 2019 ) 102825 . doi: 10 .1016/j.autcon. 2019 . 04 .020.

[28] M. Mengoni , S. Ceccacci , A. Generosi , A. Leopardi , Spatial augmented reality: An application for human work in smart manufacturing environment , Procedia Manufacturing 17 ( 2018 ) 476 - 483 . doi: 10 .1016/j.promfg. 2018 . 10 .072.

[29] C. Martinie , P. Palanque , E. Barboni, Increasing engagement and well-being of operators working with automation by integrating task models and gameful design , Personal and Ubiquitous Computing 27 ( 2023 ) 1 - 28 . doi: 10 .1007/s00779-023-01783-4.

[30] O. Korn , M. Funk , A. Schmidt , Towards a gamification of industrial production: A comparative study in sheltered work environments , in: Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS '15 , Association for Computing Machinery, New York, NY, USA, 2015 , p. 84 - 93 . doi: 10 .1145/2774225.2774834.

[31] S. Immanuel , M. N. Teferra , M. Baumert , N. Bidargaddi , Heart Rate Variability for Evaluating Psychological Stress Changes in Healthy Adults: A Scoping Review , Neuropsychobiology 82 ( 2023 ) 187 - 202 . doi: 10 .1159/000530376.

[32] A. D. Crosswell , K. G. Lockwood , Best practices for stress measurement: How to measure psychological stress in health research , Health Psychology Open 7 ( 2020 ) 2055102920933072 . doi: 10 .1177/2055102920933072.

[33] A. Goyal , S. Singh , D. Vir , D. Pershad , Automation of stress recognition using subjective or objective measures , Psychological Studies 61 ( 2016 ) 348 -- 364 . doi: 10 .1007/ s12646-016-0379-1.

[34] K. Sadeghniiat-haghighi , Z. Yazdi , Fatigue management in the workplace , Industrial Psychiatry Journal 24 ( 2015 ) 12 - 17 . doi: 10 .4103/ 0972 - 6748 . 160915 .

[35] M. Oudah , A. Al-Naji , J. Singh Chahl , Hand gesture recognition based on computer vision: A review of techniques , Journal of Imaging 6 ( 2020 ). URL: https://api.semanticscholar.org/ CorpusID:221356266. doi: 10 .3390/jimaging6080073.

[36] S. Adhikari , T. K. Gangopadhayay , S. Pal , D. Akila , M. Humayun , M. Alfayad , N. Z. Jhanjhi , A novel machine learning-based hand gesture recognition using hci on iot assisted cloud platform , Computer Systems Science and Engineering 46 ( 2023 ) 2123 - 2140 . doi: 10 .32604/ csse. 2023 . 034431 .

[37] F. Noble , M. Xu , F. Alam , Static hand gesture recognition using capacitive sensing and machine learning , Sensors 23 ( 2023 ). doi: 10 .3390/s23073419.

[38] E. Parliament , Artificial intelligence act , 2024 . URL: https://www.europarl.europa.eu/doceo/ document/TA-9 -2024-0138_EN .pdf.