In human-robot collaboration (HRC), robots interact physically with humans in a shared environment, without the typical barriers or protective cages used in traditional robotics systems, it is therefore essential to consider safety measures as a substantial part of the assembly sequence planning (ASP). In HRC, this process is complicated and time-consuming, especially when ensuring that no safety hazards are overlooked. This paper reports on first results showing a concept on how to semi-automate the process of safety analysis for human-robot ASP. Our concept integrates transformer-based natural language processing NLP models to automatically generate suggestions about potential safety hazards, their causes and consequences for each assembly step. A human safety expert reviews the pre-populated information and incorporates supplementary safety considerations via a dashboard. Preliminary results indicate a significant reduction in manual efort for the expert in the creation of safety measures.
CEUR ceur-ws.org
Human-robot collaboration (HRC) is a research field with a wide range of applications, future
scenarios, and potentially a high economic impact [
While traditionally many safety risks were eliminated by a clear separation of working areas
for humans and robots (e.g., by a protective fence) [
Therefore, risk assessment and safety hazard identification are essential steps in HRC assembly
sequence planning (ASP). The ASP process must include a comprehensive risk assessment to
identify potential safety hazards arising from human-robot interactions, equipment malfunction,
or task allocation errors. The challenge lies in accurately identifying all possible safety hazards,
their causes and consequences in diverse and often dynamic HRC scenarios [
To address this challenge, this paper proposes a conceptual safety planning framework for HRC. The semi-automatic approach generates possible safety hazards, their causes and consequences using machine learning and provide them to a human safety expert using a graphical dashboard. The expert reviews the proposed output, edits it if necessary or adds additional information. We evaluate our proposed conceptual planning framework using an exemplary use case that resembles industry needs.
This paper is outlined as follows: Section 2 discusses the current state of the art and its shortcomings. Subsequently, Section 3 develops a conceptual framework for safety planning in HRC. Preliminary evaluation results indicating the usefulness of our approach are given in Section 4. Finally, Section 5 concludes the paper.
Considering the importance safety has in the context of HRC, safety standards and designs that provide unified requirements and design guidelines for collaborative robotics exist. The work by Tan provides us with five fundamental safety designs that focuses on robots, work environment, control systems, monitoring systems, and the application of safety in their construction and usage [6]. In 2006, Steinfeld proposed the standardization of safety measures [7] and in 2016, the international organization of safety provided ISO/TS 15066, that specifies safety requirements for collaborative industrial robot systems and the work environment [8]. Arents et al. [9] provide a review of studies focusing on safety of HRC systems. 25% of all reviewed studies did not use any safety actions, and more than 50% did not use any standards to address safety issues, showing a lack of safety functionalities in HRC.
Wang et al. focus on building a JAVA based safety system with a mathematical model that takes ISO/TS 15066's restrictions in biomechanical limits and deploys it to an HRC system [10]. In this case, the tests were implemented in the HRC lab at KTH, Sweden. The system uses psychological characteristic data to provide the operator with a personalized safety strategy plan. Vicentini proposes a framework that does safety assessment through formal verification in HRC [11]. It is used for supporting dynamic safety assessment in HRC applications. The system uses model-based formal verification to explore possible workflows, to identify hazards and to introduce provisions for risk hazards. It is important to focus that it uses temporal logic language as it focuses on all details including the irrelevant ones as any type of data could add value in acknowledging hazardous situations and therefore to mitigate them by introducing risk reduction measures in the model.
Other research in this area focuses on aspects such as proposing strategies for ensuring safety and productivity in assembly stations [12], advancing HRC in automotive assembly for safer and eficient operations [ 13], improving ergonomics, reducing injury risk in automotive brake disc assembly [14], and present a disassembly sequence planner for HRC in automating disassembly tasks [15]. A precise safety analysis for each step of the assembly sequences is still missing.
In previous work, we have shown that production sequences for manufacturing tasks can be generated from model-based specifications of the product and conceptual models of the production site’s architecture, including its individual Cyber-Physical Production Systems (CPPS) and their capabilities [16]. Recently, we proposed an ASP algorithm for HRC tasks [17]. This algorithm generates possible assembly sequences based on the capabilities of the robot and a CAD-specification of the product. The algorithm allows the extraction of relevant information for the assembly steps which we can use to support the safety evaluation of the diferent assembly steps in this paper.
We implemented a safety planning algorithm generating an ontological description of the assembly sequence which is shown in Figure 1. Each possible assembly sequence plan includes a series of n assembly steps. Each step is described by its essential components (elements that are used in an assembly step like screws, bolts, plastic casings etc.), required actions (such as joining or gluing), tools (e.g., screwdriver or hammer), and the resources involved (human, robot or a combination of both).
The safety planning approach follows the safety analysis workflow based on the principles of Failure Mode and Efects Analysis (FMEA) [ 18]. Therefore, the information provided by the ASP are used to identify possible safety hazards, evaluate them, document their causes and consequences, and estimate their frequency, severity, and risk. For each safety hazard, we identify causes and consequences. A cause refers to the reason behind the occurrence of the safety hazard while a consequence outlines the potential outcomes resulting from the safety hazard. The classification of safety hazards is accomplished through frequency, severity, and risk factors (see Table 1). These values serve as indicators of the importance of each safety hazard and provide guidance for considering their significance in the overall safety analysis. Risk, in essence, is the combined efect of severity and frequency. All initial values are proposed by our prepopulated safety algorithm (PS-Algorithm) and subsequently assessed by the safety expert.
The PS-Algorithm consists of four stand-alone models which are arranged in a chain shown in Figure 3. Each model PS - 1 to PS - 4 is a transformer-based NLP model trained with the concept of transfer learning. We use existing models and retrain it with specific data gained from the human safety expert. Instead of using only one model to generate our required output for the safety analysis, we divided the whole process into four parts: 1. The first model PS-1 uses the concept of NLP to generate the safety hazards as an output based on the input data which is Component, Action, Tool and the Resource. The architecture allows converting text information into numerical representations as well as positional information in the form of a vector. The converted information is processed via diferent neural network layers. The output converts the processed data to a text sequence representing the safety hazard. 2. These safety hazards are used to feed the second NLP model (PS-2) for the generation of associated causes. 3. PS-3 takes safety hazards from PS-1 as input to form the consequences as output. 4. Our last model (PS-4) is a neural network taking safety hazard (PS-1), cause (PS-2) and consequence (PS-3) of each assembly step as input. The output layer of the neural network has three classes, frequency, severity and risk. For each class, a number between one and ifve is generated, representing ordinal scale features.
A human safety expert actively contributes expert knowledge as an input for each model. Over time, the input from the safety expert helps to refine the algorithm’s output. The expert’s corrections are incorporated into the PS-Algorithm, which undergoes retraining to enhance the various models (PS-1 to PS-2, as depicted in Figure 3).
We integrated our proposed Safety Analysis (Figure 2) into an existing planning framework [17] for HRC assembly processes, allowing us to harness all input data we need for our PSAlgorithm. The data include information about Components, Actions, Tools, and Resources for each assembly step. This comprehensive approach ensures that safety considerations seamlessly intertwine with assembly planning in the realm of HRC, enhancing overall process safety and eficiency. The presented framework is validated on an exemplary collaboration use case where human and cobot collaborate to assemble DUPLO blocks. We leveraged this information to automatically generate safety hazards for every step in the ASP, along with their associated causes and consequences. To achieve this, we utilized ChatGPT API with our models (PS-1 to PS-3). Our preliminary findings indicate that the output yielded more than ten safety hazards and up to ten causes and consequences for the DUPLO use case. Moreover, the structured representation of information has significantly expedited the safety analysis process, eliminating the need for the human expert to search for required information for each assembly step.
While this initial evaluation showed the advantages of the proposed framework, we have identified further improvements for future work. In particular, users liked the dashboard as an easy accessible information source. Although the dashboard currently focuses on representing the safety hazards and a representation of its causes, relating these to specification models can improve the analysis of the safety expert in the future. In particular, linking the safety hazards to process models showing the assembly processes, to development models showing the impact of design decisions taken, or to conceptual representations of the work pieces, can lead to a more thorough safety analysis and support better feedback on the found safety hazards.
In this paper, we introduce a conceptual safety planning framework for assembly sequences in an HRC context. The conventional approach to HRC safety analysis involves manual eforts by human safety experts which proves to be labor-intensive and non-scalable when dealing with multiple products. To address this, we propose a semi-automated support system that assists safety experts in analyzing assembly sequences for HRC applications.
Our approach combines expert knowledge with algorithms to partially automate the safety analysis process, reducing the manual efort and resources required for comprehensive safety assessment. Moreover, the algorithm’s ability to learn from the safety expert’s input and preexisting information holds the potential for continuous improvement, leading to enhanced performance and more accurate safety suggestions over time.
Preliminary results from the implementation of our safety planning framework showcase significant advantages over manual safety analysis methods. The framework exhibits the potential to elevate the quality and speed of safety analysis for HRC use cases.
Moving forward, we plan to extend the application of our semi-automated safety analysis to supports runtime evaluation and re-planning of assembly sequences considering the safety of the collaboration. To validate the eficacy and practicality of our approach, we intend to apply the PS-Algorithm to more intricate industry use cases.
This research was partly funded by the Bayerische Forschungsstiftung under grant no. AZ1512-21. We thank our industry partners Fresenius Medical Care, Wittenstein SE, Uhlmann und Zacher, DE software & control and Universal Robots for their support. [6] J. T. C. Tan, F. Duan, R. Kato, T. Arai, Safety strategy for human–robot collaboration:
Design and development in cellular manufacturing, Advanced Robotics 24 (2010) 839–860. [7] A. Steinfeld, T. Fong, D. Kaber, M. Lewis, J. Scholtz, A. Schultz, M. Goodrich, Common metrics for human-robot interaction, in: 1st ACM SIGCHI/SIGART Conf. on Human-robot interaction, 2006, pp. 33–40. [8] S. ISO, Robots and robotic devices—collaborative robots (iso-15066: 2016), Int. Organization for Standardization (2016). [9] J. Arents, V. Abolins, J. Judvaitis, O. Vismanis, A. Oraby, K. Ozols, Human–robot collaboration trends and safety aspects: A systematic review, Journal of Sensor and Actuator Networks 10 (2021) 48. [10] X. V. Wang, A. Seira, L. Wang, Classification, personalised safety framework and strategy for human-robot collaboration, in: Int. Conf. on Computers & Industrial Engineering, CIE, 2018. [11] F. Vicentini, M. Askarpour, M. G. Rossi, D. Mandrioli, Safety assessment of collaborative robotics through automated formal verification, IEEE Trans. Robotics 36 (2019) 42–61. [12] G. Michalos, S. Makris, P. Tsarouchi, T. Guasch, D. Kontovrakis, G. Chryssolouris, Design considerations for safe human-robot collaborative workplaces, Procedia CIrP 37 (2015) 248–253. [13] G. Michalos, S. Makris, J. Spiliotopoulos, I. Misios, P. Tsarouchi, G. Chryssolouris, Robopartner: Seamless human-robot cooperation for intelligent, flexible and safe operations in the assembly factories of the future, Procedia CIRP 23 (2014) 71–76. [14] S. Heydaryan, J. Suaza Bedolla, G. Belingardi, Safety design and development of a humanrobot collaboration assembly process in the automotive industry, Applied Sciences 8 (2018) 344. [15] M.-L. Lee, S. Behdad, X. Liang, M. Zheng, Disassembly sequence planning considering human-robot collaboration, in: Am. Control Conf., IEEE, 2020, pp. 2438–2443. [16] M. Daun, J. Brings, P. A. Obe, S. Weiß, B. Böhm, S. Unverdorben, Using view-based architecture descriptions to aid in automated runtime planning for a smart factory, in: IEEE Int. Conf. on Software Architecture Companion, IEEE, 2019, pp. 202–209. [17] F. Schirmer, P. Kranz, C. G. Rose, J. Schmitt, T. Kaupp, Holistic Assembly Planning Framework for Dynamic Human-Robot Collaboration, The 18th Int. Conf. on Intelligent Autonomous Systems, Suwon, Korea (2023). [18] Z. Wu, W. Liu, W. Nie, Literature review and prospect of the development and application of fmea in manufacturing industry, The Int. Journal of Advanced Manufacturing Technology 112 (2021) 1409–1436.