Different effects of cognitive load on overhead crane operators in a virtual reality simulator

Abstract

Cranes are commonly employed in a variety of industries. However, when they are mishandled, they can cause several accidents. Environmental factors (e.g., noises) and cognitive load are considered to limit crane operators' efficiency. Although still underestimated, noise is a serious physical concern in many industries having a relevant impact on human cognition. Changes in cognitive function can result in human error and, in some situations, increased accidents and a reduction in work performance and productivity. This preliminary study aims to develop an overhead crane virtual reality simulator to train crane operators in different conditions of noise exposure and cognitive load. To this end, 18 inexperienced operators participated in the study. Firstly, they were involved in a traditional training session with slide, a Go/No Go test session and a VR training session. Then, participants were fully immersed in virtual scenarios and while being exposed to the three typical industrial noises, they had to concurrently move the overhead crane and perform cognitive tasks, assessing free verbal memory recall, auditory-verbal recognition and working memory. Results showed that the combination between noise conditions and cognitive performance has a different impact on participants' working memory and verbal recall. These findings may contribute to value the role of training for crane operators throughout immersive virtual reality simulations.

Keywords

• 1 Virtual Reality
• Cognitive Abilities
• Overhead Crane
• Safety
• Noise Variability

1. Introduction

Cranes of several varieties are used in numerous industries for lifting and trans-porting huge loads or assembling heavy pieces. Although cranes make it easier for workers to operate, when used incorrectly, they can result in several accidents that have severe consequences for the workers in the handling area [ 1 ]. Moreover, the environmental conditions and the cognitive factors can reduce the efficiency of crane operators, particularly when loads are handled by inexperienced or inattentive operators [ 2,3 ].

Noise is a significant physical threat in many industries, and it may impact human cognitive performance. Alongside, changes in the cognitive load can lead to human mistakes and, in some cases, increase accidents with a drop in working performance and productivity.

Zeydabadi et al. [ 4 ] investigated the effect of exposure to noise on some aspects of cognitive function of two groups of workers in the metal sector. A group was exposed to noise levels higher than 85 dB (A), whereas another group was exposed to noise levels lower than 80 dB (A). All participants had to complete the visual attention test and the Wechsler memory test before and after the work shift. Overall, the results showed a reduction of the cognitive abilities in the first group. Moreover, Golmohammadi et al. [ 5 ] explored the effects of noise on cognitive performance at medium levels (ranged from 60 to 80 dB (A)). The N-back and the Go/No-Go inhibition tests were used to examine cognitive performance during noise exposure. The results revealed a lower performance on the N-back test in the industrial noise condition.

Throughout the last decades, governments have made it a priority to develop and train workers' abilities. Nevertheless, despite being involved in training programs regularly, workers commonly complain about training programs that are insufficient or not tailored to their needs. Virtual reality can meet these objectives by allowing users to enhance their skills in dangerous situations [ 6-9 ] and researchers to investigate potential hazardous situations.

In the literature, it is possible to find other VR crane simulators such as the EOT crane simulator made by Dhalmahapatra et al. [ 9,10 ] or the quay crane simulator "Chamaleon" made by Fadda et al. [ 11 ]. Both consisted of different training scenarios and aimed to evaluate the operator's job performance. For example, the Chamaleon simulated different environmental conditions (e.g., weather, lighting) in which the audio system played sound effects caused by vibrations, collisions, and wind, while early signs of fatigue (ECG, EMG, pupil size) were monitored. In addition, simulations of crane operation were performed during serious games encouraging trainees to acquire operational skills that will hopefully translate safely into productivity in real workplaces [ 12,13 ].

The main purpose of this preliminary study is to set up a training protocol in order to see if different forms of industrial noises can affect overhead crane users' cognitive abilities during crane manoeuvres. The Virtual Reality Overhead Crane Simulator was developed and used to investigate the effects of noise on crane operators’ manoeuvre errors [ 14 ] and their physiological responses [ 15 ]. Since inexperienced operators mainly cause accidents, here we recruited non-expert participants who had to move the overhead crane and to concurrently perform cognitive tasks assessing free verbal memory recall, auditory–verbal recognition and working memory while being exposed to different audio scenarios (low, high, and modulated frequency noise at 80 dB(A); and at 51 dB(A), as control). The choice to investigate these cognitive components in this preliminary study lies in the fact that overhead crane operators are subjected to work situations in which the memorization and recall of verbal instructions, semantic long-term memory, and the combination of executive functions and visuospatial short-term memory are particularly stressed. In line with previous evidence [ 16 ], our main expectations concern a negative impact of the noise patterns on working memory processes.

2. Methodology 2.1.

Participants

The study involved a sample of 18 (7 women) inexperienced operators (age y.o., M= 29; SD = 4.4; range = 22-38) who attended the experiment voluntarily. Participants were master and PhD students of the Department of Architecture and Industrial Design of the Università degli Studi della Campania “Luigi Vanvitelli”. No age differences between groups emerged (p > 0.05). All of them self-reported that they had never had hearing problems during their life. Before taking part in the study, all participants signed a consensus form about the test. The study complies with the ethical principles of the Helsinki Declaration (2013) and the local ethics committee. 2.2.

Virtual Reality Overhead Crane Simulator

The virtual environment depicts a 580 m2 industrial area with storage units and a single-girder overhead crane. The space is separated into three zones: (1) a loading zone where the operator can hook the load, (2) a transit zone where the load must be securely handled while avoiding the central obstacle with a variable position, and (3) a target zone where the operator must deposit the load.

The crane's area of action is half the size of the building. It can realistically imitate the movement of the beam down the tracks, the trolley's transversal movement, and the up and down movement of the hook.

ArchiCAD was used to create the 3D model and 3D Studio Max to develop and optimize the crane and other objects, while the visual aspects of the virtual environment (surface materials and lighting) have been implemented and optimized in UE4 (see Fig. 1).

The Oculus Rift S was chosen as virtual reality exploration system. The associated controllers were employed for the crane's three movements, allowing the operator to tune the speed.

Three different industrial noises were employed as background noises: a low-frequency noise (LF), a high-frequency noise (HF) and a modulated noise (MOD) defined at 80 dB(A) (±1 dB). An additional soundtrack with minimal background noise set to 51 dB(A) was used as a control (CTRL) condition. The audio of the beam, trolley and hook were added and coordinated with the crane motions to increase the realism of the virtual settings. These characteristics were set with the help of a professional crane operator. 2.3.

Cognitive Tasks

The Rey Test, the Verbal Fluency Test, and the Backward Counting Test [ 17 ] were chosen as dual tasks for this study and implemented in the Virtual Reality Overhead Crane Simulator.

The Rey Test (REY) assesses the capacity to recognize correct and false items stored in short-term verbal memory. Participants were asked to recall as many words as they could from a list of 15 words. A total of ten lists were used in this experiment. The sum of the right answers was calculated as the total score.

The Verbal Fluency test (VF) assesses the long-term lexical-semantic store's expansion and recall. Furthermore, the test engages executive functions because the subject must self-generate the list without any external cues following the run-in. Participants were instructed to name as many words as they could with a target letter. Ten letters (S, C, A, P, I, M, D, T, B, F) were chosen for the experiment. The sum of right words was calculated as the total score.

The Backward Counting test (BC) assesses executive functions and visuo-spatial working memory resources. Participants were asked to count down by seven from a target number. For the experiment, ten target numbers were chosen (90, 83, 76, 68, 96, 81, 72, 64, 93, 84). The number of correctly generated numbers was calculated as the total score. 2.4.

Procedure

The test session started with traditional training with slides. The training topic was chosen based on workplace health and safety laws. The class, which lasted about 10 minutes, covered many types of cranes as well as the primary risks in-volved with crane use and how to utilize them correctly. Furthermore, guidance and warnings were offered, as well as the banned manoeuvres.

This first training section was followed by a Go/No Go test session (lasting about 5 minutes) only to examine inhibitory control and sensitive interference. All subjects successfully overcome this phase. Then, the subject was asked to wear the head-mounted display and the headphones Sennheiser HD 280 Pro used to playback the soundtracks.

After that, a second training session in virtual reality was carried out. The subject was given the possibility of learning how to use the virtual reality overhead crane simulator and practicing with the examples trials of cognitive tasks before beginning the experiment. The subjects could repeat the training in virtual reality as many times as they wanted. For the cognitive tasks, non-target items (words, letters, and numbers) were utilized during the training session.

Next, the subject was asked to carry out the manoeuvre test (see Fig. 2) of the overhead crane moving the load from the loading zone of the virtual industrial building to the target zone while also performing the three cognitive tasks in the four background noise scenarios (LF, HF, MOD and CTRL).

In the Rey Test, the list of 15 words was visually given for 42 seconds. Then the subject was asked to recall as many words as possible (30 seconds).

In the Verbal Fluency and Backward Counting tests, the subject was asked to list as many items as possible (30 seconds) after a cue was graphically presented (a target letter or a number, respectively).

Moreover, the main physiological parameters and manoeuvre errors recorded for the whole time of the experiment were also monitored. The four background noises (included the control condition) and the cognitive tasks were presented to the subjects in a Latin Square Balanced Sequences. A total of 12 protocols were administered to the subjects. The experiment lasted about one hour on average.

3. Data analysis and results

Data were entered in separate ANOVAs with Noise conditions (CTRL, LF, HF and MOD) as a fourlevel within variable on the scores of the Verbal Fluency, Rey List and Backward counting. The Newman-Keuls test was used to analyze post-hoc effects (at least p < 0.05). The magnitude of the significant effects was indicated by partial eta squared (ηp2).

As regards Backward Counting (see Fig. 3), the LF (M=3.94, SD=2.55) and MOD (M=4.06, SD=2.36) noise conditions were less accurate than the CTRL (M=6.06, SD=2.23) and HF (M=6.11, SD=3.18) conditions (at least p < 0.05): (F (3,51) = 5.092, p < 0.01, ηp2 = 0.23).

With regards Verbal Fluency, the MOD (M=8.63, SD=1.36) noise condition showed a better performance than the CTRL (M=6.72, SD=2.14) and LF (M=7.0, SD=2.95) conditions (at least p< 0.05): F (3,51) =3.335, p< 0.05, ηp2 = 0.16. No differences appeared with Rey list scores (p > 0.05).

4. Conclusions

This study aimed to investigate if and how noise patterns could alter basic cognitive abilities while overhead crane operations were performed. Besides, this study allowed us to develop and improve a Virtual Reality Overhead Crane Simulator training protocol ad hoc for inexperienced users by investigating some critical aspects of handling heavy loads in industrial environments.

The results showed that low frequency (LF) and modulated (MOD) noise conditions negatively impacted the Backward Counting. This means that the noise conditions can undermine basic visuospatial working memory and executive functions [18]. As for the verbal fluency, the ability to freely recall long-term lexical-semantic information was penalized by the MOD as compared to the LF and CTRL noise conditions. Although unexpected, this pattern of results is in line with evidence reporting that noise conditions characterized by regularity may in some cases improve the free recall of semantic items, especially when subjected to short-term exposures [19,20]. Finally, as for the Rey list, the absence of effects would suggest that participants were able to recognize correct and false items in short-term verbal memory, and that this ability was not altered by the different noise patterns in the present manipulation.

In conclusion, we found that the training protocol employing the Virtual Reality Overhead Crane Simulator could represent a valuable candidate for simulating part of the cognitive load required while performing crane operations under different kinds of noises. Specifically, our findings showed that the executive functions of inexperienced users resulted particularly impaired. However, the present study has some limitations. When operators performed cognitive tasks and maneuvered the overhead crane, they frequently interrupted load handling, discrediting that task, and concentrating only on the cognitive task. Besides, a more comprehensive cognitive assessment should be made. For these reasons and for the small sample size of participants involved the results have to be considered preliminary. Future studies should deepen the effectiveness of virtual reality-based training in multisensory industrial contexts and extend this protocol to samples with different experience levels.

5. Acknowledgements

This study has been developed as part of a scholarship funded in the framework of the National Operational Program ESF-ESFR Research and Innovation (PON RI 2014-2020), Action I.1 related to Innovative Industrial Ph.D. (Project Code-CUP- B25D18000010006).

6. References

[18] G. Ruggiero, T. Iachini, The role of vision in the Corsi Block-Tapping task: Evidence from blind and sighted people. Neuropsychology 24(2010) 674-679. doi 10.1037/a0019594 [19] J.G. Borkowsky, A.L. Benton. O. Spreen, Word fluency and brain damage. Neuropsychologia, 5 (1967): 135–140. doi 10.1016/0028-3932(67)90015-2 [20] G. B. Söderlund, S. Sikström, J.M. Loftesnes, E. J. Sonuga-Barke, The effects of background white noise on memory performance in inattentive school children. Behavioral and brain functions, 6 (2010) 1-10. doi 10.1186/1744-9081-6-55

References

[1] M.F. Milazzo , G. Ancione , V. Spasojevic Brkic , D. Vališ , Investigation of crane operation safety by analyzing main accident causes in: Walls, Revie &Bedford (Eds) Risk, Reliability and Safety: Innovating Theory and Practice , Taylor & Francis Group, London, 2017 , pp. 74 - 80 .

[2] Small Entity Compliance Guide for the Final Rule for Cranes and Derricks in Construction, OSHA U.S. Department of Labor Occupational Safety and Health Administration , 2014 .

[3] V.W.Y. Tam , I.W.H. Fung , Tower crane safety in the construction industry: A Hong Kong study . Safety Science 49 ( 2011 ) 208 - 215 .doi doi.org/10.1016/j.ssci. 2010 . 08 .001

[4] A. Zeydabadi , J. Askari , M. Vakili , S.J Mirmohammadi , M. A Ghovveh , A.H. Mehrparvar , The effect of industrial noise exposure on attention, reaction time, and memory . International Archives of Occupational and Environmental Health 92 ( 2019 ) 111 - 116 . doi 10.1007/s00420-018-1361-0

[5] R. Golmohammadi , E. Darvishi , J. Faradmal , J. Poorolajal , M. Aliabadi , Attention and short-term memory during occupational noise exposure considering task difficult . Applied Acoustics 158 ( 2020 ) pp. doi

[6] Q. T. Le , P. Akeem , C. S. Park , A Social Virtual Reality Based Construction Safety Education System for Experiential Learning . Journal of Intelligent & Robotic Systems 79 ( 2015 ) 487 - 500 . doi 10.1007/s10846-014-0112-z

[7] R. Sacks , P. Amotz , B. Ronen , Construction safety training using immersive virtual re-ality . ” Construction Management and Economics , 31 ( 2013 ) 1005 - 1017 . doi 10.1080/01446193. 2013 .828844

[8] E. Wyk , R. de Villiers, Virtual Reality Training Applications for the Mining Industry . In AFRIGRAPH '09, Proceedings of the 6th International Conference on Computer Graphics , Virtual Reality, Visualisation and Interaction in Africa ( 2009 ). doi 10.1145/1503454.1503465

[9] K. Dhalmahapatra , O. Pradhan , S. Das , K. Singh , J. Maiti , Prioritization of human errors in EOT crane operations and its visualization using virtual simulation , in Proceedings of 4th International Conference on Recent Advances in Information Technology , IEEE , Dhandad, India, 2018 . doi 10.1109/RAIT. 2018 .8388990

[10] K. Dhalmahapatra , J. Maiti , O. B. Krishna , Assessment of virtual reality-based safety training simulator for electric overhead crane operations , Safety Science 139 ( 2021 ). doi10 .1016/j.ssci. 2021 .105241

[11] P. Fadda et al., Multidisciplinary study of biological parameters and fatigue evolution in quay crane operators , Procedia Manufacturing 3 ( 2015 ) 3301 - 3308 . doi 10.1016/j.promfg. 2015 . 07 .410.

[12] A. K. George , M. L. McLain , K. Bijlani , R. Jayakrishnan , R. R. Bhavani . A Novel Approach for Training Crane Operators: Serious Game on Crane Simulator , in IEEE Eighth International Conference on Technology for Education (T4E) , Mumbai, India, 2 - 4 December 2016. Doi 10.1109/T4E. 2016 .030

[13] Y. Cai , S. L. Goei , W. Trooster, (Eds.). Simulation and serious games for education . Springer, Singapore, 2017 . Doi 10.1007/ 978 -981-10-0861-0

[14] M. Masullo , R.A. Toma , A. Pascale , G. Ruggiero , L. Maffei , Effects of noise on overhead crane operators' performances under mental fatigue, in a virtual reality crane simulator , in Proceedings of 2020 International Congress on Noise Control Engineering , Seoul, Corea del Sud , 23 - 26 August 2020 .

[15] M. Masullo , R.A. Toma , A. Pascale , G. Ruggiero , L. Maffei, Research methodology used to investigate the effects of noise on overhead crane operator's performances , in D. Sumpor et al. (eds.), Proceedings of the 8th International Ergonomics Conference, Advances in Intelligent Systems and Computing 1313 , Zagreb, Croatia, 2 - 5 December 2020. doi 10.1007/978-3- 030 - 66937-9_ 25

[16] M. Masullo , G. Ruggiero , D. Alvarez Fernandez , T. Iachini , L. Maffei , Effects of urban noise variability on cognitive abilities in indoor spaces: Gender differences . Noise & Vibration Worldwide ( 2021 ). doi 10.1177/09574565211030703

[17] C. Barletta-Rodolfi , F. Gasparini , E. Ghidoni, Kit del neuropsicologo italiano ., Dynamicon , Milano ( 2011 ).