=Paper=
{{Paper
|id=Vol-3100/paper8
|storemode=property
|title=Different effects of cognitive load on overhead crane operators in a virtual reality simulator
|pdfUrl=https://ceur-ws.org/Vol-3100/paper8.pdf
|volume=Vol-3100
|authors=Massimiliano Masullo,Toxana Adina Toma,Aniello Pascale,Gennaro Ruggiero,Luigi Maffei
|dblpUrl=https://dblp.org/rec/conf/psychobit/MasulloTPRM21
}}
==Different effects of cognitive load on overhead crane operators in a virtual reality simulator==
https://ceur-ws.org/Vol-3100/paper8.pdf
Different effects of cognitive load on overhead crane operators
in a virtual reality simulator
Massimiliano Masulloa,, Roxana A. Tomaa, Aniello Pascalea , Gennaro Ruggierob and Luigi
Maffeia
a
Department of Architecture and Industrial Design, Università degli Studi della Campania “Luigi Vanvitelli”,
Via San Lorenzo, Aversa, 81031, Italy
b
Department of Psychology, Università degli Studi della Campania “Luigi Vanvitelli”, Viale Ellittico, Caserta,
81100, Italy
Abstract
Cranes are commonly employed in a variety of industries. However, when they are mishandled,
they can cause several accidents. Environmental factors (e.g., noises) and cognitive load are
considered to limit crane operators’ efficiency. Although still underestimated, noise is a serious
physical concern in many industries having a relevant impact on human cognition. Changes in
cognitive function can result in human error and, in some situations, increased accidents and a
reduction in work performance and productivity.
This preliminary study aims to develop an overhead crane virtual reality simulator to train
crane operators in different conditions of noise exposure and cognitive load. To this end, 18
inexperienced operators participated in the study. Firstly, they were involved in a traditional
training session with slide, a Go/No Go test session and a VR training session. Then,
participants were fully immersed in virtual scenarios and while being exposed to the three
typical industrial noises, they had to concurrently move the overhead crane and perform
cognitive tasks, assessing free verbal memory recall, auditory–verbal recognition and working
memory.
Results showed that the combination between noise conditions and cognitive performance has
a different impact on participants’ working memory and verbal recall. These findings may
contribute to value the role of training for crane operators throughout immersive virtual reality
simulations.
Keywords 1
Virtual Reality, Cognitive Abilities, Overhead Crane, Safety, Noise Variability
1. Introduction
Cranes of several varieties are used in numerous industries for lifting and trans-porting huge loads
or assembling heavy pieces. Although cranes make it easier for workers to operate, when used
incorrectly, they can result in several accidents that have severe consequences for the workers in the
handling area [1]. Moreover, the environmental conditions and the cognitive factors can reduce the
efficiency of crane operators, particularly when loads are handled by inexperienced or inattentive
operators [2,3].
Noise is a significant physical threat in many industries, and it may impact human cognitive
performance. Alongside, changes in the cognitive load can lead to human mistakes and, in some cases,
increase accidents with a drop in working performance and productivity.
Proccedings of the Third Symposium on Psychology-Based Technologies (PSYCHOBIT2021), October 4–5, 2021, Naples, Italy
EMAIL: massimiliano.masullo@unicampania.it (A. 1); roxanaadina.toma@unicampania.it (A. 2); aniello.pascale@unicampania.it (A. 3);
gennaro.ruggiero@unicampania.it (B. 1); luigi.maffei@unicampania.it (A. 4)
ORCID: 0000-0002-0958-7536 (A. 1); 0000-0003-4130-5065 (A. 4)
©️ 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
Zeydabadi et al. [4] investigated the effect of exposure to noise on some aspects of cognitive function
of two groups of workers in the metal sector. A group was exposed to noise levels higher than 85 dB
(A), whereas another group was exposed to noise levels lower than 80 dB (A). All participants had to
complete the visual attention test and the Wechsler memory test before and after the work shift. Overall,
the results showed a reduction of the cognitive abilities in the first group. Moreover, Golmohammadi
et al. [5] explored the effects of noise on cognitive performance at medium levels (ranged from 60 to
80 dB (A)). The N-back and the Go/No-Go inhibition tests were used to examine cognitive performance
during noise exposure. The results revealed a lower performance on the N-back test in the industrial
noise condition.
Throughout the last decades, governments have made it a priority to develop and train workers'
abilities. Nevertheless, despite being involved in training programs regularly, workers commonly
complain about training programs that are insufficient or not tailored to their needs. Virtual reality can
meet these objectives by allowing users to enhance their skills in dangerous situations [6-9] and
researchers to investigate potential hazardous situations.
In the literature, it is possible to find other VR crane simulators such as the EOT crane simulator
made by Dhalmahapatra et al. [9,10] or the quay crane simulator "Chamaleon" made by Fadda et al.
[11]. Both consisted of different training scenarios and aimed to evaluate the operator's job
performance. For example, the Chamaleon simulated different environmental conditions (e.g., weather,
lighting) in which the audio system played sound effects caused by vibrations, collisions, and wind,
while early signs of fatigue (ECG, EMG, pupil size) were monitored. In addition, simulations of crane
operation were performed during serious games encouraging trainees to acquire operational skills that
will hopefully translate safely into productivity in real workplaces [12,13].
The main purpose of this preliminary study is to set up a training protocol in order to see if different
forms of industrial noises can affect overhead crane users' cognitive abilities during crane manoeuvres.
The Virtual Reality Overhead Crane Simulator was developed and used to investigate the effects of
noise on crane operators’ manoeuvre errors [14] and their physiological responses [15]. Since
inexperienced operators mainly cause accidents, here we recruited non-expert participants who had to
move the overhead crane and to concurrently perform cognitive tasks assessing free verbal memory
recall, auditory–verbal recognition and working memory while being exposed to different audio
scenarios (low, high, and modulated frequency noise at 80 dB(A); and at 51 dB(A), as control). The
choice to investigate these cognitive components in this preliminary study lies in the fact that overhead
crane operators are subjected to work situations in which the memorization and recall of verbal
instructions, semantic long-term memory, and the combination of executive functions and visuospatial
short-term memory are particularly stressed. In line with previous evidence [16], our main expectations
concern a negative impact of the noise patterns on working memory processes.
2. Methodology
2.1. Participants
The study involved a sample of 18 (7 women) inexperienced operators (age y.o., M= 29; SD = 4.4;
range = 22-38) who attended the experiment voluntarily. Participants were master and PhD students of
the Department of Architecture and Industrial Design of the Università degli Studi della Campania
“Luigi Vanvitelli”. No age differences between groups emerged (p > 0.05). All of them self-reported
that they had never had hearing problems during their life. Before taking part in the study, all
participants signed a consensus form about the test. The study complies with the ethical principles of
the Helsinki Declaration (2013) and the local ethics committee.
2.2. Virtual Reality Overhead Crane Simulator
The virtual environment depicts a 580 m2 industrial area with storage units and a single-girder
overhead crane. The space is separated into three zones: (1) a loading zone where the operator can hook
the load, (2) a transit zone where the load must be securely handled while avoiding the central obstacle
with a variable position, and (3) a target zone where the operator must deposit the load.
The crane's area of action is half the size of the building. It can realistically imitate the movement of
the beam down the tracks, the trolley's transversal movement, and the up and down movement of the
hook.
ArchiCAD was used to create the 3D model and 3D Studio Max to develop and optimize the crane
and other objects, while the visual aspects of the virtual environment (surface materials and lighting)
have been implemented and optimized in UE4 (see Fig. 1).
The Oculus Rift S was chosen as virtual reality exploration system. The associated controllers were
employed for the crane's three movements, allowing the operator to tune the speed.
Figure 1: The Virtual Reality Overhead Crane Simulator. The 3D model without materials (left). The 3D
model with physically based rendering materials (right).
Three different industrial noises were employed as background noises: a low-frequency noise (LF),
a high-frequency noise (HF) and a modulated noise (MOD) defined at 80 dB(A) (±1 dB). An additional
soundtrack with minimal background noise set to 51 dB(A) was used as a control (CTRL) condition.
The audio of the beam, trolley and hook were added and coordinated with the crane motions to increase
the realism of the virtual settings. These characteristics were set with the help of a professional crane
operator.
2.3. Cognitive Tasks
The Rey Test, the Verbal Fluency Test, and the Backward Counting Test [17] were chosen as dual
tasks for this study and implemented in the Virtual Reality Overhead Crane Simulator.
The Rey Test (REY) assesses the capacity to recognize correct and false items stored in short-term
verbal memory. Participants were asked to recall as many words as they could from a list of 15 words.
A total of ten lists were used in this experiment. The sum of the right answers was calculated as the
total score.
The Verbal Fluency test (VF) assesses the long-term lexical-semantic store's expansion and recall.
Furthermore, the test engages executive functions because the subject must self-generate the list without
any external cues following the run-in. Participants were instructed to name as many words as they
could with a target letter. Ten letters (S, C, A, P, I, M, D, T, B, F) were chosen for the experiment. The
sum of right words was calculated as the total score.
The Backward Counting test (BC) assesses executive functions and visuo-spatial working memory
resources. Participants were asked to count down by seven from a target number. For the experiment,
ten target numbers were chosen (90, 83, 76, 68, 96, 81, 72, 64, 93, 84). The number of correctly
generated numbers was calculated as the total score.
2.4. Procedure
The test session started with traditional training with slides. The training topic was chosen based on
workplace health and safety laws. The class, which lasted about 10 minutes, covered many types of
cranes as well as the primary risks in-volved with crane use and how to utilize them correctly.
Furthermore, guidance and warnings were offered, as well as the banned manoeuvres.
This first training section was followed by a Go/No Go test session (lasting about 5 minutes) only
to examine inhibitory control and sensitive interference. All subjects successfully overcome this phase.
Then, the subject was asked to wear the head-mounted display and the headphones Sennheiser HD 280
Pro used to playback the soundtracks.
After that, a second training session in virtual reality was carried out. The subject was given the
possibility of learning how to use the virtual reality overhead crane simulator and practicing with the
examples trials of cognitive tasks before beginning the experiment. The subjects could repeat the
training in virtual reality as many times as they wanted. For the cognitive tasks, non-target items (words,
letters, and numbers) were utilized during the training session.
Next, the subject was asked to carry out the manoeuvre test (see Fig. 2) of the overhead crane moving
the load from the loading zone of the virtual industrial building to the target zone while also performing
the three cognitive tasks in the four background noise scenarios (LF, HF, MOD and CTRL).
In the Rey Test, the list of 15 words was visually given for 42 seconds. Then the subject was asked
to recall as many words as possible (30 seconds).
In the Verbal Fluency and Backward Counting tests, the subject was asked to list as many items as
possible (30 seconds) after a cue was graphically presented (a target letter or a number, respectively).
Moreover, the main physiological parameters and manoeuvre errors recorded for the whole time of
the experiment were also monitored. The four background noises (included the control condition) and
the cognitive tasks were presented to the subjects in a Latin Square Balanced Sequences. A total of 12
protocols were administered to the subjects. The experiment lasted about one hour on average.
Figure 2: Test Session involving a preliminary group of 18 unskilled operators.
3. Data analysis and results
Data were entered in separate ANOVAs with Noise conditions (CTRL, LF, HF and MOD) as a four-
level within variable on the scores of the Verbal Fluency, Rey List and Backward counting. The
Newman-Keuls test was used to analyze post-hoc effects (at least p < 0.05). The magnitude of the
significant effects was indicated by partial eta squared (ηp2).
As regards Backward Counting (see Fig. 3), the LF (M=3.94, SD=2.55) and MOD (M=4.06,
SD=2.36) noise conditions were less accurate than the CTRL (M=6.06, SD=2.23) and HF (M=6.11,
SD=3.18) conditions (at least p < 0.05): (F (3,51) = 5.092, p < 0.01, ηp2 = 0.23).
Figure 3: The graph illustrates the mean of correctly generated numbers at the Backward Counting
test as a function of the audio scenarios (control: CTRL; low frequency: LF; high frequency: HF;
Modulated: MOD). Error bars represent the standard deviation.
With regards Verbal Fluency, the MOD (M=8.63, SD=1.36) noise condition showed a better
performance than the CTRL (M=6.72, SD=2.14) and LF (M=7.0, SD=2.95) conditions (at least p<
0.05): F (3,51) =3.335, p< 0.05, ηp2 = 0.16. No differences appeared with Rey list scores (p > 0.05).
4. Conclusions
This study aimed to investigate if and how noise patterns could alter basic cognitive abilities while
overhead crane operations were performed. Besides, this study allowed us to develop and improve a
Virtual Reality Overhead Crane Simulator training protocol ad hoc for inexperienced users by
investigating some critical aspects of handling heavy loads in industrial environments.
The results showed that low frequency (LF) and modulated (MOD) noise conditions negatively
impacted the Backward Counting. This means that the noise conditions can undermine basic visuo-
spatial working memory and executive functions [18]. As for the verbal fluency, the ability to freely
recall long-term lexical-semantic information was penalized by the MOD as compared to the LF and
CTRL noise conditions. Although unexpected, this pattern of results is in line with evidence reporting
that noise conditions characterized by regularity may in some cases improve the free recall of semantic
items, especially when subjected to short-term exposures [19,20]. Finally, as for the Rey list, the
absence of effects would suggest that participants were able to recognize correct and false items in
short-term verbal memory, and that this ability was not altered by the different noise patterns in the
present manipulation.
In conclusion, we found that the training protocol employing the Virtual Reality Overhead Crane
Simulator could represent a valuable candidate for simulating part of the cognitive load required while
performing crane operations under different kinds of noises. Specifically, our findings showed that the
executive functions of inexperienced users resulted particularly impaired. However, the present study
has some limitations. When operators performed cognitive tasks and maneuvered the overhead crane,
they frequently interrupted load handling, discrediting that task, and concentrating only on the cognitive
task. Besides, a more comprehensive cognitive assessment should be made. For these reasons and for
the small sample size of participants involved the results have to be considered preliminary. Future
studies should deepen the effectiveness of virtual reality-based training in multisensory industrial
contexts and extend this protocol to samples with different experience levels.
5. Acknowledgements
This study has been developed as part of a scholarship funded in the framework of the National
Operational Program ESF-ESFR Research and Innovation (PON RI 2014-2020), Action I.1 related to
Innovative Industrial Ph.D. (Project Code-CUP- B25D18000010006).
6. References
[1] M.F. Milazzo, G. Ancione, V. Spasojevic Brkic, D. Vališ, Investigation of crane operation safety
by analyzing main accident causes in: Walls, Revie &Bedford (Eds) Risk, Reliability and Safety:
Innovating Theory and Practice, Taylor & Francis Group, London, 2017, pp. 74-80.
[2] Small Entity Compliance Guide for the Final Rule for Cranes and Derricks in Construction, OSHA
U.S. Department of Labor Occupational Safety and Health Administration, 2014.
[3] V.W.Y. Tam, I.W.H. Fung, Tower crane safety in the construction industry: A Hong Kong study.
Safety Science 49 (2011) 208–215.doi doi.org/10.1016/j.ssci.2010.08.001
[4] A. Zeydabadi, J. Askari, M. Vakili, S.J Mirmohammadi, M. A Ghovveh, A.H. Mehrparvar, The
effect of industrial noise exposure on attention, reaction time, and memory. International Archives
of Occupational and Environmental Health 92 (2019) 111–116. doi 10.1007/s00420-018-1361-0
[5] R. Golmohammadi, E. Darvishi, J. Faradmal, J. Poorolajal, M. Aliabadi, Attention and short-term
memory during occupational noise exposure considering task difficult. Applied Acoustics 158
(2020) pp. doi
[6] Q. T. Le, P. Akeem, C. S. Park, A Social Virtual Reality Based Construction Safety Education
System for Experiential Learning. Journal of Intelligent & Robotic Systems 79 (2015) 487–500.
doi 10.1007/s10846-014-0112-z
[7] R. Sacks, P. Amotz, B. Ronen, Construction safety training using immersive virtual re-ality.”
Construction Management and Economics, 31 (2013) 1005–1017. doi
10.1080/01446193.2013.828844
[8] E. Wyk, R. de Villiers, Virtual Reality Training Applications for the Mining Industry. In
AFRIGRAPH '09, Proceedings of the 6th International Conference on Computer Graphics, Virtual
Reality, Visualisation and Interaction in Africa (2009). doi 10.1145/1503454.1503465
[9] K. Dhalmahapatra, O. Pradhan, S. Das, K. Singh, J. Maiti, Prioritization of human errors in EOT
crane operations and its visualization using virtual simulation, in Proceedings of 4th International
Conference on Recent Advances in Information Technology, IEEE, Dhandad, India, 2018. doi
10.1109/RAIT.2018.8388990
[10] K. Dhalmahapatra, J. Maiti, O. B. Krishna, Assessment of virtual reality-based safety training
simulator for electric overhead crane operations, Safety Science 139 (2021).
doi10.1016/j.ssci.2021.105241
[11] P. Fadda et al., Multidisciplinary study of biological parameters and fatigue evolution in quay crane
operators, Procedia Manufacturing 3 (2015) 3301-3308. doi 10.1016/j.promfg.2015.07.410.
[12] A. K. George, M. L. McLain, K. Bijlani, R. Jayakrishnan, R. R. Bhavani. A Novel Approach for
Training Crane Operators: Serious Game on Crane Simulator, in IEEE Eighth International
Conference on Technology for Education (T4E), Mumbai, India, 2-4 December 2016. Doi
10.1109/T4E.2016.030
[13] Y. Cai, S. L. Goei, W. Trooster, (Eds.). Simulation and serious games for education. Springer,
Singapore, 2017. Doi 10.1007/978-981-10-0861-0
[14] M. Masullo, R.A. Toma, A. Pascale, G. Ruggiero, L. Maffei, Effects of noise on overhead crane
operators' performances under mental fatigue, in a virtual reality crane simulator, in Proceedings
of 2020 International Congress on Noise Control Engineering, Seoul, Corea del Sud, 23-26 August
2020.
[15] M. Masullo, R.A. Toma, A. Pascale, G. Ruggiero, L. Maffei, Research methodology used to
investigate the effects of noise on overhead crane operator’s performances, in D. Sumpor et al.
(eds.), Proceedings of the 8th International Ergonomics Conference, Advances in Intelligent
Systems and Computing 1313, Zagreb, Croatia, 2-5 December 2020. doi 10.1007/978-3-030-
66937-9_25
[16] M. Masullo, G. Ruggiero, D. Alvarez Fernandez, T. Iachini, L. Maffei, Effects of urban noise
variability on cognitive abilities in indoor spaces: Gender differences. Noise & Vibration
Worldwide (2021). doi 10.1177/09574565211030703
[17] C. Barletta-Rodolfi, F. Gasparini, E. Ghidoni, Kit del neuropsicologo italiano., Dynamicon,
Milano (2011).
[18] G. Ruggiero, T. Iachini, The role of vision in the Corsi Block-Tapping task: Evidence from blind
and sighted people. Neuropsychology 24(2010) 674-679. doi 10.1037/a0019594
[19] J.G. Borkowsky, A.L. Benton. O. Spreen, Word fluency and brain damage. Neuropsychologia, 5
(1967): 135–140. doi 10.1016/0028-3932(67)90015-2
[20] G. B. Söderlund, S. Sikström, J.M. Loftesnes, E. J. Sonuga-Barke, The effects of background white
noise on memory performance in inattentive school children. Behavioral and brain functions, 6
(2010) 1-10. doi 10.1186/1744-9081-6-55