CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 The effect of time pressure and task completion on the occurrence of cognitive lockup Ernestina J. A. Schreuder (elianeschreuder@hotmail.com) TNO Human Factors Research Institute, kampweg 5 Soesterberg, 3769 ZG The Netherlands University of Utrecht, Department of Psychology, Heidelberglaan 1, Utrecht 3584 CS The Netherlands Tina Mioch (tina.mioch@tno.nl) TNO Human Factors Research Institute, kampweg 5 Soesterberg, 3769 ZG The Netherlands Abstract reluctant to switch to another task, even if that task has a higher priority (Neerincx, 2003). The following flight Prior studies have suggested that time pressure and task incident illustrates the possible consequences of cognitive completion play a role in the occurrence of cognitive lockup. lockup. During landing, the pilot of flight 401 of Eastern However, supportive evidence is only partial. In this study, we Air Lines (1973) was warned about a problem with the conducted an experiment to investigate how both time pressure and task completion influence the occurrence of cognitive landing gear. To win time, the pilot canceled the landing, lockup, in order to better understand situations that could set the plane in the autopilot mode and started solving the trigger the phenomenon. We found that if people have almost problem with the landing gear. This problem fully completed a task, the probability for cognitive lockup increases. occupied the pilot and multiple warnings about a We also found that the probability for cognitive lockup decreasing altitude (a low-altitude alarm, a remark of the decreases, when people execute tasks for the second time. There air-traffic controller) were ignored. As a consequence, the was no effect of time pressure or an interaction effect found plane crashed, resulting in the death of most people on between task completion and time pressure. The results provide board. further support for the explanation that cognitive lockup up is Experimental studies exist where cognitive the result of a decision making bias and that this bias could be triggered by the perception that a task is almost complete. lockup was manifested in the data (Moray and Rotenberg, 1989; Kerstholt, Passenier, Houttuin and Schuffel, 1996), however, an explanation for the occurrence of the Introduction phenomenon was not given. The following literature overview provides accounts for the occurrence of the This study is about an inescapable part of action, phenomenon. something all human beings experience to a greater or lesser extent: human error. Human errors can happen in Reduced situational awareness as trigger for everyday situations with only limited consequences. cognitive lockup However, errors can also happen in high-performance environments like in aviation, where they can have A popular concept (Meij, 2004) is the idea that a reduced tremendous effects and be life threatening. For instance, situational awareness (SA) might cause cognitive lockup. when pilots forget to enable their landing gear when Kerstholt and Passenier (2000) argued that if operators landing and as a result crash. Hence, it is important to become less aware of the actual situation, for instance, investigate why human errors in aviation are made and due to automation, they may not be able to understand the how errors can be avoided. links between the various subsystems they have to In the past, several cognitive explanations and control. If operators lack the knowledge of the underlying theories have been proposed to understand why pilots systems cognitive lockup is more likely to occur. deviate from normative activities (e.g. Wickens and Kerstholt and Passenier suggested that knowledge of the Hollands, 2000; Dekker, 2003). The European project underlying systems is important in order to increase SA HUMAN (www.human.aero) strives to pave a way for and prevent cognitive lockup. making this knowledge readily available to designers of Jones and Endsley (1996) investigated flight new cockpit systems, in order to be able to design accidents caused by a reduced SA. They found that a cockpits that prevent pilots from making errors. They great part of the flight accidents was due to a failure to identified cognitive lockup to be among the most relevant monitor or observe relevant data that were clearly present mechanisms for modern and future cockpit human in the situation. This type of accidents could also have machine interfaces. Cognitive lockup is the tendency to been the result of cognitive lockup, as important tasks deal with disturbances sequentially (Moray and were that were triggered while dealing with another Rotenberg, 1989). As a result of cognitive lockup problem, were ignored. Therefore, it could be suggested operators are inclined to focus on the current task and are that cognitive lockup is triggered by a failure to monitor 63 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 the environment. However, Meij (2004) found in his that people have flexible control over the course of study on cognitive lockup, that cognitive lockup could secondary task processing stages. They argued that the not be explained by a neglect of environment. Hence, a fact that switching costs arise is not due to a cognitive reduced SA and specifically the failure to monitor the bottleneck but is rather optional and strategic. The environment seem not to be an underlying mechanism of reconfiguration to another task takes time and thus, cognitive lockup. switching costs arise. Meij (2004) proposed that people might decide High cognitive workload as explanation for to stick to their current task because the switching costs cognitive lockup that accompany task switching are perceived as too high. Cognitive workload refers to the information processing He found that cognitive lockup was reduced when it was demands imposed by the performance of cognitive tasks obvious that the benefits of a switch to another task were (Johnson and Proctor, 2004). In order to predict the higher than the costs of a switch. He argued that the cognitive load of a specific task, Neerincx (2003) participants were biased in their decision-making process, developed the cognitive task load (CTL) theory. This as the costs of switching to another task had to be theory proposed three underlying factors of cognitive task disproportionally low before participants decided to load: (1) time occupied, (2) number of task set switches, abandon the current task. Although the results showed which is the number of active tasks in execution or that the tendency for cognitive lockup was considerably planned to do, and (3) level of information processing. reduced when the costs to switch were low, the tendency Neerincx (2003) suggested that cognitive lockup would for cognitive lockup was still present. Therefore, Meij occur when time occupied and the number of task set suggested that besides switching costs, other factors that switches are high. Grootjen, Neerincx and Veltman, trigger cognitive lockup are involved. (2006) conducted experiments in order to validate the CTL theory. In these real-life experiments participants A decision making bias as trigger for cognitive had to deal with emergencies that appeared on a ship. lockup They found that when all three factors were high people A decision making bias refers to the fact that decision experienced cognitive overload; they did not know what making can be influenced by a prejudice or 'one-sided' to do. However, no evidence of cognitive lockup was perspective. A bias can be unconscious or conscious in found. Furthermore, Grootjen et al. (2006) found that in awareness. Meij (2004) believed that cognitive lockup is the overload situation, participants switched much more due to a decision making bias. When dealing with a task between tasks than in the optimal strategy. This result is people decide to switch or not to switch to another task likely to indicate that a high workload does not influence when triggered. This decision might be biased due to for the occurrence of cognitive lockup. instance a misperception of expected benefits. As a In the same vein, Meij (2004) investigated result, people could decide not to deal with an additional whether a lack of cognitive resources could be an more urgent task until the ongoing task is dealt with. To explanation for cognitive lockup. He argued that tasks find out whether cognitive lockup results from a decision that require a more complex diagnosis process are making bias, Meij (2004) conducted several experiments expected to demand more cognitive resources and thus with a fire control task. In this task participants had to cause a higher cognitive workload and might cause extinguish fires on a ship in a computer simulation. When cognitive lockup. He found, however, that the level of a fire appeared the participants had to detect the fire by complexity of information processing did not affect the clicking on the fire. After detection, participants had to degree of cognitive lockup. Hence, Meij (2004) indicated extinguish the fire by selecting the right treatment. He that cognitive lockup is not caused by the fact that people used this task because in his first experiment he lack the cognitive resources to switch. This result successfully demonstrated that cognitive lockup could be underscores that it is likely that a high workload does not found using this task. cause cognitive lockup. Meij (2004) proposed that both task completion and prior investments, such as money, time and effort, Too high switching costs as explanation for might bias the decision to switch to another task. Meij cognitive lockup found that when prior investments are high and the task When people switch between tasks, people are slower was near completion (high task completion), the and less accurate than when they repeat tasks (Jersild, probability for cognitive lockup increased. Remarkably, 1927; Monsell, 2003) and these switching costs have he also found that when prior investments are high and been attributed to a variety of processes. Pashler (1994) task completion is low, people have the tendency to suggested that switching costs arise because of a abandon their task. Hence, the probability for cognitive cognitive bottleneck to process or select information. lockup decreased. Meij argued that in the high prior This means that when a cognitive process is devoted to a investments condition perceived time pressure may be primary task, this process can not start for a second task. higher than in the low prior investments condition. This This second task has to wait, yielding switching costs. is because the available time in relation to invested time Schumacher (1999) and Meyer and Kieras (1997a, is lower in the high prior investments condition. 1997b) argued that switching costs arise due to an Therefore, he attributed the effect of prior investments to executive control mechanism. They proposed a class of the perception of time pressure. He suggested that when adaptive executive control models in which it is assumed time pressure is perceived as high and the ongoing task is 64 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 almost completed, people are more likely to stick to the and tend to employ more effortful systematic information ongoing task than in situations where time pressure is processing that gives serious considerations to all high and the ongoing task needs considerable activities in possible alternative solutions for a task (Kelly and order to complete it. Thus, the results of Meij’s Loving, 2004). Interestingly, DeDonno and Demaree experiments (2004) are likely to indicate that cognitive (2008) found that the mere perception of time pressure as lockup is due to a decision making bias. This decision well as real time pressure impair performance. making bias seems to be triggered when time pressure Thus, time pressure can trigger heuristic information and task completion are high. processing that make people focus on an ongoing task (van der Kleij, 2009; De Dreu, 2003; Durham et al., Current study: the effect of time pressure and 2000; Karau and Kelly, 1992; Kelly and Loving, 2004). task completion on cognitive lockup As a result, time pressure might influence the tendency to From the literature overview it seems that cognitive stick to the ongoing task and influence the occurrence of lockup is the result of a biased (un)conscious decision to cognitive lockup. However, in situations where time 1 focus on the current task and ignore others. Time pressure was high Grootjen et al. (2006) found a high pressure and task completion seem to influence this cognitive workload, but they found no relation to biased decision. The aim of this study is to investigate cognitive lockup. Therefore, we expect that the effect of how both time pressure and task completion influence the time pressure alone is not strong enough to capture occurrence of cognitive lockup. Therefore, this study people in their current task. We propose the following extends the results found by Meij (2004). Furthermore, hypothesis: the aim is to identify critical situations in cockpit environments that allow for designing cockpit systems 1. Time pressure has no effect on cognitive lockup. that help pilots avoid critical situations and decrease the That means, in case people deal with a task, and probability for cognitive lockup. In the following another more urgent task is triggered, people subsections we translate the research question into switch to the more urgent task just as often under hypotheses. time pressure as when there is no time pressure. Time pressure. Time pressure is the perception that time Task completion. Task completion literature (Garland is scarce. According to Beevis (1999) people experience and Colon, 1993; Boehne and Paese, 2000; Humphrey time pressure when the time required to execute tasks is S.E., Moon, H., Conlon, D.E., Hofmann D.A., 2004) more than 70% of the total time available for the tasks. shows that people have the tendency to complete a task Beevis (1999) suggested that people experience high time even if it is economically unwise to do so. Garland and pressure when 85% of the available time is required to Colon (1993) and Boehne and Paese (2000) found that execute the tasks. In this case performance is often this tendency is strongest when 90% of the task is impaired in that some tasks are not (well) executed. Other completed compared to 10% or 50% completion of a researchers (e.g. van der Kleij, 2009; De Dreu, 2003; task. Meij (2004) found an effect of task completion on Durham et al., 2000; Karau and Kelly, 1992; Kelly and cognitive lockup. People tend to complete a task when Loving, 2004) indicated the following consequences of they are almost finished (high task completion) even time pressure. At the individual level, time pressure leads when a more urgent task is triggered. We expect that the to (1) faster performance rates, because people stop results of Meij (2004) are replicated in this study. considering multiple alternatives, (2) lower performance Therefore, we propose the following hypotheses: quality, due to the engagement in superficial rather than thorough and systematic processing of information, and 2. Task completion has an effect on cognitive (3) more heuristic information processing, as a result of lockup. refraining from critical probing of a given seemingly a. When task completion is high, the probability adequate solution or judgment. At group level, increasing for cognitive lockup increases. levels of time pressure narrows team members’ focus on That means, in case people deal with a task, and a limited range of task-salient cues in both team another more urgent task is triggered, people interaction patterns and team task performance. This tend to stick to the current task, when they have narrow focus of attention that often manifests as a almost completed this task. restricted information exchange is due to a filtering b. When task completion is low, the probability for process (Kelly and Loving, 2004). Groups working under cognitive lockup decreases. That means, in case time pressure attend to all of the information available people deal with a task, and another more urgent but then selectively discuss only information that seems task is triggered, people tend to switch to the especially relevant (Kelly and Loving, 2004). They also more urgent task when the first task is not nearly found that under high time pressure group members see completed. task completion as their main interaction objective, and the group attempts to reach consensus and complete the task as quickly as possible, but at the sacrifice of quality. Groups under mild or no time pressure can, in contrast, 1 consider a wider set of task features, devote their In the research of Grootjen et al. (2006) time pressure was resources to performing on the task as well as possible, high when the percentage of the available time for a task that people are occupied with the task was high. 65 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 Interaction between time pressure and task Apparatus and material completion. Meij (2004) suggested that when task The experiment included two laptop computers with completion and time pressure are high the tendency for headphones. Java software was installed on the cognitive lockup increases. When time pressure is high computers to run the experimental task and a training but task completion is low, the tendency for cognitive session. Before the experiment, participants received a lockup decreases. In fact, he investigated the interaction hardcopy manual printed on paper with A4 format. effect between prior investments and task completion and During the experiment, participants could use a question- attributed the effect of prior investments to the perception tree (see Figure 2) printed on paper with A4 format for of time pressure. Thus, Meij never investigated the effect reference purposes. of time pressure and task completion. Therefore, in this study, we test whether high time pressure and high task Procedure completion indeed increase the probability for cognitive lockup and that high time pressure and low task The experiment was conducted at TNO in a computer completion decrease this probability, as suggested by room, at the University of Utrecht in a laboratory, and in Meij. We propose the following hypotheses. a private setting with the use of laptops. The experiment took ca. 30 minutes per participant: 15 minutes for a training (including test scenarios) and 15 minutes for the 3. There is an interaction effect between task experiment. completion and time pressure on cognitive The participants received an information letter lockup. upfront the training and an informed consent document a. When time pressure is high and task completion after the training. The aim of the training was to is high, the probability for cognitive lockup is familiarize the participants with the experimental task. highest compared to all other conditions. That The training involved reading the training manual, which means, in case people deal with a task, and was accompanied by a verbal instruction and executing another more urgent task is triggered, people test scenarios on the computer. After the training the tend to finish the first task before switching to participants were asked whether they felt comfortable the more urgent when they feel time pressure with performing the task. If so, the experiment was and have almost completed the first task. started. If not, questions could be asked and the test b. When time pressure is high and task completion scenarios could be done again until the participant was is low, the probability for cognitive lockup is comfortable with executing the task lowest compared to all other conditions. That The experimental task was a computer means, in case people deal with a task, and simulation in which participants had to fight fires on a another more urgent task is triggered, people ship. The fire-fighting software was chosen because Meij tend to switch to the urgent task before (2004) already demonstrated that with this fire-fighting executing the first task when they feel time task cognitive lockup could be found. pressure, but still need to complete many stages to complete the first task. Experimental task. In the experimental task participants This study extends the study conducted by Meij (2004) as had to fight fires on a ship. Two types of fires existed: we expect to provide further evidence for the explanation • normal fires, which were indicated by a red that cognitive lockup is caused by a decision making triangle bias, and that this bias could be triggered by time • urgent fires, which fires were indicated by a blue pressure and task completion. triangle in a yellow background Next to the fire type, fires had specific features. For Method example, a fire could be an oil fire, a fire could be life threatening, injured people could be involved, and/or Participants smoke could trouble the sight of the firemen. Therefore, The experiment counted 46 participants. The participants each fire required a specific action based on the fire’s consisted of: specific features. To find out the fire specifics, • TNO trainees/employees (15) participants could ask four predefined questions. These • students of the University of Utrecht (20) questions appeared as buttons on the screen. To ask a • (ex)members of the Hockey Club Rotterdam (6) question, participants had to press the question button. • other (5) Figure 1 shows the screen that was visible to the All participants were experienced computer users and participants once a fire was present. When a question was most of them were highly educated. They were all asked, the system closed for four seconds to answer the between the age of 18 and 35 years old. Psychology question with Y (Yes) or N (No) for a normal fire. In case students received course credits for participation in the of an urgent fire the system closed for one second to experiment. All participants could win 20 Euro when answer the question. This was because an urgent fire was they had the highest score in the experiment. more dangerous for the ship and needed quick handling. Please note that when the system was closed nothing could be done. Based on the answers generated by the system, participants could select the appropriate action to 66 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 extinguish the fire. Figure 2 shows the question-tree be left to extinguish the second fire. This would give the which indicates the appropriate action. Seven predefined incentive to switch to the second fire. As a result the actions could be chosen. The action buttons also appeared switch incentives in these situations would not be on the bottom of the screen, once a fire was present (see comparable. This problem was solved by not showing the Figure 1). An appropriate action extinguished the fire; a time available for the second fire. wrong action shut down the system for seven seconds. If participants extinguished a fire in time they Thereafter, a new action could be selected, if time could win points. However, if they did not extinguish the allowed it. fire in time, points were deducted and a burn down was the result (see Table 1). This was done to emphasize the fact that an urgent fire was more urgent than a normal fire, as more points could be won or lost by respectively extinguishing or missing an urgent fire. If one fire burned down, another fire could still be extinguished if time allowed it. In the test scenarios (see scenarios) the second fire was always an urgent fire. The time was set in such a way that if a participant finished the first fire before switching to the urgent fire, the urgent fire would burn down. This was also done to emphasize the fact that an urgent fire was urgent. If it was not handled quickly it burned down. This also meant that if participants suffered from cognitive lockup (finishing the first fire instead of extinguishing the urgent fire first), they would have had a lower score than participants who did not suffer cognitive lockup. To ensure the motivation of the participants a reward of 20 Euro was promised to the participant with Figure 1: Screen of the ship visible to participants once a the highest score. Participants only saw their score at the fire was present. end of the experiment. Are there any injuries? Yes No Table 1: Fire-fighting scores. Is the fire an oil fire? Are there human lives at stake? Action Points Yes No Yes No Extinguish normal fire 1 Is there any smoke output? Action: 2 Is the fire an oil fire? Is the fire an oil fire? Extinguish urgent fire 3 Yes No Yes No Yes No Burn down normal fire -1 Action: 1 Action: 3 Action: 3 Action: 4 Action: 5 Is there any smoke output? Burn down urgent fire -3 Yes No Action 1: Send a large injury team Experimental design Action: 6 Action: 7 Action 2: Transport casualties The main goal of this experiment was to investigate how Action 3: Send a small injury team Action 4: Extinguish with foam time pressure and task completion influence the Action 5: Cool cabin occurrence of cognitive lockup. In order to enhance the Action 6: Remove smoke sensitiveness to find cognitive lockup, we operationalized Action 7: Close doors cognitive lockup in two ways. In the first definition cognitive lockup was only found when participants did Figure 2: Question-tree and relevant actions to extinguish not switch to the urgent fire, when dealing with the first fires. fire. In the second definition cognitive lockup was found Participants knew how much time they had to when participants significantly delayed their switch to the extinguish the first fire, as this was indicated by the timer urgent fire. Therefore, the second analysis was more at the bottom of the screen. The time to stop a second fire sensitive to find cognitive lockup than the first analysis. was not indicated. This was done to make the decision to switch to the second fire similar for the different test 1) Cognitive lockup 1 was defined as completing the scenarios (see scenarios). In case the time available first fire before extinguishing the second more urgent would be shown for the second fire as well as, fire. Cognitive lockup 1 was measured as the mean participants could depend their decision to switch on the percentage of scenarios in which cognitive lockup time available for both fires. In the high task completion occur. condition relatively little time would be left for the first 2) Cognitive lockup 2 was defined as delayed switching fire and therefore relatively more time would be left to to the second more urgent task once presented, while extinguish the second fire, which would give the executing a first task. Cognitive lockup 2 was incentive to finish the first fire. In the low task measured as the action time to switch to the urgent completion condition relatively much time would be left fire, once present (a significantly delayed action time to extinguish the first fire, and relatively less time would indicates cognitive lockup). 67 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 fire first, but in that case the urgent fire would burn down In order to investigate the influence of time pressure and as explained before. task completion on cognitive lockup, time pressure and In the low time pressure condition the available task completion were manipulated within-subjects. time to extinguish the first fire was 55 seconds for all fires. In this condition time pressure was ≤ 40%, as 40% Task completion. Task completion was defined as the or less of the available time was required for the task. percentage of the total number of stages of a task that The available time of 55 seconds was chosen because have been completed. People have the tendency to stick perceived time pressure of the first fire could be to their current task when 90% or more of the total stages influenced by the appearance of the second more urgent of a task have been completed (Boehne and Pease, 2000; fire. To make sure that participants would perceive little Garland and Colon, 1993). We investigated whether time pressure in the low time pressure condition, 55 people refrain from switching to an urgent second task, seconds allowed the participants to start with the first when they have almost completed a first task. Therefore, fire, and when the urgent fire appeared to switch to the task completion was manipulated by the onset of the urgent fire and once the urgent fire was extinguished, to urgent fire. The onset depended on the number of switch to the first fire again and extinguish the first fire. questions that had been asked in order to extinguish the Thus, in the low time pressure scenarios the maximum first fire. In the high task completion condition the urgent score of four points could be obtained. Table 2 fire appeared after three questions had been asked. In this summarizes the test conditions in terms of task condition task completion of the first task was 75%, as completion and time pressure. 75% of the total stages of the task had been completed (three questions answered out of four). This percentage Table 2: Test conditions. was chosen because this was the pre-final stage for extinguishing the first fire, and closest to 90% task Test Time pressure Task completion completion as mentioned by Boehne and Pease (2000) condition (%) (%) and Garland and Colon (1993). In the low task Low-Low ≤ 40 25 completion condition the urgent fire appeared after one Low- High ≤ 40 75 question had been asked. In this case task completion was High-Low ≥ 88 25 25%, as 25% of the total stages of the first task had been High-High ≥ 88 75 completed (one question answered out of four). Scenarios. The experiment consisted of 25 scenarios. Time pressure. Time pressure was defined as the The scenarios included 8 test scenarios (2 times all test percentage of the available time to execute a task that is conditions) and 17 irrelevant scenarios. The irrelevant required to execute the task. People experience time scenarios were designed in order to accomplish pressure when 70% or more of the available time is uncertainty, so the participants would not understand the required for the task (Beevis, 1999). People experience test scenarios. The test scenarios can be described as high time pressure when 85% of the available time is follows: required for the task (Beevis, 1999). The average time to 1. In the scenario where time pressure was low and task fight a fire depended on the number of questions that had completion was low, participants had 55 seconds to to be asked to extinguish a fire. Based on the pilot results fight the first fire. The urgent fire appeared when it appeared that for a normal fire the time required to they had asked one question of the first fire. The extinguish a fire based on four questions was 22,1 urgent fire needed to be extinguished in 17 seconds. seconds; based on three questions this was 18,6 seconds If participants decided to extinguish the first fire and based on two questions this was 14,3 seconds. We first, the available time allowed participants to start investigated whether time pressure on a first task would with the urgent fire, after they had extinguished the refrain people from switching to a second more urgent first fire, but they would never be able to extinguish task. Therefore, time pressure was manipulated. This was it. In this way the participants would not be achieved by increasing or decreasing the time available demotivated, which would be the case when the for fighting the first fire. urgent fire had already burned down, while still In the high time pressure condition the available fighting the first fire. time to extinguish the first fire was 25 seconds for fires 2. In the scenario where time pressure was low and task that needed four questions and 20 seconds for fires that completion was high, participants had 55 seconds to needed three questions. Fires that needed two questions fight the first fire. The urgent fire appeared when were not present in the test scenarios (see scenarios). In they had asked three questions of the first fire. The this way time pressure was ≥ 88%, as 88% or more of the urgent fire needed to be extinguished in 13 seconds. available time was required for the task. As a result, in In this way it was impossible to extinguish the first the high time pressure and high task completion scenario fire first, and afterwards extinguish the urgent fire. the available time to extinguish the first fire was almost 3. In the scenario where time pressure was high and over when the urgent fire appeared. Therefore, task completion was low, participants had 20 participants had to choose for the urgent fire at the cost of seconds to fight the first fire. This was because the a burn down of the first fire in this condition. Thus, they first fire could be extinguished after three questions. could never obtain the total payoff of four points in this The urgent fire appeared when they had asked one scenario. They could also choose to extinguish the first 68 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 question of the first fire. Like scenario 1, the urgent before the participants switched to the more urgent fire needed to be extinguished in 17 seconds. second fire. 4. In the scenario where time pressure was high and 70 task completion was high, participants had 25 60 seconds to fight the first fire. This was because the 50 first fire could only be extinguished after four CL 1 (%) 40 questions. The urgent fire appeared when they had 30 asked three questions of the first fire. Like scenario 2 the urgent fire needed to be extinguished in 13 20 Time Pressure seconds. 10 Low 0 High The scenarios (test and irrelevant scenarios) were -10 Low High presented in random order to avoid order effects. Only Task Completion the test scenarios were analyzed. Figure 3: The mean percentage of scenarios in which Statistical design CL1 was found. The experimental design was a repeated measures design, as each test condition consisted of two scenarios. Thus, The main effect of task completion on CL1 was the participants received all test conditions twice. significant, F(1,44) = 36.857, p < .001. This means that Therefore, we used a 2x2x2 repeated measures ANOVA the percentage of scenarios in which CL1 was found was with attempts, time pressure and task completion as significantly higher in the high condition (Mean = 43%) factors to analyze the data. compared to the low condition (Mean = 11%). In other Before the experiment was executed, we words when participants had almost extinguished the first conducted a power analysis in order to examine the fire (one question to go to find out the correct action) number of participants needed for the experimental more people finished the first fire before switching to the design. The power analysis for a factorial ANOVA urgent fire than when the participants still had three suggested a sample size (N) of 45 to achieve a power of questions to go to find out the correct action. There was 0.80 for detecting a medium effect size (0.26) and alpha no effect of time pressure or an interaction effect found set at 0.05. Hence, this design required a sample size of between task completion and time pressure on CL1. 45 participants to be able to conduct further statistical Next to these results, a significant main effect was found analysis. Based on this result, 46 participants were for the factor attempts (not shown in Figure 3). The recruited. participants received all test conditions twice. The results show that the mean percentage of scenarios in which CL1 Results was found was significantly higher in the first attempt (Mean = 34%) compared to the second attempt (Mean = Sample data 21%) F(1,44) = 10.203 p < .003. This indicates a learning effect. In the experiment 46 cases with two repeated measures Figure 4 shows the results for attempt 1 and 2 were recorded. Three records were removed as one separately. For both attempt 1 and 2, a significant main participant did not follow the experiment instruction effect for task completion was found F(1,44 ) = 26.362, p correctly and pressed action buttons without asking < .001 and F(1,44) = 24.750, p < .001, respectively. No questions. Therefore, this data could not be analyzed as effect was found for time pressure. In addition, no the urgent fire was not triggered. Table 3 shows the interaction effect between time pressure and task number of valid records per test condition. completion was found. ATTEMPT: 1 ATTEMPT: 2 Table 3: Number of valid records per test condition. 80 70 Test condition Number of records Time Task Attempt 1 Attempt 2 60 Pressure Completion 50 CL 1 (%) Low Low 46 46 40 Low High 45 45 30 High Low 46 46 20 High High 45 46 10 Time Pressure Low Cognitive lockup 1 0 High Figure 3 shows the mean percentage of scenarios in -10 Low High Low High which cognitive lockup 1 (CL1) was found, taking time Task Completion Task Completion pressure and task completion into account. In other words, the Figure shows the mean percentage of Figure 4: The mean percentage of scenarios in which scenarios in which the normal fire was extinguished CL1 was found for attempt 1 and 2. 69 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 Cognitive lockup 2 ATTEMPT: 1 ATTEMPT: 2 Figure 5 shows the mean reaction times of participants to 9 switch to the second more urgent fire, while fighting the 8 first fire. Cognitive lockup 2 (CL2) was found when the reaction times were significantly longer in a specific 7 CL 2 (seconds) condition. 6 8,0 5 7,5 4 7,0 6,5 CL 2 (seconds) 3 Time Pressure 6,0 Low 5,5 2 5,0 High 4,5 1 Low High Low High 4,0 Task Completion Task Completion Time Pressure 3,5 3,0 Low Figure 6: Time needed in seconds to switch to the urgent 2,5 High fire for attempt 1 and 2. 2,0 Low High Task Completion General Discussions Figure 5: Time needed in seconds to switch to the urgent fire. The main goal of this experiment was to investigate how both time pressure and task completion influence the The main effect of task completion on CL2 was occurrence of cognitive lockup. Firstly, we hypothesized significant, F(1,44) = 15.182, p < .001. This means that that time pressure alone would not have an effect on the participants needed more time to switch to the urgent cognitive lockup (hypothesis 1). The results of the fire in the high condition (Mean = 5,9 seconds) compared experiment support this hypothesis. In case people deal to the low condition (Mean = 4,1 seconds). In other with a task, and another more urgent task is triggered, words, when participants had already asked three people switch to the more urgent task just as often and questions of the first fire, it took significantly longer to just as fast under time pressure as when there is no time switch to the urgent fire compared to the situation where pressure. This result implies that although time pressure they had only asked one question. The average time to can trigger heuristic information processing and thereby ‘switch’ to the first fire, once it was present, was 1,1 influence decision making (e.g. van der Kleij, 2009; De second. There was no effect of time pressure or an Dreu, 2003; Durham et al., 2000; Karau and Kelly, 1992; interaction effect found between task completion and Kelly and Loving, 2004), people are able to assess the time pressure on CL2. Next to these results, a significant priority of different tasks while dealing with a task, and main effect was found for the factor attempt (not shown switch to the most important task if necessary. They are in Figure 5). The participants received all test conditions not captured in their current task when facing time twice. The results show that it took longer to switch to pressure. the urgent fire in the first attempt (Mean = 5,8 seconds) We should mention that in the high time compared to the second attempt (Mean = 4,1 seconds; pressure and high task completion condition there was an F(1,44) = 15,444 p < .001). This indicates a learning incentive to extinguish the normal fire first, before effect. switching to the urgent fire. As a result, this incentive Figure 6 shows the results for attempt 1 and stimulated the chance to find cognitive lockup. The attempt 2 separately. For both attempt 1 and 2, a incentive existed because a choice had to be made significant main effect for task completion was found between the normal fire and the urgent fire. While F(1,44) = 5.922, p < .019 and F(1,44 )= 14.404, p < .001 dealing with the normal fire, participants could see that if respectively. No effect was found for time pressure. In they switched to the urgent fire, they would miss the addition, no interaction effect between time pressure and normal fire. However, if they extinguished the normal task completion was found. fire, there was only a chance that they would miss the urgent fire. The participants did not know that the maximum number of points could never be obtained, as they would indeed miss the urgent fire, when they extinguished the normal fire first. Participants with a risk avoiding strategy would switch to the urgent fire as this fire could be missed if they did not switch immediately (two points versus minus two points). However, participants that were very confident with extinguishing the fires and who had a risky result maximizing strategy could try to extinguish both fires instead of one (four points versus two points). They would extinguish the normal fire first. In this way, there was an incentive to finish the normal fire first in the high task completion and 70 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 high time pressure condition. While this incentive was reported by Boehne and Pease (2000) and Garland and present, nevertheless participants decided to switch to the Colon (1993). In their experiments they found the urgent fire in the high time pressure and high task tendency to complete a task when the task has already completion condition as often as in the low time pressure been completed for 90%. The present study shows that and high task completion condition. This underscores our this tendency is already present when a task has been finding that time pressure has no influence on the completed for 75%. In our experimental setup this was occurrence of cognitive lockup. achieved when three task stages had been completed out Furthermore, we should notice that we used a of a maximum number of four stages. It could be argued static time deadline to manipulate time pressure. Other that participants perceived a higher task completion ways exist to induce time pressure. For example, a more percentage as only one stage was still required to dynamic task can be used in which deadlines evolve with complete the task. different speed. This type of dynamic time pressure might These results imply that the perception that a have different effects on performance (Kerstholt and task is almost competed could lead to critical situations Willems, 1993). As a consequence, the results of this when another more urgent task is triggered. This urgent study only apply to settings in which time pressure is task might be ignored as a result of cognitive lockup. In induced by a static deadline. Further research is needed to order to avoid cognitive lockup we believe that the analyze the effect on behavior when time pressure is tendency to complete a task when it is almost completed dynamic. should be broken. For instance, this might be done by Secondly, we hypothesized that task completion altering the perception that a task is almost completed or would have an effect on cognitive lockup (hypothesis 2). by unlearning this tendency. Further research is needed to The results of the experiment support this hypothesis. investigate how to break the tendency to complete a task The results show that there is a main effect of task when it has almost been completed and a more urgent completion on cognitive lockup (CL1 and CL2). People task is triggered. that have almost completed a task tend to finish this task Finally, according to the results of Meij (2004) even when a more urgent task is triggered. In other we hypothesized that there would be an interaction effect words, when task completion is high the probability for between time pressure and task completion (hypothesis cognitive lockup increases (hypotheses 2a). People that 3). The results do not support this hypothesis as the still need to complete many stages before a task is interaction effect between time pressure and task completed tend to switch to the more urgent task, when completion on cognitive lockup (CL1 and CL2) was not triggered. Thus, when task completion is low the significant. The result implies that time pressure does not probability for cognitive lockup decreases (hypotheses enhance the task completion effect, as expected. Thus, 2b). These results were found despite a learning effect. when task completion is high the probability for We found that cognitive lockup was less present during cognitive lockup is not increased when people face time the second attempt of a test scenario compared to the first pressure. The present study shows that the interaction attempt. We believe that this was due to a learning effect. effect found by Meij (2004) between prior investments People learned from the feedback they received in the and task completion on cognitive lockup cannot be first attempt of a scenario, and if needed they changed explained by the perception of time pressure when prior their strategy in the second attempt. However, the task investments are high, as he suggested. An alternative completion effect remained significant in the second explanation cannot be given as prior investments were attempt. Thus, although participants lost points when they not investigated in this experiment. Further research completed the first fire and as a result missed the urgent should be done to find an explanation for the effect of fire in the first attempt of the high task completion prior investments on cognitive lockup and why this factor scenario, they did not change their strategy when the interacts with task completion. scenario was executed again. The results of this study provide further support It could be argued that the participants might not for the explanation that cognitive lockup up is the result have perceived the urge of the urgent fires and as a result of a decision making bias and that this bias could be stayed with the normal fires. Although the urgent fires triggered by the perception that a task is almost had a very different icon, behaved differently (system completed. This has important implications for the closure of one second instead of four after a button had designs of cockpits, as it indicates that decision support been pressed) and generated more (less) points when tools seem more important in reducing the probability for extinguished (burned down) than normal fires, the cognitive lockup than, for example, tools that reduce categorization might not have been meaningful enough cognitive workload. The decision support tool should for the participants. However, participants showed in the assist pilots to focus on the most urgent task. However, a high time pressure and high task completion condition, in critical situation exists when the pilot has almost which they had to choose between the urgent and normal completed a task and a more urgent task is triggered. The fire, that they switched to the urgent fire as often as in the decision support tool might be ignored, as a result of low time pressure and high task completion condition. cognitive lockup. To avoid critical situations it is Such behavior would not have been expected when important that this decision support tool also helps pilots categorization and consequences were not clear. to break the tendency to complete a task when it is almost This study replicates the results of Meij (2004) finished and another more urgent task is triggered. In this as he also found an effect of task completion on cognitive way pilots can act appropriately and deal with the most lockup. Furthermore, this finding extends the results urgent task. 71 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 Training Acknowledgments Although the experimental design was not specifically The work described in this paper is funded by the built to investigate an effect of training, a significant European Commission in the 7th Framework Programme, learning effect was found. Participants showed less Transportation under the number FP7 – 211988. cognitive lockup (CL1 and CL2) the second time they executed the test scenarios, compared to the first time. As References mentioned before, we believe that participants learned Beevis, D., Bost, R., Doering, B., Nordø, E., Oberman, from the feedback they received in the first attempt of a F., Papin, J., et al. (1999). Analysis techniques for man- scenario. When participants decided to stick to the first machine system design, crew system ergonomics. Ohio. fire, it resulted in a burn down of the urgent fire. Their Boehne, D.M. & Paese, P.W. (2000). Deciding whether payoff in that scenario would have been minus two. As to complete or terminate an unfinished project: a strong participants executed the test scenarios twice, they test of the project completion hypothesis. adjusted their strategy in order to improve their payoff. Organizational Behavior and Human Decision This finding supports the results reported by Kerstholt Processes, 2, 178-194. and Passenier (2000). They suggested that if people DeDonno, M.A., Demaree, H.A. (2008). Perceived time understood the underlying system, cognitive lockup was pressure and the Iowa Gambling Task. Judgment and less likely to occur. Training might therefore be a factor Decision Making, Vol.3, No. 8 December 2008, pp that influences cognitive lockup as it increases the 636-640. knowledge of a system and thereby reduces the De Dreu, C.K.W. (2003). Time pressure and closing of probability for cognitive lockup. For the design of the mind in negotiation. Organizational Behavior and cockpits this implies that the decision to automate Human Decision Processes 91, 280–295. Dekker, S. (2003). Failure to adapt or adoptions that fail: processes should be done carefully. This is because contrasting models on procedures and safety. Applied automation might decrease the understanding of Ergonomics, 34(3), 233–238. underlying systems as operators are not involved Durham, C.C., Locke, E.A., Poon, J.M.L., McLeod, P.L. anymore in the normal process (Wickens and Hollands, (2000). Effects of group goals and time pressure on 2000; Kerstholt and Passenier, 2000). As a result, the group efficacy, information-seeking strategy, and likelihood for cognitive lockup might increase. Further performance. Human Performance 13, 115–138. research should be done to provide evidence for the Conlon, D. E., & Garland, H. (1993). The role of project suggested effect of training on the occurrence of completion information in resource allocation cognitive lockup. decisions. Academy of Management Journal, 36, 402– 413. Task domain and participants Groom et al. (2002). An introduction to cognitive It can be argued that the experiment was conducted in a psychology processes and disorders. Psychology Press specific task domain (fire-fighting task) and with a Ltd, Hove, UK. specific set of participants and that generalizing the Grootjen, M., Neerincx, M.A., Veltman J.A. (2006). results should be done carefully. We assume that Cognitive task load in a naval ship control centre: from cognitive lockup is a general cognitive mechanism or identification to prediction. Ergonomics, Vol. 49, Nos. heuristic that is domain independent and can happen to 12-13, 10-22 October 2006, 1238-1264. every human being. This assumption is based on the fact Jersild, A. T. (1927). Mental set and shift. Archives of that heuristics that are used to solve well defined Psychology, 14 (Whole No. 89). problems are general-purpose or domain independent Johnson A., Proctor R.W. (2004). Attention: Theory and heuristics, in that they can be applied to a wide range of Practice. Sage Publication, Thousand Oaks. situations or domains and do not involve specific Jones, D.G., and Endsley, M.R. (1996). Sources of capabilities (Groom, 2002). Well defined problems are situational awareness errors in aviation. Aviation, well specified and the knowledge required to find the Space, and Environmental Medicine, 67, 507-512. solution is present in the instructions given, e.g. a puzzle Humphrey S.E., Moon , H., Conlon, D.E., Hofmann D.A. (2004). Decision-making and behavior fluidity: How (Groom, 2002). The fire fighting task used in this focus on completion and emphasis on safety changes experiment can be regarded as a well defined problem. over the course of projects. Organizational Behavior As cognitive lockup was found in this problem solving and Human Decision Processes 93, 14–27. setting, cognitive lockup is assumed to be a general Karau S.J., and Kelly J.R. (1992). The Effects of Time purpose heuristic. This implies that although the results Scarcity and Time Abundance on Group Performance are obtained in a fire-fighting domain, they can be Quality and Interaction Process. Journal of applied to any other domain. The same applies for the Experimental Social Psychology, 542-571. sample that has been used. The participants of the Kelly, R.J., and Loving T.J. (2004). Time pressure and experiment were Dutch, between 18-32 years old and group performance: Exploring underlying processes in most of them were highly educated. As we assume that the Attentional Focus Model. Journal of Experimental cognitive lockup does not depend on specific capabilities, Social Psychology, 40, 185-198. we suggest that the results from this sample can be Kerstholt, J.H., and Passenier, P.O. (2000). Fault applied to human beings in general. management in supervisory control: the effect of false alarms and support. Ergonomics, 43(9), 1371-1389. 72 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 Kerstholt, J., Passenier, P., Houttuin, K., & Schuffel, H. (1996). The effect of a priori probability and complexity on decision making in a supervisory control task. Human Factors, 38(1), 65–78. Kerstholt, J.H., Willems, P. (1993) The effect of the time restrictions on information search and information integration in a dynamic task environment, TNO- Report TM-01-B004, TNO technische menskunde, Soesterberg. Kleij, van der R., Lijkwan, J.T.E., Rasker P., De Dreu, C.K.W. (2009). Effects of time pressure and communication environment on team processes and outcomes in dyadic planning. Int.J. Human-Computer Studies, 67 (2009) 411-423. Meij, G. (2004). Sticking to plans: capacity limitation or decision-making bias? Doctoral dissertation, Department of Psychology, University of Amsterdam, Amsterdam. Meyers, D.E. & Kieras, D.E. (1997a). A computational theory of executive cognitive processes and multiple- task performance: part 1. Basic mechanisms. Psychological Review, 104, 3-65. Meyers, D.E. & Kieras, D.E. (1997b). A computational theory of executive cognitive processes and multiple- task performance: part 2. Accounts of psychological refractory-period phenomena. Psychological Review, 104, 749-791. Monsell, S. (2003). Task switching. Trends in Cognitive Sciences, 7,134–140. Moray, N., & Rotenberg, I. (1989). Fault management in process control: eye movements and action. Ergonomics 32 (11), 1319-1342. Neerincx, M.A. (2003). Cognitive task load design: model, methods and examples. In: E. Hollnagel (ed.), Handbook of Cognitive Task Design. Chapter 13 (pp. 283-305). Mahwah, NJ: Lawrence Erlbaum Associates. NTSB. (1973). Eastern airlines l-1011, miami, florida, december, 29, 1972 (Tech. Rep. No. NTSB-AAR-73- 14). Washington, DC: National Transportation Safety Board (NTSB). Pashler, H. (1994). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116, 220-244. Schumacher, E., Lauber, E., Glas, J., Zurbriggen, E., Gmeindl, L., Kieras, D., et al. (1999). Concurrent response-selection processes in dual-task performance: Evidence for adaptive executive control of task scheduling. Journal of Experimental Psychology: Human Perception and Performance, 25, 791–814. Wickens, C.D., and Hollands J.G. (2000). Engineering Psychology and Human Performance, 3rd ed. Upper Saddle River, NJ: Prentice-Hall. 73 CEUR Proceedings 4th Workshop HCP Human Centered Processes, February 10-11, 2011 74