Cognitive predictors of accuracy in quality control checking Hillary B. Katz (katzh@lsbu.ac.uk) Department of Psychology, London South Bank University, 103 Borough Road, London, SE1 0AA, UK James H. Smith-Spark (smithspj@lsbu.ac.uk) Department of Psychology, London South Bank University, 103 Borough Road, London, SE1 0AA, UK Thomas Wilcockson (t.wilcockson@lancaster.ac.uk) Department of Psychology, Lancaster University, Lancaster, LA1 4YF, UK Alexander Marchant (marchaa4@lsbu.ac.uk) Department of Psychology, London South Bank University, 103 Borough Road, London, SE1 0AA, UK Abstract resource that is essential for many everyday (and, by extension, work-related) tasks is working memory (e.g., Labelling errors on fresh produce are estimated to cost the UK supermarket industry £50m per year in product recalls Logie, 1993). It consists of a visuospatial sketchpad which and wastage. Such errors occur despite robust quality control underpins the temporary storage and manipulation of visual procedures. Given the financial and environmental impact of and spatial information, a phonological loop which is these errors, it is important to understand whether label- similarly engaged with auditory information, and an checking performance can be predicted by individual episodic buffer which binds together information from differences in cognitive abilities. To this end, participants different sources into coherent episodes (e.g., Baddeley, carried out a simulated label-checking task together with a number of measures of information processing speed, 2003). Monitoring and controlling these in relation to the attention, short-term/working memory, and mind-wandering. task at hand is the central executive, which also plays a Accuracy of label checking was found to be significantly major role in the deployment of attention, such that relevant predicted by three of the measures, with better short-term stimuli are attended to and irrelevant ones disregarded verbal memory being most strongly associated with (Engle, 2002). performance. Cognitive tests such as these provide a means of The measurement of relevant specific cognitive abilities, identifying how well employees are likely to perform when such as the speed of information processing, the ability to undertaking such tasks and, if necessary, how they should be supported in that role, possibly forming a screening battery direct and sustain attention, the capacity to hold and update when recruiting new quality control staff. The findings information in memory, and the executive functions highlight the importance of determining the component necessary to plan and execute behavior (Hambrick et al., processes of cognition which contribute to performance in 2010), should, in principle, provide better predictors of job real-world work environments. performance than tests of general mental ability and hence Keywords: Attention; Mind-wandering; Quality control better tools for selecting and screening employees. Yet checking; Short-term memory; Working memory research to date has provided little evidence that this is the case (Bosco, Allen, & Singh, 2015). Introduction The primary challenge for research in this area is to A long-standing concern of applied psychology has been to provide a reliable basis for matching the particular cognitive provide the practical means by which to predict how well skills of individuals with the demands of tasks they are, or individuals are likely to perform in real-world situations will be, called on to perform. Clearly there are broad along with a theoretical understanding of why this should be benefits in terms of recruitment, retention, morale and the case. Indeed, the motivation for developing the first tests quality of performance in ensuring that employees are given of intelligence was not just to measure individual work that suits their particular competencies. Failing to do differences but to assist in the appropriate placement of so will almost certainly lead to poorer performance, and individuals on the basis of their ability and likely depending on the role in question, may have high financial achievement (Anastasi & Urbina, 1997). implications or costs in terms of ill-health, injury or even With advances in the study of cognitive psychology, it has death. become clear that behavior relies on a variety of specific Advances in understanding the role of specific cognitive and qualitatively different resources, each dedicated to a abilities in task performance also promise to reduce ethnic different kind or aspect of processing (Baddeley, 2003). One and cultural biases that occur when general mental ability is used as the sole basis for employee selection, assignment. 750 Such biases are likely to reduce the chances of individuals comparison of two sets of stimuli to determine whether or with disabilities gaining employment, even though they not they match (e.g., Salthouse & Babcock, 1991). might be shown to be perfectly able to undertake the job if Short-term memory relates to the ability to store relevant specific cognitive abilities had been assessed. This information temporarily in memory over a duration of may be the case, for example, for some individuals with seconds (e.g., Cowan, 2008). The task of checking autism who have a normal or even superior ability to attend information from one source with that on another seemed to detail, even though they may be deficient in other aspects highly likely to draw on this memory system. The relative of cognition (Koshino et al., 2005). contributions of phonological (or verbal), spatial (relating to There are, therefore, compelling theoretical and practical sequential presentations of information), and visual short- reasons to pursue research that promises to provide both a term memory to label-checking were assessed in the current better understanding of the cognitive abilities that particular study. In order to determine whether executive-loaded kinds of tasks require and to map these onto specific memory processes might also be involved in checking, abilities individuals possess. Such matching would optimize further versions of the three short-term memory tasks were the performance of both the individual and the system in presented. In each of these, the simultaneous manipulation which he or she works. and storage of information was required, meaning that the The research reported in this paper investigated whether central executive as well as the slave systems in the working scores on different tests of specific cognitive processes memory model (e.g., Baddeley, 2003) was engaged. could predict the accuracy of performance on a repetitive The Attention Network Test (ANT; Fan et al., 2002) was label checking task. This task was designed to closely employed to measure visual attention, measuring three resemble work that is undertaken by quality control different networks: the alerting network, the orienting inspectors at a fresh produce packaging facility in the UK. network, and the executive control network. The alerting Measures of visual search, perceptual speed, short-term network aims to maintain an alert and vigilant state of memory, and attention were administered, together with a readiness for information processing, the orienting network self-report measure probing the propensity of individuals to selects task relevant information from the visual input, and mind-wandering during ongoing behaviour. the executive control network resolves conflict among The label-checking procedure involves an operative possible alternative responses. When checking a label, an determining whether or not the information that appears on operative has to be alert to the possibility of a mismatch a given product label correctly matches details as set out on between the label and the specification sheet. They must the product specification sheet (which includes information also be able to orient their attention to the specific about the supermarket’s order as well as the product from information being checked, whilst ignoring the potentially the producer). The number of fields of information printed distracting, but related visual information in the surrounding on a label varies between three and eleven. Example fields area. Finally, under this account, the executive control are the name of the product, its weight, its country of origin network would be called upon to decide if a mismatch and its barcode. If the information which appears on the response is valid or not. product label does not match the specification sheet, the Mind-wandering occurs when an individual has thoughts quality control checker should detect this and reject the unrelated to the task which move attention from the label. Generally three or four independent quality control intended task. The Daydreaming Frequency Subscale (DFS; checks are performed before the order is shipped from the Singer & Antrobus, 1970) was used to measure individual packaging facility to supermarket distribution depots. differences in the propensity to mind-wandering. In contrast Despite these stringent quality control procedures, to the ANT, which gives an indication of how well an products that are erroneously labelled do sometimes escape individual copes with potentially distracting information the packaging facility, necessitating the recall and disposal from the external environment, the DFS gives an indication or repackaging of produce. The recall and disposal of food of how an individual copes with distractions which are due to label errors is estimated to be £50 million industry- internally generated. Of particular relevance to the current wide annually in the UK alone (S. Hinks, Product Technical study is evidence that the incidence of mind-wandering is Manager: Fruit and Floral, Sainsbury’s Supermarkets Ltd, relatively high whilst completing undemanding tasks but personal communication). Whilst infrequent, the financial decreases as the task demands increase (McKiernan et al., and environmental costs attached to label errors are such as 2006). Since label-checking is repetitive and merely to drive research into their reduction. requires operatives to select, read, and check information on Given the accuracy-driven and time-constrained work labels against a specification sheet, it was considered likely environment in which label-checking occurs, two different that mind-wandering would occur. measures of the speed and accuracy with which information Together, the battery of tests was designed to measure a could be processed were administered. Visual search tasks broad range of specific cognitive functions that might (e.g., Wolfe, 2001) require individuals to search arrays of underpin and predict performance on label checking and letters, digits, or objects to identify a particular target other quality control tasks that require the identification of stimulus (e.g., the letter “T” amongst an array of other mismatches or mistakes. letters). Perceptual speed requires the speeded perceptual 751 Method Participants A total of 51 university students (44 females, 7 males, mean age = 24 years, SD = 6) took part in the experiment. They received a small honorarium or course credit in appreciation of their participation. All of the participants reported themselves to be naïve to the quality control processes involved in checking fresh produce labels. The participants were either native English speakers or were studying at undergraduate degree level with an International English Language Testing System (IELTS) score of at least 6.0 (the minimum requirement of London Figure 2. An example of a product label. South Bank University for entry to its degree courses). Design Materials Label checking task The label-checking and visual search tasks were A block of 50 trials was presented. The information programmed and run in Experimenter Builder Version displayed on the product specification sheet and that 1.4.128 B (SR Research Ltd., Ontario, Canada). E-Prime 2.0 presented on the label matched on 40 of these trials. For the (Psychology Software Tools, Inc., Sharpsburg, PA) was remaining 10, there was a mismatch between the two used to program and implement the remaining computerized sources of information. For each trial where there was a tasks. mismatch, only one field of information varied between the Facsimiles of the product specification sheet and labels product specification sheet and the produce label (e.g., the used in the packaging facility were created for the purpose best-before date). The field of information that differed was of the experiment (Figures 1 and 2 respectively). The varied pseudo-randomly over the 10 trials such that the number of fields of information per product on the errors appeared in different fields. Responses to these trials specification sheets and produce labels was held constant at were logged as correct when a mismatch between the seven. These fields of information were the product (the information set out on the product specification sheet and type of fruit or vegetable, e.g., baby courgettes), country of the label was indicated by the participant. origin, the grower (the name of the company which grew The participants undertook two further 50-trial label- and shipped the product), the quantity of items contained in checking blocks after this initial block. The data relating to the packet (i.e., the weight of the product), its best-before these are reported in Smith-Spark, Katz, Marchant, and date (indicated by “BB” on the specification sheet), the Wilcockson (2015). The focus of the current paper, product’s barcode number, and details of any promotion however, was purely on the extent to which the initial label- ribbon or label to be appended to the packaging (i.e., any checking performance of individuals with no prior promotional activity on the product being offered by the experience or training could be predicted on the basis of supermarket, such as “Any 2 for £2.50”). In the course of scores from the battery of cognitive tasks which was the block of trials, fifty different labels were presented. administered to them. The produce label and the product specification sheet were presented simultaneously on a 21”colour monitor Cognitive tests screen, with the former occupying the top half and the latter Visual search ability was measured using a modified version the lower half of the display. of Triesman and Souther’s (1985) letter finding task. A head-rest was used in the label-checking task in order Participants were presented with an array of 19 letter stimuli to minimize the head movements of the participants. (namely, N, C, F, K, and P). In one block of trials, they were asked to locate a normal, forward-facing letter in an array of backwards, mirrored letters. In a separate block of trials, the participants were asked to identify a backwards letter amongst an array of normal, forward-facing letters. In each case 1, 2, or 3 letters faced in the opposite direction to the others. Participants were asked to indicate how many backwards-facing letters they had seen. Performance on the backwards and forwards trials was combined to give mean RT and accuracy scores for visual search ability. Perceptual speed was measured using a letter comparison task, modified from Salthouse and Babcock (1991). Two Figure 1: An example of a product specification sheet. pages with multiple pairs of 3, 6, or 9 letters were presented 752 which participants had to decide were the same or different. congruent arrows (pointing in the same direction), The task for the participant was to write the letter “S” incongruent arrows (pointing in the opposite direction) or between the pair if the two members were the same and lines that were considered neutral. The cues (‘*’) could letter “D” if they were different. Mean perceptual speed and assist performance (in that the spatial cue was presented in accuracy scores were derived from the two measures as the the same location as the following target arrow - above or total number of correct responses made in 60s. A number below fixation), distract from performance (when the spatial comparison task followed this using the same design but cue was presented in an opposite location to the following with multiple pairs of numbers. target arrow), act neutrally with respect to performance Phonological short-term memory was assessed by the (central cue at fixation and double spatial cues above and Digit Span Task. Participants were presented with a below fixation), or there may be no cue present. sequence of single digit numbers, one at a time. Once the Performance on the alerting network was calculated by sequence was completed, they were asked to recall the digits subtracting the mean RT of the double-cue conditions from in the order they had been presented. The number of digits the mean RT of the no-cue conditions. To assess gradually increased over trials, starting with two and going performance on the orienting network mean RT of the up to a maximum of 10. Three trials were presented at each spatial cue conditions were subtracted from the mean RT of level. At least two of the three trials needed to be correct in the center cue condition. Finally, for the executive control order to advance to the next level of the task. A participant’s (conflict) network the mean RT of all congruent flanking span length was taken as the last level at which they could conditions, summed across cue types, were subtracted from reliably remember the sequence of digits in the correct serial the mean RT of incongruent flanking conditions. order. A backward digit span task was also administered in The Daydreaming Frequency subscale (DFS) of the which participants had to report the digits in reverse serial Imaginal Process Inventory (Singer & Antrobus, 1970) was order, thereby drawing on working memory rather than used to measure self-reported propensity to mind simply short-term memory to store and manipulate wandering. Participants rated twenty-four statements on a 1- information simultaneously. 5 scale, with higher scores indicating a greater frequency of The Corsi Block span test (Corsi, 1973) was used to mind-wandering. An example statement is “When I am not measure spatial working memory. An array of 12 squares paying close attention to some job, book or TV, I tend to be was presented. Squares in the array were highlighted in daydreaming ...”, with participants choosing one of the sequence one at a time. At the end of the sequence, the following options: 1 = 0% of the time, 2 = 10% of the time, participant was asked to indicate the locations of the 3 = 25% of the time, 4 = 50% of the time, and 5 = 75% of highlighted squares in the correct serial order. The number the time. of squares highlighted increased over trials from two up to a maximum of 10. Three trials were presented at each level of Statistical analysis the task, with span being taken as the last level at which the A multiple stepwise regression was run with the cognitive participant was entirely successful in recalling at least two test measures entered as predictor variables. Overall label- out of the three trials correctly. The total number of cells checking accuracy was the outcome variable. whose location was correctly recalled in serial order was recorded. A further version of the task was presented, the Procedure Corsi backward task, which required the reporting of the Informed consent was given by all participants to take part spatial sequence in reverse serial order, again tapping in the experiment. Before the checking task began, the working memory resources. participants were seated at a viewing distance of 55cm from A modified version of the Visual Patterns Test (Della Sala a 21” computer monitor. They then viewed a 10-minute et al., 1999) was used to measure visual working memory. slide show presentation. This provided them with a detailed Participants were presented with different arrays of black description of the label layout, specification sheet layout, and white squares, after each of which they had to recall the general task instructions, the nature of errors, etcetera. pattern by indicating which squares were white and which During the label-checking task, the participants indicated were black. The number of squares in the array increased whether or not the information presented on a given label during the course of the experiment. A second version of the was correct, checking it against the appropriate entry on the task which placed demands on working memory was specification sheet. They were instructed to respond as administered. It required participants to invert the colours of quickly but as accurately as possible. Responses were made the squares when reporting them. In both versions, the total by pressing designated Yes and No keys on a standard number of cells that were correctly identified was logged. QWERTY keyboard. The ANT (Fan et al., 2002) was used to measure visual The cognitive measures were administered in a separate attention. Participants were shown a cue (‘*’) and required testing session. The order in which the cognitive tasks were to indicate the direction in which a central target arrow presented was counterbalanced between participants. The pointed. This target arrow appeared either above or below letter and number comparison tasks had a pen-and-paper the fixation point in the middle of the screen. It was format, while all others were computerized. surrounded by a set of distractors that consisted of either The participants were debriefed upon completing testing. 753 Results The stepwise multiple regression analysis indicated that overall label-checking accuracy could be significantly The scores from three participants were removed on the predicted by the cognitive predictors, R = .637, adjusted- R2 backward search and two on the forwards search due to their = .358, F(3, 37) = 8.44, p < .001. Three predictors were having mean scores more than 2.5 SDs from the overall entered in the final three-step model. These were digit span mean. forwards, standardized-β = .658, p < .001, Corsi forwards, The overall mean proportion accuracy of label-checking standardized-β = -.395, p = .004, and perceptual speed, was .85 (SD = 0.05). standardized-β = -.459, p = .004. Descriptive statistics for each cognitive test are displayed in Table 1, together with Pearson’s correlations indicating the extent of the relationship between each test and label Discussion checking accuracy. The simulated label checking task used in this study resulted in a rate of errors somewhat greater than that indicated by Table 1: Descriptive statistics the historical record at the actual packing facility on which it was modelled (approximately 15% as opposed to 2% of Cognitive Cognitive test Mean Correlation process (S.D.) with label checks). While the stimuli were virtually identical, the checking laboratory-based task did entail many more checks and in a accuracy (r) more concentrated time-frame than demanded in this and most likely other real-world situations. Visual Letter finding The results indicate that label-checking accuracy can be search 1.Speed 6945.50 -.002 (1723.45) significantly predicted on the basis of the cognitive tasks 2. Accuracy 25.51 .017 employed in this experiment. Verbal short-term memory (as (3.10) measured by the digit span forwards task) was the strongest predictor of performance, with the ability to retain a larger Perceptual Comparison number of digits in memory being associated with higher speed task 1. Speed 25.65 -.110 accuracy. The next strongest predictor was perceptual speed (6.16) -.014 although, in this case, the relationship was negative. It 2. Accuracy 23.01 would appear that processing information more rapidly was (4.87) associated with lower accuracy, which may indicate a speed-accuracy trade-off. Spatial short-term memory Phono- Digit Span logical 1. Forward 7.34 .358* (measured by the Corsi forwards task) was also a significant short-term 2. Backward (1.48) .042 negative predictor of accuracy. Although it may seem memory 5.90 paradoxical that the ability to hold more spatial information (1.61) in memory would be associated with poorer accuracy, it may be that a stronger spatial memory encouraged Visual Visual Pattern memory Test individuals to adopt a non-optimal approach to label- 1. Short- 1. Original 82.46 .117 checking, in particular chunking (e.g., Miller, 1956). A term (19.91) chunking strategy in which several bits of information from 2. Working 2. Inverted 58.51 .041 the specification sheet are checked in one visual pass of the (35.05) produce label, has previously been found to be associated Spatial 1. Corsi Block with lower levels of checking accuracy than a more memory Span Test systematic approach in which one piece of information at a 1. Short- 2. 1. Forward 43.37 -.304* time is taken from the product specification sheet and term (13.01) checked against the label (Smith-Spark, Katz, Marchant, & 2. Working 2. Backward 13.63 .087 Wilcockson, 2015). (6.89) Whilst null results should be treated with caution, the Attention Attentional results suggest that cognitive tasks involving greater Network Test executive resources do not predict performance, since none 1. Alerting 23.92 .093 of the executive-loaded span tasks were significantly (26.17) associated with label-checking accuracy. Further to this, 2. Orienting 53.95 .061 (30.25) neither visual search abilities nor the ANT predicted 3.Executive 114.64 .127 performance, suggesting that neither visual search nor the control (45.35) attentional processes tested by the ANT contribute to label- checking accuracy. Finally, mind-wandering, as measured Mind Imaginal wandering Process 67.76 -.120 by the DFS), did not predict correct responses on the label- Inventory (15.94) checking task. Key: * = p < .05 The present study explored the value of tests of specific cognitive functions as predictors of performance on a 754 simulated label checking task. Unlike most research in ability, performance, and subgroup differences. Personnel applied areas of occupational psychology, this experiment Psychology. DOI: 10.1111/peps.12099 had well defined, objective outcome measures and allowed a Corsi, P. (1973). Human memory and the medial temporal reasonably close mapping between the behavioural region of the brain. Dissertation Abstracts International, requirements of the task, i.e., perceptual scanning, 34(2-B), 891. comparison, no problem solving, etc., with narrowly defined Cowan, N. (2008). What are the differences between long- cognitive processes which one would assume underpinned term, short-term, and working memory? Progress in these actions, such as visual search, focused attention, Brain Research, 169, 323–338. executive control and short-term memory. While some Della Sala, S., Gray, C., Baddeley, A., Allamano, N., & success in prediction was gained, the experiment also Wilson, L. (1999). Pattern span: A tool for unwelding demonstrated the challenge in determining the connection visuo-spatial memory. Neuropsychologia, 37, 1189-1199. between specific cognitive abilities and task performance. Engle, R. W. (2002). Working memory capacity as This is partly because there are different ways in which a executive attention. Current Directions in Psychological given task, even a relatively straight-forward task like label Science, 11, 19-23. checking, can be approached (Smith-Spark et al., 2015). Fan, J., McCandliss, B. D., Sommer, T., Raz, A., & Posner, Differences in the choice of strategy may account for a M. I. (2002). Testing the efficiency and independence of substantial proportion of the variability associated with task attentional networks. Journal of Cognitive performance and relate, in turn, to prior experience and even Neuroscience, 14, 340-347. general mental ability of individuals (Hambrick et al., Hambrick, D. Z., Oswald, F. L., Darowski, E. S., Rench, T. 2010). A., & Brou, R. (2010). Predictors of multitasking Aside from the strength and availability of specific performance in a synthetic work paradigm. Applied cognitive resources, some of which have been measured in Cognitive Psychology, 24, 1149-1167. the current experiment, performance also depends on the Koshino, H., Carpenter, P. A., Minshew, N. J., Cherkassky, demands of situational factors such as time constraint, V. L., Keller, T. A., & Just, M. A. (2005). Functional interruptions, incentives and cognitive load, and as connectivity in an fMRI working memory task in high- importantly non-cognitive factors such as previous functioning autism. Neuroimage, 24, 810-821. experience, motivation and conscientiousness. Together Logie, R. H. (1993). Working memory in everyday these lead to cognitive dynamics which are variable and cognition. In G. M. Davies and R. H. Logie (Eds.), difficult to predict, as seen in the negative contribution of Memory in everyday life (pp. 367-401). Amsterdam: spatial memory and processing speed to the accuracy of Elsevier. performance. Given the manifold nature of cognition, even McKiernan, K. A., D'Angelo, B. R., Kaufman, J. N., Binder, basic procedural tasks such as label checking, may resist an J. R. (2006). Interrupting the stream of consciousness: An exhaustive description of the contribution of specific fMRI investigation. Neuroimage 29, 1185–1191. cognitive processes to performance. This is probably why Miller, G. A. (1956). The magical number seven, plus or tests of general cognitive ability have generally proven to be minus two: Some limits on our capacity for processing superior predictors of job performance as well as the information. Psychological Review, 63, 81-97. preferred basis for employee selection and allocation Salthouse, T. A., & Babcock, R. L. (1991). Decomposing (Schmidt, 2002). adult age differences in working memory. Developmental Psychology, 27, 763-776. Acknowledgments Schmidt, F. L. (2002). The role of general cognitive ability The research reported in this paper was funded by Innovate and job performance: Why there cannot be a debate. UK (grant number 101393). The authors are very grateful to Human Performance, 15, 187-210. Simon Hinks (Sainsbury’s Supermarkets Limited), Daniel Singer, J. L., & Antrobus, J. S. (1970). Imaginal Processes Boakes (Mack), Tetyana Bennett (Mack), Trish Fox (Mack), Inventory. Princeton, NJ: Educational Testing Service. and Jez Pile (Muddy Boots Software). The authors also Smith-Spark, J. H., Katz, H. B., Marchant, A., & thank Monika Michalska for assistance with data collection Wilcockson, T. (2015). Label-checking strategies to adapt and our grant monitoring officer, John Stones, for support. behaviour to design. Proceedings paper accepted for ECCE 2015: The 33rd annual conference of the European Association of Cognitive Ergonomics, Warsaw, Poland, 1- References 3 July. Anastasi, A. & Urbina, S. (1997). Psychological Testing (7th Treisman, A., & Souther, J. (1985). Search asymmetry: A ed.). Upper Saddle River, NJ: Prentice Hall. diagnostic for preattentive processing of separable Baddeley, A. (2003). Working memory: Looking back and features. Journal of Experimental Psychology: General, looking forward. Nature Reviews Neuroscience, 4, 829- 114, 285-310. 839. Wolfe, J. M. (2001). Asymmetries in visual search: An Bosco, F., Allen, D. G., & Singh, K. (2015). Executive introduction. Perception & Psychophysics, 63, 381-389. attention: An alternative perspective on general mental 755