Comparison of Scan-Mechanisms in Augmented Reality-Supported Order Picking Processes

Abstract

Designing the change towards a digitalized production plant, we analyze in our research project whether the usage of Augmented Reality (AR)-technologies offers advantages for the worker and the process. In this paper we focus on the comparison of confirmation methods for order picking with smart glasses in the automotive industry. As logistic processes require manual or automatic confirmation to interact with the warehouse management system, we compare different scan-mechanisms for smart glasses. In our pilot-study, we analyze the picker's task completion time, error frequency, attention before and after the usage and subjective assessments regarding the usability and health-related information during an 8-hour work shift. In this pilot-study, we aim to evaluate scanmechanisms with the best fit for a smart glasses usage. With the best visualization and confirmation combination, we will conduct a consecutive field study at our production plant. Based on our results, we recommend a scan-glove in combination with smart glasses as visualization device.

Author Keywords

Augmented Reality, order picking, logistics, pick confirmation, interaction mechanisms, smart glasses

Introduction

The further advancement in digitalization of the production environment provides ample opportunities to support workers with well-fitted Augmented Reality (AR-) technologies. According to Tümler [ 1 ], mobile AR is defined as a situation-friendly visualization of computer-based information, which is positioned on portable display devices in the user’s field of view. It is important that the AR content does not bother the user during his work [ 1 ]. On the shop floor, ARtechnologies, such as head-mounted displays (HMDs), can support workers through visual guidance and hands-free work. In this paper we regard HMDs as synonymous with smart glasses. Smart glasses have the potential to replace monitors or paper-based picking lists, which show orders in a tabular structure. The advantages of smart glasses in an order picking context range from avoiding unnecessary headmovements through visualizations in the user’s field of view to a step-by-step visual guidance for the training of unskilled workers. The majority of publications in this research field discuss for example see-through calibration [ 1 ], different guidance methods [ 2 ], the comparison of projection or head mounted displays with a picking list and with auditory instructions [ 3 ]. Most studies were placed in a laboratory setting [ 1-7 ]. The pilot study, which we present in this paper, is also a laboratory study and serves as a base for a further study, which will be conducted in a real production environment, where the process time depends on the predetermined manufacturing rhythm at the assembly line.

To implement smart glasses in standard production on the shop-floor, the visualization device delivering order information needs to be connected with the warehouse management system. Additionally, the picker has to interact with the warehouse management system by confirming process steps. With this feedback of completed orders, the worker receives information about the next order sequence from the system. Various confirmation methods are conceivable solutions. Common confirmation methods in logistics contexts are buttons or scans. Work guidance through the process requires confirmation for the warehouse management system at each retrieval and placement box. When buttons are used for confirmation purposes, high investment is needed. An alternative are barcodes, attached to pallet cages and target bins, which are less cost-intensive. One advantage of using barcodes is the flexibility in case of processes changes. This is why scanning barcodes is a popular interaction method in logistics. Scanning devices can be hand-held scanners, scan gloves (see fig. 1 and 2) and the smart glasses themselves. As hand-held devices can disrupt the work flow during order picking processes and pre-assembly, these devices have been excluded.

The paper’s objective is to determine the best fit between three confirmation scenarios using wearables in order picking. While one scenario uses a ring confirmation clicker with auditory feedback with visualized order information on a monitor, two more scanning scenarios including AR-technologies are investigated. In AR-scanning option number one, the smart glasses scan the barcodes, which means that the worker visually focuses on the barcode for a set amount of time but simultaneously receives ordering information in his field of view. In AR-scanning option number two, smart glasses are combined with a scan glove. During pointing the scanner at the barcode, the worker is able to look in a different direction. With the presentation of our pilot study, we aim to find a custom-made solution for easing the work process by combining mobile devices. Current rapid development of wearables on the market has broadened the variety of available devices and scanning options. Analyzing individual process steps in a workstation-specific context will give information which device can support the worker. Retrieval and placement of items is one action, while confirmation is another action. It may be possible that one wearable, namely the smart glasses, is capable of conducting both actions. We aim to inquire, whether a combination of wearables or one wearable is better suited for the order picking workstation regarding task completion time, error frequency and ergonomic support.

Related Work

A great number of studies addresses AR-technologies in the production area. Head-mounted displays or smart glasses and beamer-based projection are favored visualization tools [ 8 ]. Hereafter, we present a sample of previous publications.

Kampmeier et al. [ 6 ] conducted a laboratory study with 60 participants analyzing the impact of AR-usage on the worker and the process. A HMD Lite-Eye LE 750 A was used for visualization. For interaction with the glasses the study conductors switched the display on participants’ request to the next order. Wiedenmeier [ 9 ] presents another use-case with AR-supported assembly tasks. A sample of 36 participants wore a Microoptical Clip-On HMD during three assembly tasks. They interacted with the HMD with a mouse click. Speech recognition as an interaction method was applied by Reif et al. [ 7 ] in a laboratory study with 19 participants. They assumed that speech recognition is partially difficult to process by the device. In Theis’ et al. [ 5 ] study, during which 45 participants performed assembly and order picking tasks, they used a portable PC carried on the belt to switch the slides in the Microvision Nomad HMD. Another interaction method is presented by Tümler [ 1 ]. During his laboratory study, 20 participants had to confirm tasks using a forearm keyboard. Funk et al. [ 3 ], Schwerdtfeger et al. [ 10 ] and Guo et al. [ 4 ] used the wizard-of-oz technique during their studies in an order picking context, which means that the study supervisor switched the visualization in the HMD.

To resume the described publications, many studies did not primarily investigate confirmation mechanisms. The applied mechanisms such as a portable PC carried on the belt or a forearm keyboard are not a step forward in contrast to buttons or scans. The suggested use of speech recognition cannot be implemented feasibly at our production plant due to the surrounding noise from tugger trains and forklifts. Wiedenmeier’s [ 9 ] confirmation method, a mouse click, resembles a button attached to a box, and can therefore feasibly applied in production areas. In logistics, scanning is a common confirmation method, which is not tested by any of the presented studies. We assume that scanners in combination with HMDs/smart glasses have not been investigated due to a recent innovation leap in scanning devices. Compared to previous hand-held scanners, the

Workflow description at the production plant

The picker starts the process between pallet cages A (left) and A (right). He first picks all A-left components in consecutive sequence of the order information from his left and then all complementary A-right components from his right.

Afterwards, the picker moves to the next set of pallet cages, which supply alternative components opposite from each other.

The current workflow which we aim to ease on the part of the worker, provides visualization via a monitor fixed to the right side of the movable shelf. new scan glove supports hands-free work and does not impair the worker through high weight.

Test environment and hardware

In our pilot study, we emulate the work environment and conditions of the real order picking workstation of our production plant, where we will conduct further studies. We selected this workstation using Rasmussen’s Skills-Rules-Knowledge framework [ 11 ]. The workstation of our choice, a so-called ‘supermarket’, is an order picking workstation, where the employee composes 16 pairs of nine possible types of footwell claddings into a movable shelf in the correct sequence. Order information can differ in color, model series, variant driver’s side and components for emergency vehicles. In addition, the process contains a pre-assembly work step of either fitting a boot switch or a shutter on the driver’s side, depending on the ordered special equipment. After picking a sequence consisting of 16 pairs of components, the picker parks the shelf at the parking position, from where a tugger train takes the shelf to the assembly line. For our pilot study, we built a reference workplace (Fig. 3), which is similar to the real workstation. We used the same pallet cages and the same target shelf, but we reduced the variability of component alternatives from nine to three. As the target shelf is identical, the number of footwell claddings to pick remain the same for one sequence. A detailed workflow description is explained on the left.

As test hardware we used a ring confirmation clicker, binocular ODG R-7 smart glasses and a scan glove from ‘ProGlove’.

Methodology of the user study

In our pilot study we compared the described status quo (using a monitor for order information and confirmation by a ring clicker) with AR-supported order picking combined with two different interaction mechanisms: scanning by the smart glasses and scanning using a ProGlove scanner. To measure the picker’s performance, we analyzed the total task completion time and the error frequency. Each participant was given the opportunity to familiarize him- or herself with the workflow and hardware for a full round of order picking (full-shelf trail) before measurements began. For a better differentiation between error characteristics, a subdivision of the error frequency into error types according to Lolling [ 12 ] into quantity errors, type errors, omission errors and quality errors is possible, which Schwerdtfeger et al. [ 13 ] describe as ‘wrong amount, […] wrong item […], missing article […] [and] damaged article’. In our pilot study, we provided only undamaged components and therefore excluded quality errors from the evaluation scope. Due to the narrowness of the target boxes, Order of the methods used in the study: • • • • • • • • • quantity errors could only be omission errors because putting more than the required quantity into the box is impossible. Therefore quantity and omission errors were both treated as omission errors. Summarizing, the evaluation focuses on the erroneous picks in wrong or missing items. If the picker pre-assembles the wrong boot switch or shutter, we view this as an error due to wrong item. In both cases, whether it concerns wrong item or wrong pre-assembly, the component has to be re-assembled.

For the evaluation of the usability-friendliness of the three different visualization-interaction scenarios we used the system usability scale (SUS) [ 14 ], consisting of ten questions alternating between positive and negative statements. The evaluation scale is a 5-point Likert scale starting from “I do not agree at all” and ending to “I fully agree”. In addition, the d2-Test of Attention [ 15 ], the Raw NASA-TLX [ 16 ] and Visual Fatigue Questionnaire (VFQ) [ 17 ] measure several aspects of psychological and physical strain.

The d2-Test of Attention can be used to assign a numeric value to an individual’s concentration capacity. The test contains 14 lines with 47 ‘d’s and ‘p’s in each line, while each ‘d’ or ‘p’ can be supplemented by one to four marks above or below the letter. During the test procedure, the respondent is was given 20 seconds processing time for one line. The participant has to find as many ‘d’s with two marks as possible during the predefined time period. The concentration capacity ‘CP’ can be calculated based on the scores for the amount of the processed quantity of letters, the omission errors and the confusion errors.

Relying on the Raw NASA-TLX questionnaire, we evaluate mental demand, physical demand, temporal demand, performance, effort and frustration. Each dimension of the questionnaire delivers a numeric value between 0 and 100. The total R-TLX is the sum of the six dimensions divided by 6 [ 16 ].

To estimate visual impairment, Bangor’s VFQ [ 17 ] contains the assessment of 17 categories of visual strain, which are evaluated on a scale from ‘not noticeable at all’ to ‘extremely noticeable’.

Results of the user study

The test procedure emulated the conditions, temporal restraints and tasks of the workstation at the production plant. We tested a sample of five participants at the reference workplace in our pilot study to determine user-friendliness and feasibility of implementing scan mechanism for the confirmation in AR-supported order picking in a larger sample. More details regarding the study design are described on the left. Regarding the mean task completion time (TCT) per shelf (Fig. 4), we observed, that baseline measurements with the monitor scenario (M) were at 5:52min (SD ±00:31min) per shelf. Scanning with the glasses (AR 1) was more time-consuming with mean TCT at 8:11min (SD ±01:08min). Figure 5 and 6 show examples of AR1-visualizations. Yet, receiving order picking information by the glasses and using a scan glove as confirmation mechanism (AR2) reduced the mean TCT per shelf by almost a minute to 4:57min (SD ±00:29min). Consequently, worker were the fastest in the AR 2 scenario even though hand and arm movement towards the target boxes to trigger the confirmation mechanism were comparable to scenario M. Since the maximum TCT is dependent on the realtime manufacturing rhythm of the production plant, a confirmation mechanism may not exceed the current TCT time limit, as this would put the entire production on hold. Thus, based on this findings, implementing the AR 1 scenario at the production plant is unfeasible at the current level of technical development. TCT results suggest that for the introduction of smart glasses to order picking merely AR 2 scenario has the potential to be tested at the plant.

Error frequency 0 2 4 6 8 Analyzing the occurrence of errors (fig. 7), the majority of errors were type errors (wrong part or wrong preassembly). The fewest errors were observed in AR 1, followed then by AR 2 and the most errors were made in the baseline scenario M. AR 1 decreases error frequency the most, but at the expense of increasing TCT. The observations from the pilot study imply thus that introduction of AR-technologies supports error reduction at the current TCT using a scan glove. Averaged Raw NASA-TLX scoring (fig. 8) provided detailed information on work strain in terms of mental, physical and temporal demand as well as effort and frustration connected to the employees’ workload. These scores were almost identical for scenario M and AR 2 (m=22.7 vs. 22.8) but considerably higher for AR 1 (m=38.0). Hence, AR 2 did not alter work strain compared to the baseline whereas AR 1 implementation affected the workers physically and mentally for the worse. Raw NASA-TLX results complimented information gathered with a SUS questionnaire in which all participants preferred AR 2 to AR 1. The mean SUSvalue for scenario M is 77 (SD ±1:,80), for AR1 59 (SD d2-Test of Attention M CP pre

M CP post AR 1 CP pre AR 1 CP post AR 2 CP pre AR 2 CP post ±14:84) and for AR2 76 (SD ±13:42). This outcome means that acceptance for AR 2 is higher and will make implementation at the plant easier. SUS outcomes gave no definitive answer whether scenario M or scenario AR 2 is the better fit in terms of workstation design as 3 participants rated M higher than AR 2 while 2 participants gave the higher scores for AR 2. Between the two AR scenarios, all participants preferred the AR 2 scenario. Analyzing the feedback on self-rated visual function with the Visual Fatigue Questionnaire, small deviations were observed, with a positive tendency regarding AR 2 in comparison to AR 1. Impact of ARuse on concentration performance individually varied. There is a tendency to reach higher concentration performance scores with the d2-Test of Attention at the post-shift assessment (in contrast to the pre-shift measurements), although one participant’s concentration performance was not (or hardly) affected by any of the scenarios, because the participants consequently reached almost the maximum of the value. For three participants, working for the duration of an entire shift under exposure to AR 1 technology lowered their d2 scoring and affected their attention span and concentration negatively. Detailed results are presented in figure 9.

Discussion and conclusion

In conclusion, we expect scenario AR 1 no to be the right solution for introduction at the plant on a trialbasis due to the high TCT and low workers’ acceptance. Based on the results from our pilot study, we recommend two of the three tested confirmation mechanisms for trial in the production area. The scenarios recommended to test in a larger sample are scenarios M and AR 2. Both M and AR 2 fulfill the prerequisites for production implementation in terms of TCT adherence and potential to reduce errors. To determine the ultimate reach of the two confirmation methods on TCT and error frequency, more studies are needed. Lastly, such studies will contribute to estimating the exact impact on visual fatigue and concentration performance levels. Through a field study with a larger sample of test persons during a full-shift usage, we hope to deliver results, based on which we can decide whether a serial production with ARtechnologies at order picking workstations is useful. Christopher M. Schlick. 2016. Head-Mounted Displays – Bedingungen des sicheren und beanspruchungsoptimalen Einsatzes. BauA Report, Project F 2288, Dortmund Berlin Dresden, Germany

References

1. Johannes Tümler . 2009 . Untersuchung zu nutzerbezogenen und technischen Aspekten beim Langzeiteinsatz mobiler AR-Systeme in industriellen Anwendungen . PhD Dissertation . Fakultät für Informatik, Otto-von-Guericke Universität Magdeburg . Magdeburg, Germany

2. Björn Schwerdtfeger . 2009 . Pick-by-Vision: Bringing HMD-based Augmented Reality into the Warehouse . PhD Dissertation . Institut für Informatik, Technische Universität München. München, Germany

3. Markus Funk , Alireza Sahami Shirazi, Sven Mayer, Lars Lischke, and Albrecht Schmidt . 2015 . Pick from here!: an interactive mobile cart using in-situ projection for order picking . In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15) . ACM, New York, NY, USA, 601 - 609 . DOI: https://doi.org/10.1145/2750858.2804268

4. Anhong Guo , Xiaolong Wu, Zhengyang Shen, Thad Starner, Hannes Baumann , Scott Gilliland . 2015 . Order-Picking with Head-Up Displays . In: Computer. June 2015. IEEE, Washington D.C., USA, 16 - 24

5. Sabine Theis , Claudius Pfendler, Thomas Alexander, Alexander Mertens, Christopher Brandl,

6. Jürgen Kampmeier , A. Cucera , L. Fitzsche , H. Brau , M. Duthweiler , G. K. Lang . 2007 . Eignung monokularer Augmented Reality - Technologien in der Automobilproduktion . In: Klinische Monatsblätter für Augenheilkunde, Georg Thieme Verlag, Stuttgart, Germany

7. Ruppert Reif and Dennis Walch . 2008 . Augmented & Virtual Reality applications in the field of logistics . In: The Visual Computer ( 2008 ) 24 . SpringerVerlag. Heidelberg Berlin, Germany, 987 - 994

8. Sebastian Büttner , Henrik Mucha, Markus Funk, Thomas Kosch, Mario Aehnelt, Sebastian Robert and Carsten Röcker . 2017 . The design space of augmented and virtual reality applications for assistive environments in manufacturing: a visual approach . In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, ACM, New York, NY, USA, 433 - 440

9. Stefan Wiedenmaier . 2004 . Unterstützung manueller Montage durch Augmented RealityTechnologien . Shaker, Aachen, Germany

10. Björn Schwerdtfeger and Gudrun Klinker . 2008 . Supporting Order Picking with Augmented Reality . In: 7th IEEE International Symposium on Mixed and Augmented Reality , 15 - 18 September, Cambridge, UK, 91 - 94

11. Nela Murauer and Steffi Gehrlicher (in press). 2018 . Evaluation of order picking processes regarding the suitability of smart glasses-based assistance using Rasmussen's Skills-RulesKnowledge framework . In Proceedings of the AHFE 2018 International Conference on Human Aspects of Advanced Manufacturing, July 21-25 , 2018 , Orlando, Florida, USA

12. Andreas Lolling . 2003 . Analyse der menschlichen Zuverlässigkeit bei Kommissioniertätigkeiten . Shaker Verlag. Herzogenrath, Germany

13. Björn Schwerdtfeger , Rupert Reif, Willibald A. Günthner , Gudrun Klinker, Daniel Hamacher, Lutz Schega, Irina Böckelmann, Fabian Doil and Johannes Tümler . 2009 . Pick-by- Vision : A First Stress Test . In: 8th IEEE International Symposium on Mixed and Augmented Reality 2009 , Orlando, USA, 115 - 124

14. John Brook. 1996 . SUS: a 'quick and dirty' usability scale . In: Usability Evaluation In Industry. Taylor & Trancis, London Bristol, 189 - 194

15. Rolf Brickenkamp . 1962 . Test d2 AufmerksamkeitsBelastungs-Test . Hogrefe Verlag für Psychologie , Göttingen Bern Toronto Seattle

16. Sandra G. Hart . 2006 . NASA-task load index (NASA-TLX); 20 years later . Proceedings of the human factors and ergonomics society annual meeting . Sage Publications , Vol. 50 . No. 9 ., Sage CA , Los Angeles, CA

17. Aaron W. Bangor . 2000 . Display Technology and Ambient Illumination Influences on Visual Fatigue at VDT Workstations. Dissertation. Faculty of the Virginia Polytechnic Institute and State University, Blacksburg, Virginia, USA