Assembly operators are experiencing ever-increasing cognitive loads due to increasing production complexity. Higher quality assembly instructions, with input from operators by sharing their knowledge, can reduce errors during the assembly process. This article describes two studies that investigate the human needs for a digital process that streamlines the input of operators in the creation or adaptation of work instructions in a Mixed-Model Assembly Systems. The first study consisted of contextual inquiries and semi-structured interviews and aimed to discover the high-level needs of the different roles involved in the creation process. The second study used the Wizard of Oz method to investigate which interaction methods could be suitable to provide feedback about erroneous assemblies. We found that any systems should take into consideration, among other things, current operator mobility, presence of multiple operators at a workstation and different technological skill levels as important factors to consider when developing new systems to capture operator knowledge. With respect to interaction methods, participants preferred manual input devices over gestural feedback methods.
Modern manufacturing environments are facing increasing challenges as a result of more variety in production [4, 14]. More specifically, Just in Time (JIT) [2] production has seen the rise of Mixed-Model Assembly Systems (MMAS) [15], where operators are tasked with the assembly of a wide variety of products. These high complexity production environments subsequently result in increased operator assembly errors Copyright © 2019 for this paper by its authors. Use permitted Creative Commons License Attribution 4.0 International (CC BY 4.0). under [5]. To reduce operator cognitive load, recent years have seen efforts towards better assembly instructions along with calls to including the views of operators when creating instructions [16].
Recent technological advancements in wearable electronics allow include systems to measure operator well-being [13]. These include augmented reality systems [11], or suggestions to integrate smartwatches in assembly [1] partly in an effort to support operators with assembly tasks.
This article describes 1) a study on exploring the needs of operators in MMAS along with 2) a Wizard of Oz study on exploring modes of interaction when communicating assembly errors. This was done due to the rise in complexity along with the notion that operators are important when creating assembly instructions. The focus in the Wizard of Oz study was a task that consisted of flagging production errors. Our study exists as part of a larger project on how operators can be provided with the appropriate assembly instructions that leverages their knowledge and minimizes the work required from both operators and process engineers.
More specifically, we investigated the current situation and requirements for a system supporting operator input to adapt digital work instructions through contextual inquiries with operators. Interviews were performed with those responsible for creating work instructions of four multinational manufacturing companies (Study 1) in Flanders, Belgium. Company A assembles airplane wing support. Company B builds projectors for non-consumer market. Company C assembles agricultural vehicles, while Company D is a manufacturer of weaving machines, with the focus of the study situated in the assembly zone where customized weaving machines are assembled. Based on this information, a flow was created where we focused on the question how operators can best interact with the work instructions with a Wizard of Oz study (Study 2). The flow was also discussed with representatives of these companies as well as potential providers of digital work instructions.
Our research in study 2 shows similarities with the work of Werrlich et al. [18] and Funk et al. [6]. As where Werrlich et al. focus on the Hololens for interaction, our study compares interaction on AR-glasses (on the device and gestures) with interaction on a smart watch. In the research of Funk et al. design guidelines are given for interaction with AR-systems. The ways of interaction were not specified in this research. 2
To gain a better insight into the workflow of operators and guide our exploratory study and experiment, we first conducted semi-structured interviews followed by contextual inquiries [3] on location; we combined observations with qualitative interviews with operators while they worked. In both cases, participants signed informed consent documents. The ethical commission of Ghent University approved the overall goal and setup of the study.
We performed semi-structured interviews were with line managers and process engineers (n=10) at all locations that participate in this study (3, 3, 3 and 1 persons for company A, B, C and D respectively). The reason why there was only one process engineer in the last company was that the line manager was not available. Topics of discussion included the bidirectional information need from the perspective of management and operator, the current use of technology by operators, the current workflow for creating and updating assembly instruction information, followed by a discussion about the potential improvements during this process. 2.2
Following formal interviews, we performed 4 contextual inquiries [7] during the assembly tasks of operators. This process generally entails observing users in their work context accompanied by in situ informal discussions about their work practices, or clarifications about particular activities or tasks. Our sessions ranged in duration from 30 minutes (in the case of Company C where only one assembly was performed), to 2 hours. Where possible, operators were asked about their information need during assembly, the use of technology, the use of assembly instructions and the way work is performed (i.e. the order of instructions).
Special attention was paid to how instructions were currently used, how operators communicate any errors found during the assembly process (i.e. missing parts), and time pressure during assembly. Additionally, we focused on contextual factors such as environmental noise, workspace cleanliness, operator mobility, wearing of safety glasses and gloves. Finally, we also questioned participants about their skill level and how frequently particular assemblies occur. During observation, we took pictures for clarification and we used pen and paper for further documentation, with results later being described digitally by three researchers. 2.3
Both the preceding semi-structured interviews and contextual inquiries were subsequently summarized using the MoSCoW requirements prioritization [17], where user requirements are given a priority between musts, should, could and won’t. Below we summarize important results and challenges identified, followed by a summary of the main user requirements for a system where the operator can also give input towards the process engineers.
Across all companies, existing workflows exist to suggest changes for work instructions. These range from suggestion cards to team meetings with operators and process engineers. However, it can take several weeks for changes to be reflected. Looking specifically at work practices that might impact how operators can be supported to adapt and create work instructions, we note that operators have considerable mobility. For example, firms use buffer operators (sometimes called butterflies) that help others in need or fill in for absent workers.
Additionally, operators follow the assembly line whenever they cannot manage to finish an assembly before the line moves forward. With respect to types of assemblies, a significant challenge is the frequency with which an assembly occurs, with some assemblies occurring very rarely (once in a few months).
Most significantly, participants rarely relied on formal instructions, either digital or paper based, even if these were continually available on the workspace. Exceptions on this include new operators and rare assemblies. Along with this, the assembly instructions provided to operators and the actual assembly might differ, with ad-hoc additions being added to particular models, without these changes being reflected in the actual assembly instructions. This leads to a lot of tacit knowledge that is not shared, which furthermore strengthens the need for strategies to capture operator knowledge.
Additionally, large variations in environmental noise conditions exist across and between pilot-sites. Noise peaks on the assembly floor lead to limitations on the use of voice activated systems. We here limit the discussion of the resulting requirements to the most significant aspects a data acquisition system supporting operators’ input should address.
Operators should not be constrained to a fixed screen or PC that they have to walk towards every time they need to interact with a digital system. In other words, the system should allow physical mobility and not impede mobility when the operators are working (see also Longo et al. [10]).
Since experienced operators do not often need digital tools to support their way of working and are not always tech savvy, it is required that the system should be considered as little intrusive as possible. By introducing this system, the operator can take action when he is already interrupted and should not be interrupted when everything is fine. The system should be quite flexible and expandable for possible changes in the future (see also Funk et al. [6]).
There is the need for the developed systems to work with multiple users. The system should not be used as an exclusive system for each operator separately but be more flexible towards also connecting different operators at the same time.
During the contextual inquiry there was a lot of feedback from the more experienced operators about the younger inexperienced operators. With the main comment that they don’t learn as intensely as the older generation used to because they don’t get the right amount of time and instructions to learn. By taking into consideration that there are many different skill levels between operators the problem would be less of a pressing issue (see also Funk et al. [6]).
A further important consideration is that firms have existing infrastructure for creating instructions. It is important that their own way of working, which is different from company to company, is not obstructed by the integration of the new system. Most of these companies are already using digital systems for displaying work instructions. However, for the capture of feedback in their workspace there is no digital system present.
The interaction with the system will be done using gloves in sometimes dirty environments. These have an impact on the way the operators will interact with the system, considering the fact that the system should always be easy and safe to use. 2.4
After performing the interviews, contextual inquiries and the definition of the requirements, we created user scenarios that sketch how a system that supports operators to input and update digital work instructions could function [8, 9]. Scenarios also have the added benefit that they allow concepts to be materialized and discussed. The user flow was evaluated with the management of the companies.
Implicit in the scenario is that task progression is recognized, in order to limit the need for explicit operator feedback. Flagging is followed by an option to capture the specific issues occurring during the assembly task. These can take the form of either recording an error via wearable devices, an existing array of sensors embedded in the workstation, or optionally requesting assistance from line managers. 3
After making the scenarios concrete we performed a Wizard of Oz study [3]. We focused on how the operators should be able to flag the system, considering different modalities of doing so, while considering important user requirements, including minimal intrusiveness, skill levels and existing infrastructures. 3.1
For the Wizard of Oz study, six participants (three female, average age 26.3 years, SD=1.51) assembled a Fischertechnik model (which is a technical education tool). The participants were university co-workers. The assembly simulates the opening and closing of a refrigerator door, whereby the lights turn on when the door is opened and shut off when the doors close. To align our Wizard of Oz study with factory floor conditions we took several steps. First, we chose an assembly that requires detailed and accurate work (i.e. connecting a wire to the correct input). This level of detail matches the majority of assemblies described in our contextual inquiry. However, we purposefully limited the complexity and length of the assembly. Since we intended to understand how to support the user providing input to work instructions when errors occur and not how to support complex assemblies, an assembly with limited complexity was necessary.
Furthermore, recall that participants rarely - if ever - rely on the written assembly instructions and can usually perform their work from memory. Given this, participants had to learn our assembly task relatively quickly (i.e. after two or three assemblies). However, we did produce digital assembly instructions that were available to participants throughout their assembly, which also mirrors the situation as observed on the factory floor. As a result, participants performed six assemblies. Assembly one to three familiarized them with the task and assembly four to six each contained an assembly error. They knew that errors would occur during the final three assemblies.
However, participants did not know the exact nature of the error, and it varied for every faulty assembly. This prevented participants from learning what was wrong. To allow the Wizard time to setup each assembly and introduce errors, we asked participants to fill out a questionnaire in an adjacent room after each assembly. The questionnaire was about their assembly tasks was based on items adapted from the Dundee Stress State Questionnaire [12].
Thus, for assembly four to six, participants were instructed which way of system interaction (smart glasses, smartwatch or gesture) they should use to record the assembly error. For both the smart glasses and the smartwatch, participants could press a button on the device and the Wizard (who was in the same room) would remotely switch on the mounted camera (via a smartphone). For the final assembly, participants could wave at a second camera located on the desk, which would also trigger the mounted camera (via the Wizard). To ensure that participants knew that the system was filming the mounted camera gave a visual and auditory indication that it was recording. After triggering the system, participants could subsequently point to the error, or explain verbally what they thought was wrong with the assembly. After all the assemblies, there was an interview to find out what worked and what was found to be the best interaction to be used for starting up the video-based error-recording system. Data was captured using two mounted cameras. The goal of this system is to record errors where a remark can be attached to so that the process engineers can optimize the workflow.
A significant concern when doing Wizard of Oz type studies is that the role of the Wizard is too obvious and that the time between a stimulation and response is too lengthy (i.e. interaction latency) [3]. Before asking participants how they felt about the interactions, we first questioned them about how believable they felt the system functions. They were also asked whether they experienced any latency and if this influenced their experience about a particular interaction. The results here were positive, with participants all noting that latency had little impact.
Additionally, five participants found the system believable, without pondering whether it actually worked or whether it was triggered by the Wizard. Of the three ways to provide operator feedback, four participants overall gave preference to variation two, which involved tapping a smartwatch. A common opinion was that a watch is a familiar interface (two participants owned a smartwatch) and it can be useful beyond triggering the system (i.e. you could read the time).
However, it is also important to consider that operators are not always allowed to wear watches on the factory floor and that operators might already own their own watch. By contrast, the smart glasses - which are often mandatory - were viewed less favorably, with four participants remarking that it was an unfamiliar interface, even if some also wore glasses. Additionally, there was some confusion about the exact placement of the button, with two participants having to fumble on the side of the glass for the exact location, which contributed to participants’ negative view. In addition, two participants noted their dissatisfaction with the idea of wearing the glasses the whole day.
However, our third gesture-based interaction caused significant confusion, with three participants unsure whether they were waving correctly, but more importantly, expressing distress that they might inadvertently trigger the system while performing their assembly task (n=4). From a privacy perspective, participants also felt that they were being continuously observed (n=2), but also noted the relative awkwardness of having to wave at the camera (n=2). More generally, participants favored solutions that involved physical feedback (i.e. the button press) (n=5) as opposed the gestural interface, with the exception of participant 6 who noted that the gesture-based interface does not require any other wearables.
Presented with the option of having a physical button on the workspace, four participants responded positively. Inquiring about the system generally, all participants noted that the clear feedback provided by the mounted camera helped participants in feeling that they controlled the system and that it was recording only events that they explicitly wanted to show. From a privacy perspective, this was thus viewed positively. In sum, we also find overlap between the findings of Funk et al. [6] and Werlich et al. [18] who argued for operator control of feedback processes and an emphasis on minimal manual interaction. 4
This paper reports on our first results on the search towards appropriate interfaces that will allow operators working in high-variety low-volume assembly to interact in a minimally intrusive way with work instructions, not only to access information but also to provide feedback. To our knowledge, this is the first study to report on such involvement of operators in keeping instructions up-to-date. Earlier studies primarily focused on creation and usage. Scenarios that explored updating mostly looked at it from an automation perspective. Based on interviews and contextual inquiries within four companies doing high-variety low-volume production, we defined high-level requirements for such a system.
We used these results in a main scenario where operators can give feedback about digital work instructions during assembly tasks with minimal interaction exploiting all contextual information available in the system. This provides a first answer to how operators can give input at workflow level. The two user studies explore different aspects of the interaction in more detail.
The Wizard-of-Oz study (Study 2) gave indications that physical interaction may be preferred to give feedback over gesture input. Participants were positive about the level of control offered on sensors and the provision of clear, multimodal feedback. More refined prototypes and evaluation with (a larger) sample of actual operators are however needed before more definitive conclusions can be drawn on which interface technologies are best suited in such settings.