=Paper=
{{Paper
|id=Vol-2503/paper2_6
|storemode=property
|title=Understanding and Exploring Operator Needs in Mixed Model Assembly
|pdfUrl=https://ceur-ws.org/Vol-2503/paper2_6.pdf
|volume=Vol-2503
|authors=Jamil Joundi,Peter Conradie,Jan Van Den Bergh,Jelle Saldien
|dblpUrl=https://dblp.org/rec/conf/eics/JoundiCBS19
}}
==Understanding and Exploring Operator Needs in Mixed Model Assembly==
https://ceur-ws.org/Vol-2503/paper2_6.pdf
Understanding and Exploring Operator Needs in Mixed
Model Assembly
Jamil Joundi1, 2[0000-0002-3437-1972], Peter Conradie1,2[0000-0003-4495-9136],
Jan Van Den Bergh3[0000-0002-3054-0628] and Jelle Saldien1,2[0000-0003-2557-3764]
1 imec-mict-UGent, De Krook, Ghent, Belgium
2 Department of Industrial Systems Engineering and Product Design, Faculty of Engineering
and Architecture, Ghent University, Kortrijk, Belgium
{jamil.joundi,peter.conradie,jelle.saldien}@ugent.be
3 Hasselt University - tUL - Flanders Make, Expertise Centre for Digital Media, Diepenbeek,
Belgium
jan.vandenbergh@uhasselt.be
Abstract. Assembly operators are experiencing ever-increasing cognitive loads
due to increasing production complexity. Higher quality assembly instructions,
with input from operators by sharing their knowledge, can reduce errors during
the assembly process.
This article describes two studies that investigate the human needs for a digital
process that streamlines the input of operators in the creation or adaptation of
work instructions in a Mixed-Model Assembly Systems. The first study con-
sisted of contextual inquiries and semi-structured interviews and aimed to dis-
cover the high-level needs of the different roles involved in the creation pro-
cess. The second study used the Wizard of Oz method to investigate which in-
teraction methods could be suitable to provide feedback about erroneous as-
semblies.
We found that any systems should take into consideration, among other things,
current operator mobility, presence of multiple operators at a workstation and
different technological skill levels as important factors to consider when devel-
oping new systems to capture operator knowledge. With respect to interaction
methods, participants preferred manual input devices over gestural feedback
methods.
Keywords: Work Instructions, User Centered Design, Assembly, End-User
Feedback.
1 Introduction
Modern manufacturing environments are facing increasing challenges as a result of
more variety in production [4, 14]. More specifically, Just in Time (JIT) [2] produc-
tion has seen the rise of Mixed-Model Assembly Systems (MMAS) [15], where oper-
ators are tasked with the assembly of a wide variety of products. These high complex-
ity production environments subsequently result in increased operator assembly errors
Copyright © 2019 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
2
[5]. To reduce operator cognitive load, recent years have seen efforts towards better
assembly instructions along with calls to including the views of operators when creat-
ing instructions [16].
Recent technological advancements in wearable electronics allow include systems
to measure operator well-being [13]. These include augmented reality systems [11],
or suggestions to integrate smartwatches in assembly [1] partly in an effort to support
operators with assembly tasks.
This article describes 1) a study on exploring the needs of operators in MMAS
along with 2) a Wizard of Oz study on exploring modes of interaction when com-
municating assembly errors. This was done due to the rise in complexity along with
the notion that operators are important when creating assembly instructions. The fo-
cus in the Wizard of Oz study was a task that consisted of flagging production errors.
Our study exists as part of a larger project on how operators can be provided with the
appropriate assembly instructions that leverages their knowledge and minimizes the
work required from both operators and process engineers.
More specifically, we investigated the current situation and requirements for a sys-
tem supporting operator input to adapt digital work instructions through contextual
inquiries with operators. Interviews were performed with those responsible for creat-
ing work instructions of four multinational manufacturing companies (Study 1) in
Flanders, Belgium. Company A assembles airplane wing support. Company B builds
projectors for non-consumer market. Company C assembles agricultural vehicles,
while Company D is a manufacturer of weaving machines, with the focus of the study
situated in the assembly zone where customized weaving machines are assembled.
Based on this information, a flow was created where we focused on the question how
operators can best interact with the work instructions with a Wizard of Oz study
(Study 2). The flow was also discussed with representatives of these companies as
well as potential providers of digital work instructions.
Our research in study 2 shows similarities with the work of Werrlich et al. [18] and
Funk et al. [6]. As where Werrlich et al. focus on the Hololens for interaction, our
study compares interaction on AR-glasses (on the device and gestures) with interac-
tion on a smart watch. In the research of Funk et al. design guidelines are given for
interaction with AR-systems. The ways of interaction were not specified in this re-
search.
2 Study 1: Interviews and Contextual Inquiries
To gain a better insight into the workflow of operators and guide our exploratory
study and experiment, we first conducted semi-structured interviews followed by
contextual inquiries [3] on location; we combined observations with qualitative inter-
views with operators while they worked. In both cases, participants signed informed
consent documents. The ethical commission of Ghent University approved the overall
goal and setup of the study.
3
2.1 Interviews
We performed semi-structured interviews were with line managers and process engi-
neers (n=10) at all locations that participate in this study (3, 3, 3 and 1 persons for
company A, B, C and D respectively). The reason why there was only one process
engineer in the last company was that the line manager was not available. Topics of
discussion included the bidirectional information need from the perspective of man-
agement and operator, the current use of technology by operators, the current work-
flow for creating and updating assembly instruction information, followed by a dis-
cussion about the potential improvements during this process.
2.2 Contextual inquiries
Following formal interviews, we performed 4 contextual inquiries [7] during the
assembly tasks of operators. This process generally entails observing users in their
work context accompanied by in situ informal discussions about their work practices,
or clarifications about particular activities or tasks. Our sessions ranged in duration
from 30 minutes (in the case of Company C where only one assembly was per-
formed), to 2 hours. Where possible, operators were asked about their information
need during assembly, the use of technology, the use of assembly instructions and the
way work is performed (i.e. the order of instructions).
Special attention was paid to how instructions were currently used, how operators
communicate any errors found during the assembly process (i.e. missing parts), and
time pressure during assembly. Additionally, we focused on contextual factors such as
environmental noise, workspace cleanliness, operator mobility, wearing of safety
glasses and gloves. Finally, we also questioned participants about their skill level and
how frequently particular assemblies occur. During observation, we took pictures for
clarification and we used pen and paper for further documentation, with results later
being described digitally by three researchers.
2.3 Results
Both the preceding semi-structured interviews and contextual inquiries were subse-
quently summarized using the MoSCoW requirements prioritization [17], where user
requirements are given a priority between musts, should, could and won’t. Below we
summarize important results and challenges identified, followed by a summary of the
main user requirements for a system where the operator can also give input towards
the process engineers.
Across all companies, existing workflows exist to suggest changes for work in-
structions. These range from suggestion cards to team meetings with operators and
process engineers. However, it can take several weeks for changes to be reflected.
Looking specifically at work practices that might impact how operators can be sup-
ported to adapt and create work instructions, we note that operators have considerable
mobility. For example, firms use buffer operators (sometimes called butterflies) that
help others in need or fill in for absent workers.
4
Additionally, operators follow the assembly line whenever they cannot manage to
finish an assembly before the line moves forward. With respect to types of assem-
blies, a significant challenge is the frequency with which an assembly occurs, with
some assemblies occurring very rarely (once in a few months).
Most significantly, participants rarely relied on formal instructions, either digital or
paper based, even if these were continually available on the workspace. Exceptions on
this include new operators and rare assemblies. Along with this, the assembly instruc-
tions provided to operators and the actual assembly might differ, with ad-hoc addi-
tions being added to particular models, without these changes being reflected in the
actual assembly instructions. This leads to a lot of tacit knowledge that is not shared,
which furthermore strengthens the need for strategies to capture operator knowledge.
Additionally, large variations in environmental noise conditions exist across and
between pilot-sites. Noise peaks on the assembly floor lead to limitations on the use
of voice activated systems. We here limit the discussion of the resulting requirements
to the most significant aspects a data acquisition system supporting operators’ input
should address.
Mobile operators
Operators should not be constrained to a fixed screen or PC that they have to walk
towards every time they need to interact with a digital system. In other words, the
system should allow physical mobility and not impede mobility when the operators
are working (see also Longo et al. [10]).
Minimally intrusive
Since experienced operators do not often need digital tools to support their way of
working and are not always tech savvy, it is required that the system should be con-
sidered as little intrusive as possible. By introducing this system, the operator can take
action when he is already interrupted and should not be interrupted when everything
is fine. The system should be quite flexible and expandable for possible changes in
the future (see also Funk et al. [6]).
Multiple people
There is the need for the developed systems to work with multiple users. The sys-
tem should not be used as an exclusive system for each operator separately but be
more flexible towards also connecting different operators at the same time.
Different skill levels
During the contextual inquiry there was a lot of feedback from the more experi-
enced operators about the younger inexperienced operators. With the main comment
that they don’t learn as intensely as the older generation used to because they don’t
get the right amount of time and instructions to learn. By taking into consideration
that there are many different skill levels between operators the problem would be less
of a pressing issue (see also Funk et al. [6]).
5
Existing infrastructure
A further important consideration is that firms have existing infrastructure for cre-
ating instructions. It is important that their own way of working, which is different
from company to company, is not obstructed by the integration of the new system.
Most of these companies are already using digital systems for displaying work in-
structions. However, for the capture of feedback in their workspace there is no digital
system present.
Environmental conditions
The interaction with the system will be done using gloves in sometimes dirty envi-
ronments. These have an impact on the way the operators will interact with the sys-
tem, considering the fact that the system should always be easy and safe to use.
2.4 Defining the design space
After performing the interviews, contextual inquiries and the definition of the re-
quirements, we created user scenarios that sketch how a system that supports opera-
tors to input and update digital work instructions could function [8, 9]. Scenarios also
have the added benefit that they allow concepts to be materialized and discussed. The
user flow was evaluated with the management of the companies.
Implicit in the scenario is that task progression is recognized, in order to limit the
need for explicit operator feedback. Flagging is followed by an option to capture the
specific issues occurring during the assembly task. These can take the form of either
recording an error via wearable devices, an existing array of sensors embedded in the
workstation, or optionally requesting assistance from line managers.
3 Study 2: Exploratory Wizard of Oz study
After making the scenarios concrete we performed a Wizard of Oz study [3]. We
focused on how the operators should be able to flag the system, considering different
modalities of doing so, while considering important user requirements, including
minimal intrusiveness, skill levels and existing infrastructures.
3.1 Method
For the Wizard of Oz study, six participants (three female, average age 26.3 years,
SD=1.51) assembled a Fischertechnik model (which is a technical education tool).
The participants were university co-workers. The assembly simulates the opening and
closing of a refrigerator door, whereby the lights turn on when the door is opened and
shut off when the doors close. To align our Wizard of Oz study with factory floor
conditions we took several steps. First, we chose an assembly that requires detailed
and accurate work (i.e. connecting a wire to the correct input). This level of detail
matches the majority of assemblies described in our contextual inquiry. However, we
6
purposefully limited the complexity and length of the assembly. Since we intended to
understand how to support the user providing input to work instructions when errors
occur and not how to support complex assemblies, an assembly with limited complex-
ity was necessary.
Fig. 1. Left: Part of the assembly instructions provided to participants. Right: recorded view
from participant after triggering the mounted camera using safety glasses.
Furthermore, recall that participants rarely - if ever - rely on the written assembly
instructions and can usually perform their work from memory. Given this, participants
had to learn our assembly task relatively quickly (i.e. after two or three assemblies).
However, we did produce digital assembly instructions that were available to partici-
pants throughout their assembly, which also mirrors the situation as observed on the
factory floor. As a result, participants performed six assemblies. Assembly one to
three familiarized them with the task and assembly four to six each contained an as-
sembly error. They knew that errors would occur during the final three assemblies.
However, participants did not know the exact nature of the error, and it varied for
every faulty assembly. This prevented participants from learning what was wrong. To
allow the Wizard time to setup each assembly and introduce errors, we asked partici-
pants to fill out a questionnaire in an adjacent room after each assembly. The ques-
tionnaire was about their assembly tasks was based on items adapted from the Dun-
dee Stress State Questionnaire [12].
Thus, for assembly four to six, participants were instructed which way of system
interaction (smart glasses, smartwatch or gesture) they should use to record the as-
sembly error. For both the smart glasses and the smartwatch, participants could press
a button on the device and the Wizard (who was in the same room) would remotely
switch on the mounted camera (via a smartphone). For the final assembly, participants
could wave at a second camera located on the desk, which would also trigger the
mounted camera (via the Wizard). To ensure that participants knew that the system
was filming the mounted camera gave a visual and auditory indication that it was
recording. After triggering the system, participants could subsequently point to the
7
error, or explain verbally what they thought was wrong with the assembly. After all
the assemblies, there was an interview to find out what worked and what was found to
be the best interaction to be used for starting up the video-based error-recording sys-
tem. Data was captured using two mounted cameras. The goal of this system is to
record errors where a remark can be attached to so that the process engineers can
optimize the workflow.
Results
A significant concern when doing Wizard of Oz type studies is that the role of the
Wizard is too obvious and that the time between a stimulation and response is too
lengthy (i.e. interaction latency) [3]. Before asking participants how they felt about
the interactions, we first questioned them about how believable they felt the system
functions. They were also asked whether they experienced any latency and if this
influenced their experience about a particular interaction. The results here were posi-
tive, with participants all noting that latency had little impact.
Additionally, five participants found the system believable, without pondering
whether it actually worked or whether it was triggered by the Wizard. Of the three
ways to provide operator feedback, four participants overall gave preference to varia-
tion two, which involved tapping a smartwatch. A common opinion was that a watch
is a familiar interface (two participants owned a smartwatch) and it can be useful
beyond triggering the system (i.e. you could read the time).
However, it is also important to consider that operators are not always allowed to
wear watches on the factory floor and that operators might already own their own
watch. By contrast, the smart glasses - which are often mandatory - were viewed less
favorably, with four participants remarking that it was an unfamiliar interface, even if
some also wore glasses. Additionally, there was some confusion about the exact
placement of the button, with two participants having to fumble on the side of the
glass for the exact location, which contributed to participants’ negative view. In addi-
tion, two participants noted their dissatisfaction with the idea of wearing the glasses
the whole day.
However, our third gesture-based interaction caused significant confusion, with
three participants unsure whether they were waving correctly, but more importantly,
expressing distress that they might inadvertently trigger the system while performing
their assembly task (n=4). From a privacy perspective, participants also felt that they
were being continuously observed (n=2), but also noted the relative awkwardness of
having to wave at the camera (n=2). More generally, participants favored solutions
that involved physical feedback (i.e. the button press) (n=5) as opposed the gestural
interface, with the exception of participant 6 who noted that the gesture-based inter-
face does not require any other wearables.
Presented with the option of having a physical button on the workspace, four par-
ticipants responded positively. Inquiring about the system generally, all participants
noted that the clear feedback provided by the mounted camera helped participants in
feeling that they controlled the system and that it was recording only events that they
explicitly wanted to show. From a privacy perspective, this was thus viewed positive-
ly. In sum, we also find overlap between the findings of Funk et al. [6] and Werlich et
8
al. [18] who argued for operator control of feedback processes and an emphasis on
minimal manual interaction.
4 Conclusion
This paper reports on our first results on the search towards appropriate interfaces
that will allow operators working in high-variety low-volume assembly to interact in a
minimally intrusive way with work instructions, not only to access information but
also to provide feedback. To our knowledge, this is the first study to report on such
involvement of operators in keeping instructions up-to-date. Earlier studies primarily
focused on creation and usage. Scenarios that explored updating mostly looked at it
from an automation perspective. Based on interviews and contextual inquiries within
four companies doing high-variety low-volume production, we defined high-level
requirements for such a system.
We used these results in a main scenario where operators can give feedback about
digital work instructions during assembly tasks with minimal interaction exploiting all
contextual information available in the system. This provides a first answer to how
operators can give input at workflow level. The two user studies explore different
aspects of the interaction in more detail.
The Wizard-of-Oz study (Study 2) gave indications that physical interaction may
be preferred to give feedback over gesture input. Participants were positive about the
level of control offered on sensors and the provision of clear, multimodal feedback.
More refined prototypes and evaluation with (a larger) sample of actual operators are
however needed before more definitive conclusions can be drawn on which interface
technologies are best suited in such settings.
References
1. Aehnelt, M., Urban, B.: Follow-Me: Smartwatch Assistance on the Shop Floor. In:
HCI in Business. pp. 279–287 Springer International Publishing (2014).
2. Claeys, A. et al.: Framework for evaluating cognitive support in mixed model
assembly systems. IFAC Proc. Vol. 48, 3, 924–929 (2015).
3. Dahlbäck, N. et al.: Wizard of Oz studies - why and how. Knowledge-Based Syst. 6, 4,
258–266 (1993).
4. ElMaraghy, W. et al.: Complexity in engineering design and manufacturing. CIRP
Ann. 61, 2, 793–814 (2012).
5. Fast-Berglund, Å. et al.: Relations between complexity, quality and cognitive
automation in mixed-model assembly. J. Manuf. Syst. 32, 3, 449–455 (2013).
6. Funk, M. et al.: An Overview of 4 Years of Combining Industrial Assembly with
Augmented Reality for Industry 4.0. Proc. 16th Int. Conf. Knowl. Technol. Data-
driven Bus. 2–5 (2016).
7. Holtzblatt, K., Beyer, H.: Contextual design. In: CHI ’99 extended abstracts on Human
factors in computing systems - CHI ’99. (1999).
8. Lambert, J., Lambert, J.: Storyboarding. Digit. Storytell. 115–120 (2018).
9
9. Landay, J.A., Myers, B.A.: Sketching storyboards to illustrate interface behaviors. In:
Conference companion on Human factors in computing systems common ground -
CHI ’96. pp. 193–194 ACM Press, New York, New York, USA (1996).
10. Longo, F. et al.: Smart operators in industry 4.0: A human-centered approach to
enhance operators’ capabilities and competencies within the new smart factory
context. Comput. Ind. Eng. 113, 144–159 (2017).
11. Makris, S. et al.: Augmented reality system for operator support in human–robot
collaborative assembly. CIRP Ann. 65, 1, 61–64 (2016).
12. Matthews, G. et al.: Validation of a comprehensive stress state questionnaire: Towards
a state big three. In: Deary, I.J. et al. (eds.) Personality psychology in Europe, Vol 7.
pp. 335–350 , Tilburg (1999).
13. Mattsson, S. et al.: Assessing Operator Wellbeing through Physiological
Measurements in Real-Time—Towards Industrial Application. Technologies. 5, 4, 61
(2017).
14. Michalos, G. et al.: Automotive assembly technologies review: challenges and outlook
for a flexible and adaptive approach. CIRP J. Manuf. Sci. Technol. 2, 2, 81–91 (2010).
15. Miltenburg, J.: Level Schedules for Mixed-Model Assembly Lines in Just-In-Time
Production Systems. Manage. Sci. 35, 2, 192–207 (1989).
16. Söderberg, C. et al.: Design of simple guidelines to improve assembly instructions and
operator performance. In: The sixth Swedish Production Symposium. (2014).
17. Stapleton, J.: DSDM: Dynamic Systems Development Method: The Method in
Practice. (1997).
18. Werrlich, S. et al.: Design recommendations for HMDbased assembly training tasks.
In: CEUR Workshop Proceedings. pp. 58–68 CEUR-WS (2018).