Involvement of end user in Scrum-driven organizations: Four anti- patterns Md Tanvir Hasan1,∗,†, Annika Wolff2,† and Antti Knutas3,† Software Engineering & Digital Transformation, Lappeenranta Lahti University of Technology, Lappeenranta, Finland Abstract Agile software development processes, especially Scrum, have become more common in the software industry due to the fundamental principles that focus on speed and communication. However, it is noteworthy that even though the Scrum framework emphasizes frequent stakeholder involvement, the involvement of end users is often not a primary focus within organizations using Scrum, nor is usability a high-priority task. In the worst cases, the result is a low-quality product that is unable to meet user needs. To better understand the role of end users and how usability issues are handled in Scrum processes in practice, we performed interviews with professionals practicing Scrum, focusing on two areas: 1) the role of end users and their involvement in the Scrum process, 2) the importance of usability and usability evaluation techniques used in practice. Our results identify four novel anti- patterns in organizations using Scrum: 1) Involvement of end user is not mandatory, and a customer maybe used to represent an end user; 2) Product demonstration occurs with no end user involvement; 3) Low focus on usability and usability is often ignored; 4) No formal usability techniques used in Scrum process. Based on our interviews we discuss the reasoning of why certain practices are performed by a Scrum practitioner. Finally, the study is subject to several limitations that have also been addressed. Keywords Scrum Challenges, End User Involvement, Agile Software Development Process, Usability 1 1. Introduction the development process. A systematic literature review on “Obstacles Faced by Requirements Engineering Agile is a popular approach within software Professionals in an Agile Context by Applying User development processes [31]. However, requirements Experience” also showed that “lack of user participation” engineering (RE) challenges in agile have been identified is the common problem mentioned in most of the studies by many studies. Agile software development is a [1]. More specifically, to identify agile requirements lightweight method, that focuses on delivering working engineering practices and challenges a systematic software, organizing, and making coding more efficient. literature review was conducted by Inayat et al. [10]. The As such, issues related to usability are often neglected, study identified eight challenges posed by agile and end user involvement – while advocated for is not requirements engineering and one of them is neglecting mandatory in the process [3, 5]. A Delphi study was nonfunctional requirements such as testability and conducted by Schön et al. [20] with 26 experts in agile to external quality usability. According to Helmy et al. [9] identify the key challenges in agile requirement the issue related to nonfunctional requirements should engineering. The result showed six challenges for be taken care of to improve the product quality. The industry practicing agile processes and two of these aforementioned factors demand the necessity for challenges are related to stakeholders and users. A engaging end users and focus more on usability when summary of these challenges is i) lack of involvement of enacting agile methodologies. direct end users ii) difficulty in involving stakeholders in TKTP 2024: Annual Doctoral Symposium of Computer Science, 10.-11.6.2024 Vaasa, Finland ∗ Corresponding author. † These authors contributed equally. Md.Tanvir.Hasan@lut.fi (M.T. Hasan); Annika.Wolff@lut.fi (A. Wolff); Antti.Knutas@lut.fi (A. Knutas) 0000-0002-6628-4152 (M.T. Hasan); 0000-0002-6638-6677 (A. Wolff); 0000-0002-6953-0021 (A. Knutas) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings In this paper, we focus on inspecting the most having communication only with the customer. It is popular agile approach, Scrum [22], and how Scrum is possible that the customer who is representing the end utilized in industry. Scrum is a lightweight agile project user is inexperienced, not efficient in providing feedback management framework mostly used for software and not having enough knowledge. In that case the development. Scrum focuses on frequent stakeholder product may not work well with the end user [27]. There involvement and short releases, which is why Scrum is are exceptions where a sample of real users are also broadly adopted by industries. Since Scrum is an agile involved, but this happens only occasionally. Usually, it approach, it also faces some of these same requirements is the customer who represents user needs, requirements engineering challenges when put into practice [23]. and gives feedback [24]. Specific challenges include lack of time to focus on the Even the role of end users versus customers is still end user’s needs [2], lack of techniques for the design unclear in many software development companies [18]. process [21], and usability requirements being ignored Whilst doing an analysis on the fitness of Scrum and [29]. However, proper utilization of Scrum depends on Kanban towards user experience, researchers [25] professionals. The identified challenges are not a Scrum noticed that interviewees were using the terms framework challenge but a Scrum application challenge. “customer” and “user” as synonyms which motivated According to the Scrum guide the Scrum framework is them to do additional research on the issue. The result purposely incomplete. Different types of techniques and showed that whilst many respondents did understand methods can be used within the framework in different the difference between “user” and “customer” there were situations and environments [8]. So, our goal is to focus still a significant number who were unclear about the on how practitioners are involving end users in Scrum differences and couldn’t answer properly or answered processes and how they are ensuring usability of the incorrectly. product. The paper aims to answer the following One of the core values of agile development is that research questions: working software should be demonstrated to the end In practice what is the role of end users? users throughout short development iterations. The How are they involved in the Scrum process? reason behind this value is that working software elicits How are usability issues handled? more detailed information toward understanding end Our results identify four novel anti patterns in user needs than written words. However, the situation organizations using Scrum. Here we refer to anti is different in practical life. In one organization, team patterns as the common Scrum practices that appear members were interviewed after a transformation to convenient in the beginning but harmful in the long run Scrum from a more traditional waterfall process. In their and therefore must be avoided [28]. The results also interview, they specified that they never did a full provide holistic understanding of the challenges faced product demonstration. In other words, there wasn’t any by software companies related to end user involvement demonstration after each sprint where they could gather when employing Scrum. feedback. Instead, they delivered the software when all The structure of the paper organized as follows. the required functions were working. So essentially, Section 2 provides related work. Section 3 discusses the they ended up doing a waterfall approach every three research methodology and data analysis. The results of weeks [4]. the study are presented on Section 4 and discussed on Section 5. Section 6 and Section 7 respectively address 2.2. Usability techniques used in Scrum the limitations and conclusions of the study. In many Scrum projects the Scrum team (product owners & developers) concentrate more on the goal of 2. Related work implementing a working function rather than the 2.1. Involvement of end user in Scrum usability or usefulness of that function. A common argument here is that the usability can be improved later Normally in Scrum, most frequent review activities such [17]. In practice, the overall result is a product that may as end of sprint review and testing involve the product obtain all the functional requirements but still fails in the owner and the development team. End users are not market due to poor user satisfaction [14]. In [11] authors commonly involved in these activities, but instead may performed a survey study to find what usability be involved in feedback sessions when they are using the techniques are used in Scrum processes and how often product. By this stage, change is still possible, but they are used. They listed 13 usability techniques used requires costly rework [15]. Even though agile processes by the practitioners. Later, they performed another such as Scrum are iterative approaches that conduct user survey to find the usefulness and usage of these 13 testing in the process, when put into practice Scrum techniques. Results showed that according to 75% of the often does not focus on creating user experience based respondents, formal usability evaluation with users is a on user needs [2]. Most of the time, the Scrum team is very good technique but only 30% of participants used formal usability in their project. The author mentioned been interested in end user’s perspectives in Scrum, the one possible explanation and that is that Scrum is all research does not reveal what is the role of end users in about speed. Scrum sprints are short (2-4 weeks). It takes practice, or how practitioners are (or are not) involving a longer time to use a specific usability technique which them. Therefore, our research will focus on may slow the whole Scrum process down. Also, the understanding not just what gaps exist in end user output of such sprints is minor, and evaluating them in involvement in practice but why the involvement of end a quantitative way is not useful as changes are small. users and usability is often ignored. Therefore, Scrum professionals tend to rely on informal usability techniques such as lo-fi prototyping, and 3. Method meetings with users instead of formal techniques such as heuristic evaluation, questionnaires, field studies, To familiarize ourselves with the issues related to end scenarios, or digital prototyping. One advantage of user involvement and usability within companies’ agile informal usability techniques is they can be performed processes, we conducted semi-structured interviews quickly without prior preparation. In some cases, with professionals having experience working with informal usability techniques are helpful, but the main Scrum or working as a product owner (PO), scrum drawback of this approach is the number of users master (SM) or developers in a Scrum driven software involved. Only a few users are taking part in informal organizations. We continued conducting interviews usability evaluation, so evaluators are not sure if they until a saturation point was reached in which no new can rely on the result or not [12]. In agile, especially information was emerging, which happened after 11 within Scrum, usability requirements and their interviews had been conducted. We observed implementations are ignored most of the time. Usability consistency among the interviewees as they articulated or usefulness is often ignored or a less prioritized task in recurring themes and issues. All interviews were a Scrum process. Usability features are often absent in conducted online (MS Teams) and recorded by one product backlog [29]. Another study result, on the user researcher. The interviews lasted from 45 minutes to an perspective in Scrum practice, showed that the hour. involvement of a user in the process is informal, meaning that user involvement is done based on an 3.1. Participants individual’s own initiative, preferences and knowledge. The 11 individuals were selected randomly through There’s no systematic plan or approach within the personal networks, recommendations from people who development process and often there’s no specific had already been interviewed and via a post on social person responsible for usability issues, nor any clear media (Facebook, LinkedIn). All individuals who usability goals. As such, the user perspective is only participated in the research had practical experience considered if the product owner or team has an interest working in Scrum or Agile as SM, PO and developer. in usability [6]. One survey study was conducted on user There were in total 9 males and 2 females. More involvement methods in Icelandic software industry information can be found in Table 1. [13]. 37% of the respondents were using Scrum as the software development process and rest of the Table 1 respondents were using other methods including Description of interviewees organization’s own process models, waterfall, and XP. Number Role Experience The result shows that developers who were using Scrum of are the most skeptical compared to others when asked interviewees about whether usability is important or not. The 6 SM, PO 4-15 years definition of usability within that study described it as 2 SM 1 year “Usability is a qualitative attribute that assesses how 2 SM, PO 2-5 years easy user interfaces are to use. Usability is mainly made 1 SM, PO, 5 years up of three factors: Effectiveness – Can the users solve Developer their tasks with the software? Efficiency – Can the user solve their tasks without major problems? Satisfaction – How satisfied are the users? Most of the literature on Agile or Scrum mainly When participants were asked about what type of focused on integrating UCD, UX [2, 17, 5, 3, 1, 15, 25] or software development process they were using, most of requirement engineering challenges [4, 31, 7, 20, 10, 9]. them didn’t answer directly Scrum. Only two Usability issues are considered in several studies [11, 12, participants replied they were using Scrum. The Scrum 29, 13, 6] but with no proper discussion on the reasons process that others were using had been modified to for ignoring usability. While several researchers have adapt to the needs, requirements, and context of the organization. Similar to the outcome presented in [25], most of the organizations were using features of Scrum education, experience) and sign the ethical form and combined Scrum with another agile development (voluntary participation & data privacy), we performed process, Kanban. A mix of Scrum and Kanban. a survey, using the GDPR-compliant survey tool Participants answered the questions based on the project Webropol, prior to the interview. they were working on. 3.2. Data analysis Table 2 Example of coded transcripts Interview recordings were transcribed and analyzed with inductive thematic analysis [30]. The first phase Transcript Codes was to get familiarized with the data by reading transcripts, listening to the audio recordings, and making notes. In our case we highlighted all potential Interviewee1: “It Have weekly transcribed conversations related to our research is a weekly sprint review question, putting notes and rephrased when necessary. meeting for the session. The second phase was to generate initial codes which whole software Involvement of provided us with a summary of the highlighted team. So, all the the software conversations or describing the contents (see Table 2). software teams team In phases 3 and 4 we constructed potential themes where the No end user from codes and reviewed the themes against the software teams involvement extracted data. We reviewed all coded data from present what interviews for each question to generate themes, they have collapsing potential themes together or splitting them accomplished.” into multiple different themes. Example: For question, “What is the role of end user in your Scrum process?”, we Interviewee2: Have sprint noticed that the codes focused on either involving end “Two weeks review users in the Scrum process or no involvement of the end sprint and sprint sessions. users at all. We then constructed one theme containing review session. No direct end all codes relating to no end user involvement in the So, I haven't user Scrum process and one theme relating to involving end really got to involvement users. We further divided the theme involving end user invite any of the Involvement of into several different themes: involving end users, like real end expert from involving end users through product owner, involving users, but we end user end users through customer, involving end users have from some organization through user representative, involving end users sort of the through sales or marketing team. experts from the In phase 5 we defined and named the themes end user addressing our research questions. We also noticed organization.” some identified themes can be subthemes of another theme. Example: involving end users through the Interviewee3: Product owner product owner, involving end users through the “The product is having direct customer, involving end users through user owner listens to or indirect representative, and involving end users through the the customers connections service or marketing team can be described as either directly or with end users subthemes of a broader theme “indirect involvement of indirectly.” to list the end users”. requirements 4. Results They were also asked to answer from one previous Here the interview results will be presented focusing on project where they had to involve end users, or the two major areas: 1) the role of end users in Scrum, 2) the involvement of end users was mandatory. The interview importance of usability and usability evaluation questions were designed to be adaptable to the level of techniques used in practice. experience of the individuals and in accordance with the situation. The interview questions focused on two areas: 1) end user and 2) usability in the Scrum process. To collect demographic, background information (role, 4.1. Involvement of end users in sprint 4.3. Role of end user in the Scrum review session process According to the Scrum guide, a sprint review is one of After defining end user and customer, all interviewees the mandatory events to gather feedback from key were asked about the role of the end user in their Scrum stakeholders (which includes end users) but in practice, process. Analysis of the answers resulted in three involving end users is not necessary. In sprint reviews, different themes: a) Indirect involvement of end users b) feedback sessions and testing are often done without No involvement of end users c) Direct involvement of any involvement of end user. Instead, domain experts end users. Four out of eleven interviewees mentioned are playing the role of an end user. In almost all cases, that they are indirectly involving end users through a major participants in sprint review are the Scrum team middle person such as a product owner, sales team, UX and internal stakeholders or customer representatives designers, customers, or user representatives. However, who play the role of an end user. In this case, it is hard one interviewee also mentioned that even though end to determine how relevant the feedback is with respect users are not involved in the Scrum process, they have to the needs of real end users [26]. Almost half of the close contact with them. Almost half of the interviewees interviewees (five out of eleven) answered that they are (5 out of 11) replied they are having no end user only involving internal stakeholders in the sprint review involvement in their Scrum process. Some are involved, session. Three interviewees had two different sessions but only in special cases: “I would say that no, not in but only on demand. One is for the internal stakeholders daily, like all the sprints that we have. But it will be more and another session is for the customers or end users. of a special case when we have a special need. So, it is more Three interviewees informed us that they are involving of this special occasion when we decide that, hey, OK, now, external stakeholders in the sprint review session. we want to talk with them.” Stakeholders are those groups of people who are “But in the scrum process we don't involve the end user impacted by the product outcome. Based on our analysis, actually”. we identified the following categories of stakeholders Only 2 interviewees were directly involving end users in from interviews. The categorization is necessary to their process however one of them stated that they are understand different types of stakeholder interviewees involving end users only when they need, not on a addressed during the interview: regular basis: “We haven't gotten them involved in the like every Sprint, so to speak, but there when we do something 1. External stakeholders: Customers who finance some kind of feature a little bit bigger thing we usually uh the project and end users who use the product set up some kind of pilot group and we talk to them over or benefit from the product. In many cases teams and sort of gather their feedback from that” customers and end users can be the same group of people. 4.4. Ensuring end users' feedback during 2. Internal stakeholders: Management of the the development process company, Scrum team, product owners, program manager, functional manager, We already noticed from the results that most of the supervisors, customer support people, UX interviewees are not involving end users in their Scrum designer. In many cases an internal process, so how do they ensure their feedback is heard? stakeholder can also be a customer. Everyone described their own different approaches. We categorized them into two sections: 1) getting feedback 4.2. Defining the term “end user” and from end users; 2) getting feedback from user “customer” representatives: “but their customer make decision should we do the changes or not? We have these different persons Customer and end user are different terms, and both who are basically listening the users.” 8 participants were have different roles in the development process. Scrum collecting feedback from user representatives. Only 3 teams and literature sometimes do not specify the participants mentioned that feedback is coming directly differences between a customer and an end user in a from end users. project. From the previous research [25, 18] we identified that agile practitioners are not always clear on 4.5. Importance of usability, user the role of end user and customer. The purpose of experience and benefits of defining “end user” and “customer” was to determine the involving end users in the Scrum participant’s understanding of these terms. Based on the process answers almost all participants had a proper understanding and clear distinction between the two Even though from the responses it is apparent that end terms “customer” and “end user”. users are seldomly involved in the Scrum process, almost all of the participants (8 out of 11) agreed that having regular product demos and another two had usability or UX is very important, and their priority is irregular product demos. The rest of the interviewees either high or very high: “it is always about providing the had product demonstrations with no end user user with the best experience. So, it’s definitely the most involvement. important thing.” “I would say that it should not be the highest but one of 4.8. Main challenges of involving end the highest. If it doesn’t satisfy a customer, then what’s the users in Scrum process point anyway.” We asked participants to point out the challenges of However, it was also pointed out from the conversation involving end users in Scrum from their own that the priority of usability or UX also varies from case experiences. It was interesting to see that there wasn’t to case: “let's say if the customer or end user is some that much diversity in the answers. Almost everyone external customer or external client, then I would say the had similar opinions. There were mainly two types of user experience should come first but for internal answers: 1) User characteristics, expertise, and customers the situation is a bit different.” Furthermore, background; 2) Scrum’s limitation in facilitating optimal participants were asked about their opinions on the end user engagement. According to the interviewees, benefits of involving end users and two themes emerged choosing the right end user is very important and a big from the analysis: 1) Enhanced User Understanding and challenge: 2) Enhanced Product Quality. “I would say it depends on the product and what you 4.6. Techniques or methods used to consider as end user and it it's also very important to choose the right end users.” collect feedback or evaluate the “You need to get involved the right people on the right time usability of the product and on the right place.” When asked about the methods used for usability Some interviewees also pointed out one issue and that is evaluation and collecting feedback, we got quite diverse lack of knowledge and experience. Sometimes you need answers. We tried to sort the answers into two to train the general end users to get meaningful categories: 1) formal usability evaluation; 2) informal feedback. Also, the feedback should not come from just usability evaluation (no systematic plan or approach one individual’s opinion. Feedback should come from sometimes based on individuals’ own initiative). Only 3 the whole user group. So, it is always easier to involve participants mentioned that they are using some type of someone knowledgeable and experienced who’s able to process although two of these participants are not using represent end user groups such as super users or user any techniques in the current project. They referred to a representatives. The other challenges mentioned by past project where they have used formal usability interviewees are all related to Scrum’s limitation in techniques such as questionnaires, or feedback tools. facilitating optimal end user engagement. The most Only one participant was using formal methods for common challenge was not being able to include end usability testing. The rest of the participants didn’t have users in the sprint review process. According to the any systematic plan or approach to collect feedback or interviewees, the Scrum sprint review is intended to be evaluate usability. Mostly they were using team only for the internal team members. Involving general conversations and sending emails: “Uh, yeah, just uh, we users is often not possible as these sessions are mostly are sometimes requesting this information in teams.” technical: “they're talking sort of user language and we're Some also mentioned initiating meetings when needed: talking developer language. So, if you involve them in the “Uh is the informal techniques, so if we need it then we same kind of in this well in the Scrum, scrum day-to-day initiate a meeting request and then book a calendar and or Sprint week to week thing, then that could be a little bit then get the feedback from them”. challenge.” One interviewee mentioned that it is the role of the product owner to translate the technical language 4.7. Product demonstration with end and have conversations with end users. Involving end user users in sprint review just will create a chaotic environment and also might delay the whole process. Product demonstration is one of the core values of Agile Furthermore, one interviewee added that Scrum focuses to understand end user needs. Almost all interviewed more on delivery so fitting user’s perspective in two to individuals informed us that they are having either three weeks sprints is hard. End user involvement needs regular or irregular product demonstrations. But our different planning. goal was to find out if they are having these demonstrations with end users or not. Analysis showed that only three interviewees were involving end users in product demonstrations, where one company was 5. Discussion important, such as the telecommunication sector where the priority is to have a system that works as a mediator 5.1. Involvement of end user is not to other systems. Optimizing something for human use mandatory, and a customer is not always a key requirement. So, the necessity of maybe used to represent an end involving end users in the Scrum process varies case by user case. There are several implications for Scrum due to low user 5.2. Product demonstration occurs with involvement, these are (1) software testing: In Scrum, no end user involvement testing is done by the product owner and development team instead of the user, (2) Poor software quality: The A product demonstration can be useful after short user might not be satisfied from the software as no development iterations to better understand user needs. testing has been performed, (3) No feedback: There are However, even though product demonstration is one of no feedback sessions after sprints. A feedback session the core values of agile, in practice it is not happening. takes place after the software has been created. As a We found out from our study that in most cases, product result, the Scrum team has little idea if they are going in demonstrations happen without any involvement of the the right direction or not [15]. In our interviews we end user. In few cases, users are involved but the session asked interviewees about the role of end users in their is not regular. Only when there’s something big to Scrum process and whether they are involved in sprint demonstrate. Based on our results reasons behind not review sessions. The results show that the majority of having a product demonstration are: 1) the outcome of a the interviewees are not involving end users in their 2–4-week sprint lacks substantial significance for Scrum process. Interviewees who agreed that they are demonstration; 2) thinking of the product demo as a involving end users mostly having indirect technical event only for the internal stakeholders; 3) lack communication with the end user through product of communication with end users. owners, user representatives, or customers. From studies, it is indicated that customer representatives 5.3. Low focus on usability and usability often have daily contact with the developers which can is often ignored make them too familiar with the inner software working Usability is not something that is only related to the process. So, they cannot truly represent an actual end aesthetic or visual design of the interface. Usability has user during the feedback session [21]. According to the a deep relationship with Human-Computer Interaction Scrum guide it is true that the product owner should be (HCI). There are several human factors in HCI and if the link between end user and development team. But in they are implemented properly the usability of software practice this is not always true. One study [19] showed could be improved. Studies have shown that even that some product owners think that identifying end completed software increments have sometimes been users need and communicating those needs with rejected by the end user due to poor usability [29]. Our development team is not product owner’s responsibility results contradict the previous research outcome [13], but a team effort. Also, product owners in Scrum are [29] on the importance of usability among Scrum mostly overwhelmed with marketing and sales concerns practitioners as our study shows that almost all the and often lack formal education and skills to design participants agreed that usability or user experience is effective user experiences [17]. Sprint review sessions very important. But no matter how important usability are mainly for the internal Scrum team or internal or user experience is according to the interviewee’s stakeholders. Only a few mentioned they involve answers, their focus and necessity in the Scrum process external stakeholders, but external stakeholders are not vary based on the product and customer. As mentioned always end users. According to one of these before, end users and customers can be the same group interviewees, their external stakeholders are experts of people. The customer can be an internal or external from an end user organization. So, our study results stay stakeholder. The priority of usability depends on the partially in line with [2, 26, 28, 15] that product owners, nature of the customer. For external stakeholders, the user representatives, or customers play the role of end priority is high: “let’s say if the customer or end user is users and participate in the Scrum sprint review. End some external customer or external client, then I would say user involvement is not mandatory in the Scrum the user experience should come first. But for internal process. We pointed out from our interviews that some customers the situation is a bit different. The expectations of the interviewees are developing products for internal are different, so they just want something that works for customers who are also end users. When end users are them, so they don’t care about the usability or such.” The internal customers then the focus is more on same goes for the product a Scrum team is developing or functionality rather than usability. Moreover, some the service they are providing. For the team who are interviewees were working in sectors where UCD is not developing products to support internal stakeholders, 5.4. No formal usability techniques used usability is not the top priority. But if the team is in Scrum process developing something for an external stakeholder such as a mobile app for a bank’s general users, then the Professionals practicing Scrum often do not follow any priority is high. Based on the interviewee’s answers, UX systematic approach to usability. Scrum practitioners or usability is not important or a top priority task in the rely on informal qualitative evaluation. There’s a case of internal stakeholders who are also the customer difference between the top usability techniques and and end users. According to [13], the first two factors of usability techniques commonly used by Scrum usability are: Effectiveness – Can the users solve their practitioners. Even though some techniques are tasks with the software? Efficiency – Can the user solve effective, they are not being used in practical situations. their tasks without major problems? - means the product Our results show that the commonly used usability a team is developing should be accessible to the end user technique by the participants is informal usability regardless of whether the end user is an internal or evaluation. Only one participant was using a specific external stakeholder. This leads us to a new argument user testing method (formal usability evaluation) in their that Scrum practitioners do not always have a very good ongoing project. Our result stays consistent with understanding of UX and usability. practitioners’ top and frequently used usability A common reason for neglecting usability issues in techniques [11] which discussed that formal usability Scrum is having a short time frame. Scrum sprints are techniques are not used by practitioners even though very short, and the focus is on the delivery. According practitioners think formal usability techniques are to [6, 25, 17], Scrum practitioners do not have enough highly effective [18]. time to take usability activities into account. Even if usability is in the backlog, it is not on the top and is often 6. Limitations ignored as functionality is the top priority. Now it can The first limitation of this study is the sample size of the be asserted that it’s the product owner’s responsibility data. 11 interviews might be considered relatively small to enhance product quality by focusing on usability to generalize the findings to a larger population of aspects. It is indeed factual that the product owner is an Scrum practitioners or organizations. Also, the indirect leader and has some authority to make perspective of 11 individuals may not cover the diversity decisions. However, a product owner rarely wants to be within the Scrum community. Secondly, the data was the only person making decisions for the whole team. analyzed by one researcher and reviewed by two other This was identified as one of the challenges of a product researchers. Many aspects of the result might have been owner. A product owner is often guided by the influenced by the researcher’s assumptions, biases, and development team in what is technically right. Another perspectives. Lastly, project type can influence the need challenge is to act as a communication link between end to involve end user in the Scrum process. The findings users and developers. Because there are development of the study may be limited to projects that may not teams who do not know who their end users are [19]. It prioritize end user involvement such as infrastructure is challenging to prioritize usability when there’s no development or internal process improvement. clear understanding of the end users. The output from a 2–4-week sprint is minor. So, there’s no point in evaluating in a quantitative way. We 7. Conclusion found the same from our study by showing one of the This study investigates the role of end users and main challenges of involving end users in Scrum by usability in Scrum processes by conducting interviews professionals is its own limitations such as not having with Scrum practitioners. The study addresses our any proper feedback session with end users: “that could research question by identifying four novel Scrum anti be then a nice thing that maybe we could have some patterns. Anti-patterns identified in our research can lightweight user involvement methods that we could use provide insights to the Scrum practitioners in improving inside the scrum process”, short sprint cycles (usually 2-3 their Scrum process by 1) reflecting on their process and weeks): “It might take a bit more time, so it doesn't always becoming aware of any harmful practices; 2) having a fit into the basic cycle of two or three weeks”, focusing clear understanding of different stakeholders (internal, more on output not usability: “well, scrum process isn't external) of the product, what are their roles and value; really it…it doesn't really take usability aspects or UX into 3) reconsidering the role of a customer and end user in account much.” their Scrum process. Engage the right stakeholder at the right time; 4) seek ways to ensure end user’s feedback such as by introducing frequent product demonstrations with stakeholders. Sprint review is an event to engage with different stakeholders and gather feedback. Our recommendation is to avoid turning sprint review into a [3] Anwar, S., Motla, Y.H., Siddiq, Y., Asghar, S., technical event only for the internal stakeholders. The Hassan, M.S. and Khan, Z.I., 2014, December. outcome of a short sprint may be a minor update, in that ‘User-centered design practices in scrum case, practitioners should set a milestone where the development process: A distinctive advantage?’. In increment would be worth testing. Involve end users 17th IEEE International Multi Topic Conference 2014 from the beginning of the process when necessary to (pp. 161-166). IEEE. ensure objectives aligned with end user needs; 5) [4] Flouri, K. and Berger, H., 2010. ‘Agile focusing on usability regardless of whether the product Development–Scrum adopted in practice but not is for an internal stakeholder or external stakeholder; 6) in principle.’ using formal usability techniques (such as formal [5] Blomkvist, S., 2005. ‘Towards a model for bridging usability evaluation, prototyping, workshops, agile development and user-centered design’. In interviews, surveys) instead of getting feedback from Human-centered software engineering—integrating online conversations; 7) managing time effectively so usability in the software development lifecycle (pp. that usability activities cannot be ignored due to time 219-244). Dordrecht: springer Netherlands. limitation. The study contributes to the literature [6] Cajander, Å., Larusdottir, M. and Gulliksen, J. regarding usability in Scrum and extends the line of (2013) ‘Existing but not explicit - The user research by highlighting a significant gap: how end perspective in scrum projects in practice’, Lecture users are involved in the Scrum process, why end users Notes in Computer Science (including subseries are often neglected and what are the challenges of Lecture Notes in Artificial Intelligence and Lecture involving end users? Furthermore, we discussed certain Notes in Bioinformatics), 8119 LNCS(PART 3), pp. limitations experienced by Scrum practitioners and the 762–779. doi: 10.1007/978-3-642-40477-1_52. rationale behind specific practices. The outcome we [7] Curcio, K., Navarro, T., Malucelli, A. and Reinehr, derived from the interviews gives a good picture of why S., 2018. ‘Requirements engineering: A systematic end user involvement and usability are often ignored mapping study in agile software development.’ and difficult in many organizations. The field of research Journal of Systems and Software, 139, pp.32-50. is building a deeper understanding of the challenges and [8] Schwaber, K. and Sutherland, J. (2020) ‘Scrum as future research, specific strategies such as lightweight Guide V7’, Agile Metrics : Agile Health Metrics for methods for integrating usability aspects and end user Predictability, (November), pp. 133–152. evaluation within the Scrum framework, to mitigate the [9] Helmy, W., Kamel, A. and Hegazy, O. (2012) identified challenges are needed. ‘Requirements engineering methodology in agile environment’, International Journal of Computer Acknowledgements Science Issues, 9(5 5–3), pp. 293–300. [10] Inayat, I., Salim, S.S., Marczak, S., Daneva, M. and We thank all the interviewees who shared their Shamshirband, S., 2015. ‘A systematic literature knowledge and participated in this study. Further we review on agile requirements engineering thank all individuals from academia and industry who practices and challenges.’ Computers in human contributed. behavior, 51, pp.915-929 [11] Jia, Y., Larusdottir, M. K. and Cajander, Å. (2012) References ‘The usage of usability techniques in scrum projects’, Lecture Notes in Computer Science [1] Almeyda, S., Zapata Del Río, C. and Cohn, D. (including subseries Lecture Notes in Artificial (2021) ‘Integration of User Experience and Agile Intelligence and Lecture Notes in Bioinformatics), Techniques for Requirements Analysis: A 7623 LNCS, pp. 331–341. doi: 10.1007/978-3-642- Systematic Review’, Lecture Notes in Computer 34347-6_25. Science (including subseries Lecture Notes in [12] Lárusdóttir, M., Cajander, Å. and Gulliksen, J. Artificial Intelligence and Lecture Notes in (2014) ‘Informal feedback rather than performance Bioinformatics). Springer International measurements - User-centred evaluation in Scrum Publishing. doi: 10.1007/978-3-030-78221-4_13. projects’, Behaviour and Information Technology, [2] Almughram, O. and Alyahya, S. (2017) 33(11), pp. 1118–1135. doi: ‘Coordination support for integrating user 10.1080/0144929X.2013.857430. centered design in distributed agile projects’, [13] Larusdottir, M. K., Haraldsdottir, O. U. and Proceedings - 2017 15th IEEE/ACIS International Mikkelsen, B. A. (2009) ‘User involvement in Conference on Software Engineering Research, icelandic software industry’, CEUR Workshop Management and Applications, SERA 2017, pp. Proceedings, 490, pp. 0–1. 229–238. doi: 10.1109/SERA.2017.7965732. [14] Lundström, M., Åberg, J. and Blomkvist, J. (2015) Better Society, ICITEE 2018, pp. 581–586. doi: ‘Perceptions of software developers’ empathy 10.1109/ICITEED.2018.8534905. with designers’, ACM International Conference [23] Alhazmi, A. and Huang, S., 2020, May. ‘Integrating Proceeding Series, 2015-Janua, pp. 239–249. doi: design thinking into scrum framework in the 10.1145/2783446.2783563. context of requirements engineering [15] Rahayu, P., Sensuse, D.I., Fitriani, W.R., management.’ In Proceedings of the 3rd Nurrohmah, I., Mauliadi, R. and Rochman, H.N., International Conference on Computer Science 2016, October. ‘Applying usability testing to and Software Engineering (pp. 33-45). improving Scrum methodology in develop [24] Stettina, C.J. and Smit, M.N., 2016. ‘Team portfolio assistant information system.’ In 2016 scrum: an action research on multitasking in International Conference on Information multi-project scrum teams.’ In Agile Processes, in Technology Systems and Innovation (ICITSI) (pp. 1- Software Engineering, and Extreme Programming: 6). IEEE. 17th International Conference, XP 2016, [16] Scrum, A., 2013. What is Scrum?. We’Ve Been Edinburgh, UK, May 24-27, 2016, Proceedings 17 Covering Scrum in Such Detail It Makes You Sick. (pp. 79-91). Springer International Publishing. and Now [25] Law, E.L.C. and Lárusdóttir, M.K., 2015. ‘Whose [17] Singh, M. (2008) ‘U-SCRUM: An agile experience do we care about? Analysis of the methodology for promoting usability’, fitness of scrum and kanban to user experience.’ Proceedings - Agile 2008 Conference, pp. 555–560. International Journal of Human-Computer doi: 10.1109/Agile.2008.33. Interaction, 31(9), pp.584-602. [18] Ogunyemi, A.A., Lamas, D., Lárusdóttir, M.K. and [26] Bordin, S. and De Angeli, A., 2016. ‘Focal points for Loizides, F., 2019. ‘A systematic mapping study of a more user-centred agile development.’ In Agile HCI practice research.’ International Journal of Processes, in Software Engineering, and Extreme Human–Computer Interaction, 35(16), pp.1461- Programming: 17th International Conference, XP 1486. 2016, Edinburgh, UK, May 24-27, 2016, [19] Kristinsdottir, S., Larusdottir, M. and Cajander, Å., Proceedings 17 (pp. 3-15). Springer International 2016. ‘Responsibilities and challenges of product Publishing. owners at spotify-an exploratory case study.’ In [27] Hoda, R., Noble, J. and Marshall, S. (2011) ‘The Human-Centered and Error-Resilient Systems impact of inadequate customer collaboration on Development: IFIP WG 13.2/13.5 Joint Working self-organizing Agile teams’, Information and Conference, 6th International Conference on Software Technology, 53(5), pp. 521–534. doi: Human-Centered Software Engineering, HCSE 10.1016/j.infsof.2010.10.009. 2016, and 8th International Conference on Human [28] Eloranta, V. P., Koskimies, K. and Mikkonen, T. Error, Safety, and System Development, HESSD (2016) ‘Exploring ScrumBut - An empirical study 2016, Stockholm, Sweden, August 29-31, 2016, of Scrum anti-patterns’, Information and Software Proceedings 8 (pp. 3-16). Springer International Technology, 74, pp. 194–203. doi: Publishing. 10.1016/j.infsof.2015.12.003 [20] Schön, E.M., Winter, D., Escalona, M.J. and [29] Rafi, U., Mustafa, T., Iqbal, N. and Zafar, W.U.I., Thomaschewski, J., 2017. Key challenges in agile 2015. ‘US-Scrum: a methodology for developing requirements engineering. In Agile Processes in software with enhanced correctness, usability and Software Engineering and Extreme Programming: security.’ Int J Sci Eng Res, 6(9), p.377. 18th International Conference, XP 2017, Cologne, [30] Braun, V., & Clarke, V. (2012). Thematic analysis. Germany, May 22-26, 2017, Proceedings 18 (pp. 37- In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, 51). Springer International Publishing. D. Rindskopf, & K. J. Sher (Eds.), APA handbook of [21] Teka, D., Dittrich, Y. and Kifle, M. (2018) ‘Adapting research methods in psychology, Vol. 2. Research lightweight user-centered design with the scrum- designs: Quantitative, qualitative, neu- based development process’, Proceedings - ropsychological, and biological (pp. 57–71). International Conference on Software American Psychological Association. Engineering, pp. 35–42. doi: https://doi.org/10.1037/13620-004 10.1145/3195528.3195530. [31] Herdika, H.R. and Budiardjo, E.K., 2020, [22] Kikitamara, S. and Noviyanti, A. A. (2018) ‘A September. Variability and commonality conceptual model of user experience in scrum requirement specification on agile software practice’, Proceedings of 2018 10th International development: Scrum, xp, lean, and kanban. In 2020 Conference on Information Technology and 3rd International Conference on Computer and Electrical Engineering: Smart Technology for Informatics Engineering (IC2IE) (pp. 323-329). IEEE.