Agile methods are used more and more frequently to develop products to reduce development time. However, for developing desirable products with good user experience, Agile methods alone are not enough. This Ph.D. thesis explores how an iterative and incremental process model in the context of Agile methods must be designed in order to manage and improve product quality in terms of user experience. For this purpose, the design science research methodology is used. Using existing and new methods, an iterative and incremental process model is developed, which is validated in an international study. This Ph.D. thesis contributes to the knowledge base of software development by providing a process model that offers an iterative and incremental improvement of product quality in terms of user experience and process quality.
Software development companies use agile methodologies to develop products more efficiently. Agile methodologies (e.g. Scrum [24], Kanban [1], or Extreme Programming (XP) [2]) reduce the time taken to develop a product available to the market [25]. The iterative approach to developing software minimises the risk of developing software that is not in line with the market [3]. By performing retrospectives [24] at the end of an iteration, both the product quality and the agile process quality can be improved.
Nowadays, it is no longer enough to be faster on the market—the product is also expected to have a high level of user experience (UX). Users expect to be able to use the product to solve their tasks in a quick and efficient manner without having to put in any major effort. Furthermore, for a product to succeed, it is essential also to consider a good UX.
A well-known definition of UX is given in ISO 9241-210 [9]. Here UX is defined as “a person’s perceptions and responses that result from the use or anticipated use of a product, system or service”. Thus, UX is seen as a holistic concept that includes all types of emotional, cognitive or physical reactions concerning the concrete or even only the assumed usage of a product formed before, during and after use. A different interpretation is to define UX as a set of distinct quality criteria [19] that includes classical usability criteria, like efficiency, controllability or learnability, and non-goal directed or hedonic quality criteria [6], like stimulation, fun-of-use, novelty, emotions [17], or aesthetics [28]. This has the advantage that it splits the general notion of UX into a number of simple quality criteria (UX factors), which describe distinct and relatively well-defined aspects of UX that can be measured independently.
Agile methods themselves focus on developing useful products and not on developing products with a good UX. Development approaches involving Human-Centred Design (HCD) [9] are used to develop products with a good UX. These are, for example, Design Thinking [16], Design Sprint [13], etc. Additionally, processes have been developed to put HCD into practice [10]. But these HCD processes are isolated from the Agile methods themselves. Rather, it can be assumed that HCD and Agile methods are not integrated, as described in Section 2.
This article gives an overview of the research objective of my Ph.D. thesis and presents the actual progress. A first draft of an iterative and incremental process model for Agile methodologies is proposed, including new methods and artefacts. The paper is structured as follows: Section 2 briefly summarises the related work. Section 3 presents the research objectives and research questions of this Ph.D. thesis. Section 4 outlines the research methodology, covering a description of each step of the research process. Section 5 presents the first draft of an iterative and incremental process model for Agile methodologies. The paper ends with Section 6, with conclusions and ideas for future work. 2
In literature, many reviews have been conducted to research regarding the integration of Agile methods and User Centred Design (UCD) or Human-Centred Design (HCD). In the following paragraphs, the most important ones are briefly summarised in terms of their chronology.
Silva et al. [26] conducted an SLR on the integration of Agile methods and UCD in 2011. The authors analyse how usability problems are handled in agile projects and identify the following key aspects that play an essential role in integration: little upfront design, prototyping, user stories, user testing, inspection evaluation, and one sprint ahead.
In 2014, Salah et al. [21] analysed a similar area with their SLR, similar to Silva et al. The analysis of the authors identify the factors for the integration of Agile methods and UCD. Moreover, the authors examine the challenges and key aspects to ensure successful integration. The identified key aspects are lack of allocated time for upfront activities, difficulty of modularization, optimizing the work between developers and UCD practitioners, performing usability testing, and lack of documentation.
Brhel et al. [4] published in 2015 an SLR capturing the principles of user-centred agile software development (UCASD). The authors aimed to assess the current state of the art regarding the integration of agile methods and UCD. Using a coding system, the authors extracted five principles: separate product discovery and product creation, iterative and incremental design and development, parallel interwoven creation tracks, continuous stakeholder involvement, and artefact-mediated communication.
In a study published in 2018, Silva et al. [5] analyse the results reported by Brhel's SLR [4] concerning the current status of the integration of Agile methods and User Experience Design (UXD). The outcomes of the respective publications are divided into three dimensions: Process and Practice, People and Social, and Technology and Artefacts. The individual outcomes are arranged on a timeline so that the chronological sequence of the publications can be visualised. The authors state in their analysis that solutions concerning integration are already being offered for the dimensions Process and Practice and People and Social. Finally, the authors conclude that solutions for Technology and Artefacts are still needed in order to achieve integration between Agile methods and User Experience Design to Agile UXD.
The overall impression from the related work is that most of the problems regarding the integration of Agile methods and UCD have been addressed. SLRs have adequately described the problem area in recent years. However, not all issues regarding integration have been solved. In the Technology and Artefacts dimension, there are still large gaps to be closed. Technology means to support and coordinate activities within the development process. On the other hand, Artefacts from UCD and Agile methods are understood as a means to communicate information or knowledge in the development process. This Ph.D. thesis aims to contribute to the closure of these gaps. 3
The primary research objective of this Ph.D. thesis is to investigate how quality in terms of UX can be managed and improved in Agile methods. For this purpose, an iterative and incremental process model is to be developed that can significantly support the improvement of the UX. The process model is used in the context of Agile methodologies and should be integrated into existing agile process models. To this end, the following research questions (RQ) guide the research: • RQ1: What are the problems by using existing technologies and artefacts supporting the integration of UCD and Agile methods? • RQ2: What requirements must an iterative and incremental process model fulfil to support the improvement of the product quality and process quality in the context of the integration of UCD and Agile methods? • RQ3: How should an iterative and incremental process model be designed to meet the requirements of RQ2? • RQ4: How can the developed process model be applied in the real world? • RQ5: How can the developed process model be evaluated? The classification system given by Wieringa [31] can be used to classify the research questions. RQ1 and RQ5 can be classified as knowledge questions. These questions aim to identify the problems to integrate UX methods and techniques into Agile methods (RQ1), and to determine whether the proposed process model is adequate for solve these problems (RQ5). On the other hand, RQ2, RQ3, and RQ4 can be classified as practical problems. These research questions focus on developing and applying the process model itself. 4
This Ph.D. thesis follows the Design Science (DS) research methodology provided by Johannesson and Perjons [11] in combination with the guidelines by Wieringa [31] and Peffers et al. [18]. Fig. 1 gives an overview of the process model, which includes the different activities performed in this Ph.D. thesis.
In the next paragraphs, the current status of each activity from Fig. 1 is described in more detail. Additionally, an evaluation strategy is given for each resulting artefact from the research questions. The author has gained practical experience in the implementation of agile software projects with Scrum and Kanban in the past years. Thereby, the research and use of UX methods have always been of importance, especially the measurement of UX [7, 8, 23]. One research aspect is to identify techniques that could be used to determine the UX before actually developing the product increment. After some research, we come to the first conclusion that hardly any methods are used to determine the UX.
The UX can be evaluated after the product increment was developed. This is already a known and common practice. One requirement for software development, however, may be that the expected UX can be evaluated before development begins.
In agile software development methodology, all requirements from the Requirements Engineering (see Fig. 2, left side) are usually recorded in a product backlog. These are developed into a product increment by the development team after an evaluation (such as complexity and required time). The developed product increment is then used by the user (see Fig. 2, right side).
A product backlog item can or maybe should have an impact on the UX. After development, the UX can be measured using various methods [29]. Comparison with the previously assumed UX, however, is not possible, unless it has been systematically measured. Nowadays, the assumed influence on UX is not systematically measured. From our point of view, the resulting problem is as follows: There is no systematic verification between the assumed and the actual UX.
We are convinced that the problem and outcome of the Ph.D. thesis are essential and relevant for many companies worldwide. One reason for this is that processes, in general, can only be improved if they have been systematically recorded and evaluated beforehand. As stakeholders, we address all roles that are responsible for the product. These are, for example, the product owner [24] or the product manager. UX professionals such as UX researchers or UX designers are also addressed as stakeholders.
The Ph.D. thesis will be based on an SLR. The SLR aims to provide the current status regarding the integration of UX methods and techniques into agile software development. As a result, potential problems by using existing technologies and artefacts supporting the integration of UCD and agile methods will be addressed. The SLR answers the following RQs: • RQ1.1: Which technologies and artefacts are used or supported in the integration of UCD and Agile methods? • RQ1.2: What are potential problems by using founded technologies and artefacts found in RQ1.1? • RQ1.3: How can the user experience in agile software development methodologies be estimated before development? • RQ1.4: What proposals exist to determine the user experience for a product backlog item or a requirement? • RQ1.5: What proposals exist to improve the user experience in the agile software development process? Evaluation strategy: The SLR itself is the evaluation strategy to figure out potential problems. Furthermore, the guidelines given by Kitchenham and Charters [12] are adopted.
The iterative and incremental process model proposed in the Ph.D. thesis is to be applied to existing agile software development methodologies such as Scrum [24], Kanban [1] or Extreme Programming [2], and scaled Agile methods as well as LeSS [15] or SAFe [14]. The process model can be understood as a product. Each product, including the process model, should fulfill certain requirements for its users [30, 31], represented by the stakeholders (see Section 4.1). The following proposed requirements of the process model are to be evaluated as suggestions. • The process model should consider the existing agile methodologies or be integrated into the agile methodologies. • No additional roles should be created. • Existing artefacts such as product backlog or user stories should be used. However, the basic structure of the artefacts should be preserved.
In addition to these requirements, certain structural qualities should also be considered: • Modularity: The process model should be modular—i.e. either entirely or partly applicable. • Learnability: The process model should make it possible to improve the process continuously. • Adaptability: The process model should be easy to use and to integrate into existing agile software development projects.
Evaluation strategy: The requirements presented in this section still need to be evaluated with the stakeholders addressed in Section 4.1. Also, the list of requirements should be checked for completeness. Expert reviews should evaluate the requirements. If the results of the expert reviews are not clear, interviews or a Case study should be conducted. 4.3
This phase mainly comprises of the designing and development of the artefact. The first task, which is completed, is done by creating the first draft of an iterative and incremental process model for agile development (see Section 5), which integrates UX in the agile development process. The second task, which is still ongoing, covers the iterative evaluation of the process model through expert reviews and stakeholders (see Section 4.1). We can learn how the process model can be tailored for different agile approaches (e.g. Scrum [24] or Kanban [1]) with focus on UX. The following task will be to develop the new methods of the process model. Existing methods must be named and integrated into the process model.
Evaluation strategy: The process model, including new and existing used methods, will be validated as a whole (see Section 4.5). The newly developed methods will be validated in a multi-level Delphi study.
The iterative and incremental process model is demonstrated by providing an application in a real-world context. The requirements, as defined in Section 4.2, can thus be checked in an initial validation. Any corrections can be made. The results can also be used to prepare the evaluation or validation described in Section 4.5.
Evaluation strategy: For this phase, quantitative evaluation methods such as laddering [20] are offered. Moreover, a combination of interviews and a questionnaire are also preferred. 4.5
The evaluation of the artefacts from RQ1 to RQ4 was partially carried out individually and in isolation. The methods developed in the Ph.D. thesis and integrated into the process model are individually assessed. It is planned to use questionnaires, Delphi studies, and case studies.
Evaluation strategy: The process model is validated through an international case study. With this strategy, each method developed in this Ph.D. thesis and the process model itself with the individual methods are validated as a whole. 5
Based on the problem that there is no systematic verification between the assumed and the actual UX, (see Section 4.1), the first draft of a solution is developed. This draft represents a process model [27] for agile software development methodologies. This process model is based on the requirement given in Section 4.2. To be sure, the requirements have to be checked to see whether they are desired (see Section 4.2).
The process model consists of five well-known activities (see Fig. 3, no. 1 and 5– 8) and four new activities (see Fig. 3, no. 2–4 and 9). In addition to the name of the activity, the resulting artefact and the method for validation are mentioned. The process model presented in this section integrates the plan-do-check-act (PDCA) principle in agile software development methodologies around the UX aspect. The process model aims to develop improvements through a continuous review of the achieved UX. The following list describes the individual activities from Fig. 3. 1. Requirements engineering: This continuous activity is well-known in software development. Several models and methods are used to apply requirement engineering in agile software development methodologies [22]. As a result of this activity, a list of requirements is created. In literature research, methods are collected that manage requirements.
which UX factors or quality criteria are to be measured in the further course of development. The method to be developed aims to select from a pool of UX factors only those UX factors that are relevant for the product and the user. 3. Define a sprint UX goal: To measure whether a UX goal has been achieved, that goal must be established. In this activity, a UX goal is defined for a period of time. This can include one or more sprints, for example. The purpose is divided into sub goals, which in turn refer to the list of UX factors from Activity 2. 4. Determine a target UX value for each requirement: Each requirement is evaluated against the list of UX factors from Activity 2, as it supports the goal. Consequently, it can be decided which requirements should be processed in the next iteration. 5. Develop a Prototype: In this activity, the prototype (artefact) is created by the requirement selected in Activity 2. Various methods, such as Design Thinking [16] or Design Sprint [13], can be used. As a result, a low or high fidelity prototype is developed. 6. Evaluating prototype: The UX is evaluated based on the UX factors selected in this activity. The result is then compared with the goal defined in Activity 3. From this, conclusions can be drawn for the subsequent development. 7. Develop the product increment: In this activity, the product increment is developed. The results from Activities 3 and 6 are considered. 8. Evaluating product increment: After the product increment has been developed, the UX is re-evaluated based on the selected UX factors. 9. Conducting a UX retrospective: Finally, this activity checks whether the UX goal, as defined in Activity 3, has been achieved. The results of the different evaluations of the UX, based on the selected UX factors, serve as a basis. A comparison is made between the defined UX goal (Activity 3), the evaluated UX of the prototype (Activity 6), and the product increment (Activity 8). This serves as a basis for possible improvements, which can then be applied in the next iteration by, which begins again with Activity 3. If necessary, the next iteration begins with Activity 2 by recreating the list of UX factors.
This process model is to be evaluated as a draft. Further iterations and evaluations are needed to refine the process model. Besides, the process model must be checked to see whether integration within agile software development methodologies is feasible. Also, it should be examined whether a software tool should be developed to support the process model. 6
This paper presents the proposal and the current state of my Ph.D. thesis. The primary objective of the Ph.D. thesis is the development of an iterative and incremental process model. With this process model, development teams should be able to improve product quality concerning UX.
For this purpose, five research questions are asked. RQ1 is answered through an SLR. On this basis, the existing requirements for the process model are further refined and validated (RQ2). Afterward, the process model is developed under consideration of the requirements (RQ3). In a real-world application, the process model must prove itself (RQ4). Feedback then flows into the further development or correction of the process model. In the final step, the process model is evaluated (RQ5).
This Ph.D. thesis contributes to the knowledge base of software development by providing a process model that offers an iterative improvement of product quality in terms of UX and process quality. [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28]