User story ambiguity has been identified as a serious challenge in Requirements Engineering (RE) due to various linguistic and contextual constraints. These challenges have been addressed by research employing mainly Natural Language Programming (NLP) techniques and Conceptual Modeling. However, existing studies have only limitedly investigated the impact of personal and organizational elements on user story ambiguity and user awareness of ambiguity issues. Therefore, this research aims to mitigate user story ambiguity by developing a knowledge repository for user story writing from different domains. To do so, we aim to investigate the role of cognitive elements to enhance user story clarity. Technically, the use of context-free grammar will be investigated to maintain the quality of the knowledge repository. Finally, for raising awareness of ambiguity issues, gamification concepts (e.g., levels/tags, points) will be applied to monitor user progress.
As a widely known RE artifact, the user story is perceived as a compelling yet challenging artifact in Agile Software Development (ASD). Despite their effectiveness as requirements documentation instruments, user story creation and/or review have also been recognized as challenging processes due to implicit ambiguity. This ambiguity emerges as a result of differences in linguistics expressions and context understanding.
Regarding linguistic issues, user story expression is prone to misunderstanding because one word might have several related meanings regarding specific contexts. For example, the term “key account” might be perceived by marketing as valuable customers, while procurement comprehends the term as an important supplier. Distributed cognition (DCog) theory explained that these various understandings emerge due to information processing in people’s minds and associated environment where the knowledge have been acquired and distributed. The personal and organizational aspects that influence knowledge understanding consists of knowledge, skills, and experiences, which is referred as sociocultural cognition [1]. Furthermore, Harris [2] described that the DCog may unconsciously prompts different perceptions in a particular field (i.e., business, IT) through human and machine interactions. This phenomenon has been shown by Wautelet et al. [3] and Jia et al. [4], who demonstrated a significant involvement of socio-cultural cognition (i.e., knowledge background, experience) in creating a shared understanding of requirements between user and developer communities.
Besides apparently emanating in requirements elicitation and documentation activities, the literature study we performed indicates that ambiguity also affects requirements analysis and negotiation activities. Regarding requirements elicitation and documentation, ambiguity issues have typically been resolved by employing linguistic techniques. The solutions were generally concerned with NLP algorithms detecting user story similarity [5, 6] and inconsistency [7, 8]. Studies focusing on requirements analysis and negotiation have approached ambiguity in terms of a lack of shared understanding of requirements between user and developer communities. Proposed solutions have generally mapped user stories to (graphical) conceptual models [9, 10]. However, our literature study revealed that the number of studies addressing ambiguity issues related to requirements documentation with user stories is relatively small. Furthermore, this research topic is also immature, as almost half of the proposed solutions have not been empirically verified.
Apart from these gaps, the studies emphasizing linguistic solutions have barely considered human factors (e.g., knowledge, skills, experience) in the search for a solution to overcome user story ambiguity. Therefore, we aim to extend the state of the art by exploring personal and organizational factors that lead to user story ambiguity during requirements elicitation and documentation. Our intention is to investigate how a focus on these factors might help creating solutions for mitigating user story ambiguity and raising awareness of user story ambiguity. To do so, we propose the development of a knowledge repository of user stories, based on context-free grammar for duplication and similarity detection, augmented with a gamification concept (e.g., levels/tags, points) for promoting individual cognitive skills and organizational culture to enhance a shared understanding between user and developer communities. 1.1.
Requirements ambiguity has been studied from linguistics [11] and contextual [12] perspectives. Compared to the contextual approach, the linguistics approach has drawn smaller attention from researchers. Given that situation, our study aims to develop a comprehensive solution to mitigate user story ambiguity by elaborating socio-cultural cognition that have generally been neglected in these studies. Our solution will take advantage of the DCog theory to investigate which socio-cultural cognition aspects influencing user story writing. We will also exploit NLP techniques to elaborate these finding by constructing a knowledge repository comprised of word glossaries and ontology as a standard terminology for avoiding semantic ambiguity. Finally, a gamification concept will be introduced in the shape of points and levels to observe the adequacy of the solution to improve user awareness for mitigating ambiguity while writing user stories.
Our research plan comprises of four sections: research problems and questions, research methodology, proposed solution, and novelty of the solution 2.1.
In order to identify the main problems in user story ambiguity, we conducted a systematic literature review in 2020 (in progress). From the literature study, we extracted 38 documents addressing user story ambiguity for the last 20 years. The study then identified four shortcomings or research gaps: 1. Numerous studies highlighting user story ambiguity have promoted the use of conceptual models, while a smaller number of the studies have proposed a linguistics-based approach. Especially, the latter approach is interesting for avoiding, rather than detecting and remedying, ambiguity in user stories. 2. Only a few studies have addressed socio-cultural cognition (e.g., knowledge background, experience) as a factor impacting the shared understanding of user stories. 3. The existing studies have mainly focused on providing solutions for user story ambiguity for one particular business or societal sector (e.g., e-commerce, university) 4. The existing solutions were limitedly verified and/or applied in a real-world setting. Given this analysis of state of the art, our research questions have been formulated as follows: RQ1: To what extent have socio-cultural cognition been investigated in the search for solutions for mitigating user story ambiguity?
RQ2: How do socio-cultural cognition improve/impede a shared understanding of user stories? RQ3: How can the sharing of contextualized knowledge of user stories help in writing unambiguous user stories and improve the awareness of potential ambiguity issues?
Hence, our study strives to leverage mechanisms related to distributed cognition (i.e., the contextualized knowledge) to reduce ambiguity and misunderstanding of user stories. In order to do so, socio-cultural cognition aspects related to individual and organizational elements will be explored to improve user story clarity and promote a shared understanding of requirements contexts. Semantic ambiguity will be detected and reduced with a knowledge repository based on a glossary, ontology, and context-free grammar. The result will be evaluated through a case study involving a medium-sized enterprise with Scrum practitioners. 2.2.
Our research methodology will be implemented in sequential steps as follows. The steps consist of a literature review, data collection and exploratory study, solution development, and validation.
Literature review
Ambiguity challenges in the use of the User Story technique
2 Investigation of the
socio-cultural cognition in user story ambiguity
(RQ1)
Socio-cultural cognition in user story ambiguity
3
Development of a knowledge repository
comprising sociocultural cognition for user story writing (RQ2) Knowledge repository
comprising sociocultural cognition for user story writing
4
Validation of the knowledge repository solution to improve user awareness of potential ambiguity issues (RQ3)
Empirical validation using case study research adoption has encountered many challenges due to requirements complexity, features misfit, and others [14, 15].
Our research study aims to apprehend individual and organizational determinants to a shared understanding of user story context. Hence, the data collection will be focused on three categories of items: (1) business or societal sector, (2) user profiles and organizational culture, (3) participant qualification. For the business or societal sector, the information will be collected from manufacturing, information technology, and financial services. The information should be presented in user stories or text-based requirements, which has similar components to user stories: as a [role], I want [goal] so that [reason].
Regarding determinants, we intend to conduct an exploratory study to verify which factors of user profile and organizational culture have impacted user story ambiguity related to a shared understanding. As a starting point, we refer to the study of Wautelet et al. [3] and Jia et al. [4], who have confirmed knowledge background as an important factor to a shared understanding. Subsequently, we attempt to investigate working experience (i.e., entry-level, intermediate, mid-level, senior) and managerial responsibility level (i.e., operational, middle manager, senior manager, executive) as we posit that those factors might significantly influence a common concept of user story context. In respect of organizational culture, we determine to explore the organizational structure, communication culture, and employee turnover because these factors have been committed as a crucial elements to ASD adoption [16] (see Table 1). Information on user-profiles and organizational culture will be carefully analyzed to understand the impact of those elements on word selection and grammatical expression in user story writing and/or review.
Finally, we select Scrum practitioners as participants considering that the method is the most widely practiced in ASD [17]. Additionally, we require five years of experience and a geographically distributed team.
Table 1.
Personal and Organization determinants
Qualifications type
Personal
Step 3: Development of a knowledge repository comprising socio-cultural cognition aspects for user story writing (RQ2)
Next, the requirements from different domains will be structured to ontology, while terminology will be arranged in a glossary. For a terminology having different interpretations across domains (i.e., homonyms), the meanings will be classified accordingly to support different interpretations for the various objects/actions. Furthermore, synonyms will be recorded to facilitate alternative phrases to refer to the same object/action (i.e., polysemy). Context-free grammar will be adopted as a classifier to verify sentence similarity, predict subsequent phrases, and provide alternatives for user story sentences.
With respect to socio-cultural cognition factors, suppose there is evidence of personal and organizational determinants to word choice, the element will be carefully considered a branch of the ontology. Next, corresponding words will be examined to distill the glossary. The recommendation for
Organizational
Educational background Working experience Responsibility Organization structure Communication culture Employee turnover
Pair programming user story writing will be provided according to a user profile and project domain that have been recorded into our knowledge repository. User awareness of user story ambiguity will be assessed by comparing the sentence grammar with the standard terminology, and the result will then be converted into points. User improvement of producing unambiguous user stories will be evaluated based on standard vocabulary from the knowledge repository. The shared understanding of user story context will be validated by specifying user stories into product backlogs.
Step 4: Validation of the knowledge repository solution to improve user awareness of potential ambiguity issues (RQ3)
The effectiveness of our solution will be evaluated using case study research. The study will observe the solution performance to improve a shared understanding of user story context and raise user awareness of user story ambiguity. We determine accuracy, efficiency, and consistency as key variables to measure the adequacy of our solution in practice. The validation will be performed by measuring user acceptance and duration to formulate unambiguous user stories with the help of our knowledge repository.
The accuracy of the repository will be evaluated by inviting Agile practitioners to write/review user stories from the given scenario using our knowledge repository. Then, they will be requested to specify those stories into product backlogs. The differences between the backlogs and the expected features will be compared to predict accuracy.
The evaluation of solution efficiency will be performed by measuring time spent by the participants to fulfill the expected requirements of the user stories. If the time to write/review user stories supporting the knowledge repository is less than manual writing/review, our solution will be considered sufficient.
The solution consistency will be verified through the prevalence of the sentences to precisely interpret user needs in the form of the product backlog. To do so, user performance during the reference periods (i.e., points and levels) will be measured. 2.3.
Our solution will be proposed by incorporating socio-cultural cognition as a result of personal behavior (related to user profiles) and the organizational cultures of their working environment as components of an integrated solution to mitigate user story ambiguity. These factors have been carefully selected based on empirical studies confirming the positive influence of those factors to improve a shared understanding of user stories [4], [6]. The factors will be verified through an exploratory study to finalize what cognitive factors have influenced word selection in writing and/or reviewing user stories. The factor will then be inserted as an ontology branch, while the corresponding word will be installed in the parallel glossary.
User story duplication and similarity will be avoided by adopting context-free grammar in the knowledge repository. Finally, user activity will be awarded into a point, and the awareness of user story ambiguity will be monitored through level every time they work using our repository. 2.4.
Our research focuses on mitigating user story ambiguity by means of a knowledge repository, which is developed by adopting DCog theory as a theoretical framework to establish socio-cultural cognition aspects in user story writing. Owing to socio-cultural cognition, we determine user profile and organizational culture as the solution attributes because these factors have been limitedly studied in user story research.
The knowledge repository will be developed by reusing user stories on the basis of ERP systems from different business domains. The standard terminology will be extracted from the business domain by utilizing context-free grammar. Then, the terminology will be classified using an ontology. Furthermore, the gamification concept (i.e., points, levels) will also be adopted by virtue of reward and level to raise user awareness of user story ambiguity.
In order to provide a solution to mitigate user story ambiguity, our study follows the design science research method by developing a knowledge repository. In order to do so, we will contact software companies to participate in our research. We intentionally set some minimum requirements for our participants (see Table 2.) as we expect diverse working experiences and practices. Then, an online survey will be conducted to collect user profiles and organizational culture data from our participants (see Table 1). User profiles will be collected by applying a purposive sample technique, targeting Product Owners, Business Analysts, System Analysts, and Developers. This information will be useful to initiate a corresponding glossary with regard to word selection and grammatical expression.
Organizational culture data will be compiled from an in-depth interview that will be conducted with Scrum Masters and Managers (e.g., software team managers, product line managers, COO, CIO) as such profiles might best comprehended organizational culture. Additionally, requirements documentation (i.e., user stories) will be solicited from participants as a base for our glossary and ontology. Therefore, the participants are required to supply documentation of some ERP projects from various business sectors. Afterward, the knowledge repository will be developed by reusing user stories, user profiles, and organizational culture parallel with the business domain. Context-free grammar will be used to avoid user story similarity and duplication by detecting similar words or meanings of a new user story. The new user story could be added as new knowledge by the Scrum team after getting authorization from the Scrum Master. Eventually, a case study will be conducted by observing the knowledge repository performance in terms of accuracy, efficiency, and consistency. The accuracy will be assessed by comparing the recommended user stories with expected requirements from users, while efficiency will be measured by calculating time to write and/or review user stories with the repository. The consistency will be evaluated by observing user experience working with our knowledge repository. Furthermore, the improvement of user awareness to user story ambiguity will be monitored through a gamification concept (i.e., levels/tags, points) installed as a feature of our knowledge repository
Concerning the research plan that has been presented in Section 2, the first stage, a systematic literature review, has nearly been completed (see Figure 1). This proposal is structured based on research gaps that have been found during our literature study.
From selected papers addressing user story ambiguity, our study indicated that the ambiguity emerged due to natural language and contextual understanding of the user story. Despite less mature, existing solutions focusing on ambiguity have mostly neglected socio-cultural cognition aspects as crucial elements influencing ambiguity. Our study discloses that linguistics-related solutions have typically been proposed to overcome ambiguity during requirements elicitation and documentation, while the (graphical) conceptual model has generally promoted requirements analysis and negotiation to enhance a shared understanding between user and developer communities.
At this stage, we are in the second stage of our research study. Therefore, we attempt to contact software companies in Belgium to join our research study. Scrum practitioners will be called participants, considering that the Scrum method has been widely adopted. The participants are required to have a minimum of five years of working experience, besides working in distributed teams. On top of that, we expect to conduct an exploratory study to collect user profiles and organizational culture data as a basis of our proposed solution. We also aim to collect user stories or other similar user requirements representation from ERP projects to construct glossary and ontology components for our knowledge repository.