Technical debt is considered to have negative e ects to the long term success of software projects. However, how the debt metaphor applies to requirements engineering is yet not signi cantly explored. Previously, we proposed a framework to identify Requirements Debt (ReD) in three stages of the software development lifecycle. One of these stages is the formalization of stakeholder needs into natural language requirement speci cations. In this work, we propose a live study aiming at surveying requirements engineering experts to gain further insights on the issues taking place at this stage and how they t in our de nition of ReD.
We extended the de nition of requirements debt by Ernst to include upstream activities involving the elicitation of requirements (particularly in user-centered requirements engineering [MNJR15]) and their translation into speci cations [LF19]. Moreover, we outlined the future assessments to conceptualize and de ne the decision framework [LF19]. Within ReD, we proposed three de nitions of debt based on Incomplete Users' needs (ReD type 0), Requirement smells (ReD type 1), and Mismatch implementation (ReD type 2), de ning how to detect, how to quantify, and how to pay back each of ReD type. Our vision of ReD will be empirically evaluated in a series of studies with industry partners and individual stakeholders.
In this study at hands, we focus on ReD Type 1 to understand and compare the perceived harmfulness of requirement smells from a theoretical and practical perspective|i.e., an indicator for a quality violation of a requirements artifact [FWE17]. We focus on requirement speci cations written in natural language. Based on the results obtained in the study presented in here, we will design and conduct a large-scale surveys involving companies in order to monitor the requirement elicitation process. 2
Research Problem. We aim at understanding how the ReD Type 1 and the associated idea of code smells are perceived by requirements engineering experts.
By means of a cross-sectional questionnaire survey, we plan to elaborate on Type 1 ReD as there is little evidence of harmfulness of requirement smells when following the de nition and the detection approach proposed by Femmer et al. [FWE17]. We will follow the approaches widely adopted to assess the harmfulness of code smells on di erent software qualities [OCS10], [SYA+13], [HZBS14], [PBP+18].
In a second, later phase, will plan to triangulate the results of the survey with quantitative data from issue trackers and requirements repositories platforms [LT19] and qualitative data from case studies involving requirements engineers, business analysts, and software developers [TJL17].
Type of study. We design our live study as a cross-sectional survey based on Surveys.
Research goal. The goal of the survey is to understand and compare the theoretical and practical perceived harmfulness of requirement smells. To this end, we survey experts on their understanding and assessment of the relevance of the requirements smells for their context and their daily activities including di erent context factors. Moreover, the participants feedback will help us to ful ll the additional goal of laying ground for future follow-up studies.
Research questions. We formulated the following research question.
RQ1. How harmful are the di erent requirement smells perceived by practitioners after only reading their de nitions?
In order to answer to our RQ, we will provide the description and an example of each requirement smell and we will collect the perceived harmfulness of a requirement smell from the practitioners' point of view. In this RQ we do not evaluate whether practitioners know a speci c requirement smell but only whether they consider it as a threat in the requirement elicitation process.
Population of interest The target population of the survey includes roles interacting with a requirement speci cation artifact|in particular, for this study, a natural language formalization of a user need. Such roles include business analysts, product owners, team leads&developers. Moreover, we consider academic people in case they have conducted research and/or taught on requirement and/or technical debt elds. For this study, we are not interested in a sample of the above roles attached to any speci c domain.
Participants get deeper insights into the notion of requirements debt and smells in particular, thus, strengthening their learning curve.
Study Design. The survey will be based on a questionnaire organized into three sections. In order to provide the requirement smells' knowledge to the participants, we will include the list of smells in the questionnaire. Moreover, we will provide some examples of selected requirement smells. 1) Personal information. We aim to collect the pro le of the practitioners, considering age, country, gender, predominant roles, and working experience in requirement engineering. Moreover, we will collect the organization size via the number of employees and the common application domain. 2) Knowledge of Requirement Smells. We aim to understand whether the participants are familiar with Requirement Smells and whether they already consider the removal of Requirement Smells in their requirement elicitation process. This section of the questionnaire is useful for understanding whether the answers provided are based on personal experience and previous knowledge of Requirement Smells or only on the reading of the description we provided in the next question.
Requirements smells. We considered the Requirements Smells de ned by [FWE17], based on the ISO 29148 requirements engineering standard 1.
Requirement Smells Subjective Language Ambiguous Adverbs and Adjectives Loopholes Open-ended, non-veri able terms Superlatives Comparatives Negative Statements Vague Pronouns 3) Perceived Harmfulness of Requirement Smells. We aim to capture the general criticality perception of from our respondents. We will ask to rate how concerned they are about a smells in general and about the requirement smells reported in Table 1.
Study Methods and Procedures. We scheduled 30 minutes for the survey and other 30 minutes for introduction and questions. The surveys will be carried out by means of a questionnaire based on open-ended questions and ordinal 4-points Likert scale, where 1 corresponded to not concerned at all and 4 to very concerned. Moreover, at the end of the survey, we left space for further comments. We will provide the questionnaire also by online version.
Feedback session. The last step will be a semi-structured feedback session. First, we will ask the participants to re ect on the form and contents of each survey questions to get improvement ideas. On top of that, we will discuss with the participants further improvement for the survey (e.g., additional questions which would better help us measuring one of the relevant constructs).
Equipment and infrastructure needed for performing the live study We require a computer with access to the Internet in case the participants will not have access to their own laptops/wi . Moreover, we require a whiteboard and post-it notes for collecting and aggregating feedback.
Data Collection. We will conduct the survey by administering the questionnaire to the participants during the dedicated sessions.
Participants will have the chance to enter their data in an anonymous form or to report their email for the followup studies.
The questionnaire will be GDPR-compliant. We will add a section describing the goal of the questionnaire, how we collect the data, how we threat the data and giving the chance to the participants to withdraw their participation also after the data collection.
Data Analysis. We will partition our responses into more homogeneous sub-groups based on demographic information and compare the responses obtained from all the participants with the di erent subgroups. For questions measuring the association between categorical variables, we will use a Chi-square test. Ordinal data, such as Likert scales, will not be converted into numerical equivalents since using a conversion from ordinal to numerical data entails the risk that subsequent analysis will give misleading results if the equidistance between the values cannot be guaranteed. Moreover, analyzing each value of the scale allows us to better identify the possible distribution of the answers [MPM19]. Open questions will be analyzed via open and selective coding [Bee00]. We will extract codes from the answers provided by the participants and answers group them into di erent code smells. The authors will independently conduct the data analysis; the nal set of themes will be constructed iteratively based on the authors agreement. 3
Relevance of study for research and/or for practice. Requirement elicitation is one of the most important activities in the software development lifecycle. In case of issues during this phase, problems will be very expansive to be xed. Therefore, a clear and validated approach to reduce issues during requirement elicitation is to avoid to introduce requirement smells. Thanks to this study we will validate the di erence between theoretical and practical perceived harmfulness of requirement smells.
Bene ts to the subjects of participating in the study. Participants will have the chance to get in touch with requirement smells. We will provide an appendix to the questionnaire that practitioners will keep for them reporting a clear summary of the di erent smells, that can be used as guideline during their daily work. Plan to make publicity of the study if selected, and to attract respondents at the conference. We will advertise the survey by social networks, especially using the channels adopted by the conference. During the conference, the authors of this proposal will be present and will discuss with potential participants.
Sharing of preliminary and summary results with attendees during the conference. During the closing session, we will publish in social media the summary of the results of the closed-answer questions. Open-questions will require more time to be analyzed.
Dissemination of the results. Results will be submitted to international journals. Moreover, we will create a blog post for practitioners in order to ease the accessibility of the results itself. 4
We identi ed some threats to validity to our study. Since we designed the survey as a questionnaire, the participants cannot ask clari cation regarding the questions. We will ask experts in empirical software engineering to review the questionnaire to improve its comprehensibility.
The survey design, its execution, and the quantitative analysis will follow a strict protocol to ease its replication. The qualitative analysis of the open questions|which is more subjective to some extent|will be documented in both process (e.g., how con icts between annotators are resolved) and output (i.e., each code will be documented).
One limitation is that surveys can only reveal the perceptions of the respondents which might not fully represent reality. The responses will be analyzed and quality-checked by a team of four researchers.
Given the settings in which we will run the survey, we foresee two main threats due to sampling, self-selection |i.e., as the participation in the survey will be voluntary, the characteristic of the people who selected themselves to be part of the group can have an impact on the results|and sampling frame bias|i.e., the method used to select participants, which in our case is limited to the conference attendants. Nevertheless, the goal of the survey is not to achieve generalization but rather validating the constructs (e.g., the ReD stages) and obtain early feedback on the survey tool itself. 5
The study will be conducted by two senior researchers (Davide Fucci and Valentina Lenarduzzi) and one associate professor (Daniel Mendez) who have strong and solid experience in empirical studies.
Valentina Lenarduzzi is a Researcher in Software Engineering at LUT University. Her research is on Empirical Software Engineering with a particular focus on Technical Debt. She has experience in conducting empirical studies and especially these type of studies during practitioners and academic conferences ([TJL17], [TL18], [TLP17]). Further information on her activities, projects, and publications are available at http: //www.valentinalenarduzzi.it.
Davide Fucci is an Assistant Professor with the department of Software Engineering at the Blekinge Institute of Technology, Sweden. His research interests lies in Empirical Software Engineering applied to data-driven requirements engineering activities and the application of natural language processing techniques to software engineering problems. Further information are available at http://www.dfucci.co.
Daniel Mendez is an Associate Professor in Software Engineering at the Blekinge Institute of Technology, Sweden, and Senior Scientist at fortiss, the research institute of the Free State of Bavaria for software-intensive systems and services. His research is on Empirical Software Engineering with a particular focus on interdisciplinary, qualitative research in Requirements Engineering and its quality improvement. Further information on his activities, projects, and publications are available at http://www.mendezfe.org. [Bee00]
N. Beech. Basics of qualitative research: Techniques and procedures for developing grounded theory.
In 2nd edn. Management Learning, 2000. [BML+18] T. Besker, A. Martini, R. Edirisooriya Lokuge, K. Blincoe, and J. Bosch. Embracing technical debt, from a startup company perspective. In International Conference on Software Maintenance and Evolution (ICSME), pages 415{425, Sep. 2018. [FWE17] H. Femmer D. Mendez Fernandez, S. Wagner, and S. Eder. Rapid quality assurance with requirements smells. Journal of Systems and Software, 123:190{213, 2017. [HZBS14] T. Hall, M. Zhang, D. Bowes, and Y. Sun. Some code smells have a signi cant but small e ect on faults. ACM Trans. Softw. Eng. Methodol., 23(4):33:1{33:39, September 2014.
Z. Li, P. Avgeriou, and P. Liang. A systematic mapping study on technical debt and its management.
Journal of Systems and Software, 101:193 { 220, 2015. [LBT+19] V. Lenarduzzi, T. Besker, D. Taibi, A. Martini, and F. Arcelli Fontana. Technical debt prioritization:
State of the art. a systematic literature review, 2019. [MPM19] J. Seide Molleri, K. Petersen, and E. Mendes. An Empirically Evaluated Checklist for Surveys in
Software Engineering. Information and Software Technology, page 1 33, 2019.
M.S. Olbrich, D. Cruzes, and D. Sj berg. Are all code smells harmful? a study of god classes and brain classes in the evolution of three open source systems. In IEEE International Conference on Software Maintenance, ICSM, pages 1{10, 09 2010.
K. Schmid. On the limits of the technical debt metaphor some guidance on going beyond. In 4th International Workshop on Managing Technical Debt (MTD), pages 63{66, 2013. [TJL17] [TL18] [SYA+13] D. Sj berg, A. Yamashita, B. Anda, A. Mockus T., and Dyba. Quantifying the e ect of code smells on maintenance e ort. 39:1144{1156, 08 2013.
D. Taibi, A. Janes, and V. Lenarduzzi. How developers perceive smells in source code: A replicated study. Information and Software Technology, 92:223 { 235, 2017.