A Closer Look on the Difficulties to Determine the Quality of Software Requirements Patrick Kummler Hansjörg Fromm Karlsruhe Service Research Institute Karlsruhe Service Research Institute Karlsruhe Institute of Technology Karlsruhe Institute of Technology Karlsruhe, Germany Karlsruhe, Germany patrick.kummler@kit.edu hansjoerg.fromm@kit.edu Abstract—Increasing demands on quality and complexity experts would have done. The performance of such an are a major challenge for the development of industrial algorithm should always be evaluated against a benchmark. software products. Automotive software in particular is subject Since quality is not objectively measurable, this benchmark to additional safety, security and legal demands. In such can only be provided by human judgement. Thus, our software projects, the specification of requirements is the first research question is how reliable and consistent human concrete output that is mostly written in natural language. experts can rate the quality of software requirements. To However, in practice, two problem areas exist: First, due to answer this research question, we draw upon the groundwork reasons like lack of knowledge and missing experience of on requirements quality that has resulted from different engineers, requirements quality often is not at a satisfactory standardization efforts within the software engineering level. Second, a massive increase of the number of requirements for software poses a scalability issue. In our community. The standard ISO/IEC/IEEE 29148:2011 [11] research, we want to take a closer look on the quality provides us with a set of characteristics that are believed to determination of software requirements. We present an determine the quality of a single requirement. The main overview of existing research approaches based on the objective of our research is to take a closer look on these standard ISO/IEC/IEEE 29148:2011 that offers nine essential characteristics, especially asking the question how well characteristics for requirements quality. In addition, we human experts can rate them. analyze results from several sessions in which experts rate This paper contains an overview of the characteristics automotive software requirements. from [11] and presents relevant and current research about Keywords—Requirements Quality, Natural Language the quality measurement of textual requirements. In addition, Requirements, Quality Characteristics, Requirements Rating, a method for requirements rating is described and results Software Requirements. from expert sessions are presented. The research is done in cooperation with an international automotive engineering and consulting company and enables us to have access on I. INTRODUCTION software requirements from the automotive industry. The In the automotive industry, innovative services and rating sessions are conducted with experts from industrial software functions decide the success of today’s vehicles. practice that are currently working in automotive software Automotive manufacturers invest a major part of their development projects. The use of real requirements and a resources in the development of customer functions and rating through experts from the automotive industry ensures valuable services. This trend leads to a growing number of reliable results for our research. software requirements in development projects. With increasing complexity and strict safety, security and legal In the following chapter, we present and explain demands the manufacturers are faced with various challenges characteristics for requirements according to [11] and especially in the specification process [1-4]. identify initial issues. Chapter three contains an overview of related work and reveals the research gap. In chapter four, In general, the success of a software project strongly the preparation, the execution and the results from the rating depends on the quality of specified requirements [5], [6] and sessions are presented. The last chapter contains existing “requires fluid collaboration and communication between limitations. clients and software engineers” [7, p. 2]. Several established approaches, such as formal specification methods to describe II. REQUIREMENTS QUALITY AND CHARACTERISTICS and specify software requirements, already exist. But, “despite the significant advantages attributed to the use of Initially, we introduce several definitions of a formal specification languages, their use has not become requirement. Reference [12] defines a requirement as common practice” [8, p. 1]. “Natural language is key in “something required, something essential to the existence or requirements engineering” [9, p. 1]; specifications are still occurrence of something else”. The unspecificity in this organized in text documents and used as basis for the definition reveals the challenges when defining the term communication between relevant stakeholders [8, 10]. requirement. According to this, a requirement needs to be required and necessary. A more specific definition is given In our research, we explore and analyze the quality by [13] where a requirement is defined as: (1) a condition or determination of textual requirements based on capability needed by a user to solve a problem or achieve an characteristics, attributes and desirable properties of a objective; (2) a condition or capability that must be met or requirement. Different approaches largely rely on methods of possessed by a system or system component to satisfy a natural language processing that are complemented with contract, standard, specification, or other formally imposed machine learning techniques. A machine learning algorithm document; (3) a documented representation of a condition or is designed to automate the task that previously human capability of the previous two arguments. ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 53 In [14], a requirement describes the quality that a  Verifiable. The requirement necessitates the verification software system must possess, as well as the prevailing of the statement by using the standard methods conditions that are in force for its life cycle. Therefore, a inspection, analysis, demonstration or test [17]. requirement is responsible for general quality aspects of the implemented software. The authors in [15] describe a The fulfilment of the characteristics complete, singular requirement as “something the product must do to support its and clear and concise are mentioned as main challenge owner’s business, or a quality it must have to make it during the specification [18-20]. The characteristic clear and acceptable and attractive to the owner” [15, p. 9]. A broader concise is also described as the Achilles heel [20] of software definition proposed by [16] defines a requirement as “a requirements specifications. Considering [21] and [22] clear specification of what should be implemented. They are and concise is mainly responsible to enable the descriptions of how the system should behave, or of a system determination of the remaining characteristics. If a property or attribute” [16, p. 6]. This definition considers requirement is not fulfilling this characteristic, other different types of information and input that can be defined characteristics can hardly be determined. Authors describe as a requirement. In most definitions a requirement serves as this phenomenon as “surface understanding” and “concept basis for the implementation of a target. For our research, we understanding” [21], or “clarity” and “content” [22]. Surface use the second expression of [13] as it represents the and clarity consider “what is stated” [21]. Concept or content challenges of software requirements and the belonging focus on the question “what is meant or implied” [21]. quality determination. According to [11] and [17], the characteristics are applied The quality of software requirements can be defined by to individual requirements. However, with the given different criteria. References [11] and [17] provide a set of definition for traceable, a distinction between characteristics characteristics to define a well-written requirement as that apply to individual requirements and characteristics that something that fulfils different criteria. Whereas in [11] strongly depend on the existence of further relevant unambiguous1 is defined as characteristic, [17] offers a more requirements is necessary. Characteristics of the first group holistic approach by using the characteristic clear and can be applied to individual requirements without further concise instead that also includes the understandability and information needed. We identify clear and concise, feasible, the preciseness of a requirement. Thus, we use the implementation independent, singular and verifiable in this characteristics from [11] and the modification of [17] for the group. The characteristics necessary and traceable relate to quality determination of textual requirements. In the the second group. The application of these characteristics following the description of each characteristic is presented. necessitates additional relevant requirements. For traceable, information about linked requirements would be most  Clear and concise. The requirement ensures that helpful. Same applies for necessary where the necessity of a statements can only be interpreted unambiguous. “[T]he requirement can be detected when the whole requirements terms and syntax used must be simple, clear and exact” specification is available. [17, p. 10]. For a clear and concise requirement, the use of weak terms, synonyms and unclear sentence structure The characteristics complete and consistent have a lead to misunderstandings. special role and relates to both groups. Complete, as example, can be applied to individual requirements and  Complete. The requirement describes adequately “the considers, whether the requirement “needs no further capability and characteristics to meet the stakeholder’s amplification” [11, p. 11]. Also, complete can be applied to a needs” [11, p. 11]. Further explanation and enhancement set of requirements where “it contains everything pertinent to of the requirement is not necessary. the definition of the system or system element being  Consistent. The requirement has no conflicts. Defined specified” [11, p. 11]. Same applies for the characteristic terms are used consistently throughout the requirement. consistent. The categorization is similar to the distinction of  Feasible. The requirement can be implemented requirement characteristics from the ISO standard technically and does not need further advanced 26262:2011 [23]. technologies. The system constraints are considered regarding legal, cost and schedule aspects. These are differences between characteristics that can be  Implementation independent. The requirement is found by a purely theoretical consideration. For our research, specified independently from the implementation: “The we are interested in the detection of differences between requirement states what is required, not how the characteristics as experts see them. We want to reveal the requirement should be met” [11, p. 11]. possibility to rate a characteristic by humans and the  Necessary. The requirement contains relevant influence on an overall quality of a requirement that is information and is not deprecated. perceived by experts.  Singular. A requirement cannot be divided in further requirements. It includes one single statement. III. RELATED WORK AND RESEARCH GAP  Traceable. “The requirement is upwards [, downwards Many efforts have been targeted towards modeling, and horizontally] traceable” [11, p. 11]. Every rating and measuring the quality of textual requirements. In requirement at each development stage can be traced to a Table I, we present related work that is divided in three requirement either to the current or to the previous and research categories: subsequent development stage. The requirement considers the dependency and possible conflicts among  Assistance (A). Tools that support requirements software. engineers in the specification of requirements. These tools mostly follow a defined and static set of metrics for the quality determination. 1 For better identification quality characteristics are in italic. ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 54  Transformation (T). Approaches that transform textual independent and necessary are not investigated at all. Similar requirements into formal and logic specifications. characteristics – like correctness, modifiability, validability,  Classification (C). Methods that enable the testability [8, 38], understandability [25, 28] and abstraction classification into good and bad requirements by using [7] – are used as well to determine the quality of e.g. machine learning techniques. requirements. The characteristics traceable and verifiable are focused mostly in research that deals with assisting tools. We allocate related work to the research categories and analyze the occurrence of characteristics from the modified Table I reveals the gap regarding the quality ISO standard [11, 17] (Identical or Similar). An identical determination of textual requirements. Whereas the characteristic in a paper is used with a corresponding characteristics clear and concise, complete and consistent are definition as in [11] and [17]. Similar indicates a related use adequately investigated in the research, the determination for (e.g. realizability is similar to feasible). The presented feasible, implementation independent and necessary is barely overview in Table I does not claim to be conclusive. It available. contains an extract of relevant approaches, methods and tools For our research, we do not exclude characteristics for from the literature and reveals the research gap. the determination of requirements text quality, although the Most of the latest papers can be assigned to the categories evaluation of some characteristics for a single requirement Transformation (T) and Classification (C). Especially the use does not seem to be conducive at first glance. Despite the of machine learning techniques to analyze and evaluate the discussion in the previous chapter about the distinction of quality of textual requirements is a common topic. These characteristics that apply to individual requirements and papers build upon the research of assisting tools (A) and characteristics that apply to a set of requirements, we are enhance these approaches with methods from different convinced that the characteristics traceable and necessary disciplines, such as natural language processing and are also relevant for the quality determination of an analytical methods. There is a trend towards developing individual requirement. Although further relevant algorithms and models that automatically evaluate the quality requirements would be helpful at this point, an individual of textual requirements by using attributes and indicators to requirement could also offer indications whether a classify good or bad requirements. Most of the related work requirement is traceable or necessary. focus on the characteristics clear and concise, complete and The research papers presented in Table I consist of a consistent. variety of approaches and methods. In the following, some Almost every author uses at least one of these papers are presented shortly regarding the use of characteristics. The characteristics feasible, implementation characteristics from [11] and [17]. TABLE I. ANALYSIS OF RELATED WORK Research Category clear and concise implementation independent consistent necessary verifiable traceable complete singular feasible Author(s) Wilson et al. 1997 [8] A ● ● ● ● ● Mich & Garigliano 2000 [24] A ● Fabbrini et al. 2001 [25] A ● ● ● ○ Fantechi et al. 2003 [26] A ○ ● ○ Ilieva et al. 2005 [27] T Kaiya & Saeki 2006 [29] T ● ● ● Ormandjieva et al. 2007 [21] C ● Berry et al. 2007 [28] A ● ● ● ○ Verma & Kaas 2008 [30] T ● ● ● Holtmann et al. 2011 [31] T ○ ● ● Yang et al. 2012 [32] C ○ Genova et al. 2013 [7] A ● ● ● ○ ○ ● ● Huertas et al. 2013 [33] C ● ● ○ Ghosh et al. 2014 [37] T ● ● ○ Soeken et al. 2014 [34] C ○ ○ ○ ○ Arellano et al. 2015 [35] T ● ● Parra et al. 2015 [36] C ○ ○ ○ ○ ○ ● Identical ○ Similar ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 55 In [8], the authors present a tool (ARM) to identify property correctness is used further in the approach. requirements that need to be improved. They use However, the authors do not offer a description of “desirable characteristics” [8, p. 2] based on IEEE Std 830- correctness. The research is mainly based on the properties 1993 [38] – a predecessor of the current ISO standard [11]: and approaches from [7] and additionally neglects complete, consistent, correct, modifiable, ranked, testable, consistency and completeness. Their method is composed traceable, unambiguous, understandable, valid and of two tasks. In the first task, they generate classifiers. verifiable. Identical characteristics can be found in the They let experts classify requirements according to their current ISO standard [11]. The characteristics feasible, quality and use a set of metrics associated with the implementation independent, necessary and singular are requirements to build their classifiers. In the second task, not used. However, this research paper serves as they estimate and evaluate the quality of new requirements groundwork and is mentioned in several approaches as based on the same set of metrics. The tool developed by starting point. [7] is used to extract the metrics for each requirement. In [25], a tool (QuARS) displays requirements together The characteristic correctness is not mentioned in the with automatically detected indicators. Four quality ISO standard [11]. However, previous standards [38] and properties are mentioned: non-ambiguity, specification several authors use it to describe properties of a completion, consistency, understandability. The latter requirement [8], [25], [3]. “There is no tool or procedure property is influenced by multiplicity that is pointed out if that ensures correctness” [38, p. 4]. In [39] correctness is the requirement “has more than one main verb or more defined as the state when “[t]he requirement [is] an than one direct or indirect complement that specifies its accurate representation of the entity need from which it subject” [25, p. 4]. Reference [11] describes the was transformed” [39, p. 18]. According to this definition, characteristic singular in a similar way: “[t]he requirement correctness is linked with the traceability of a requirement. statement includes only one requirement with no use of Reference [8] defines the correctness of a requirement conjunctions” [11, p. 11]. Clear and concise, complete and when it “accurately and precisely identif[ies] the individual consistent are used analogue. Other characteristics are not conditions and limitations of all situations” [8, p. 2]. In mentioned. [25] the authors describe correctness evaluation as “the verification that the system to be constructed is correctly Reference [27] presents an approach for the automatic described by them” [25, p. 2]. For [28] correctness is “the transition of natural language software requirements into lack of factual errors […] and matching what the customer formal presentation. The authors dissect each requirement wants” [28, p. 3]. The description of correctness differs, sentence in its constituents subject, predicate and object and the characteristic is not clearly defined. In our and arrange word groups in tabular form. An entity research, we see correctness as a comprehensive criterion relationship diagram is constructed from the tabular that influences several characteristics from the ISO presentation using nouns as entities and verbs and standard. However, due to an inconsistent and not prepositions as relationships. Characteristics are not used standardized definition, correctness is not used as at all to ensure the transition into formal specification from characteristic in the rating sessions. a qualitative point of view. Reference [30] describes a tool (RAT) to detect and IV. RATING SESSIONS flag requirements that violate presented best practices. The best practices build upon “common requirements With the overview and analysis of related work, we problems” [30, p. 753] like ambiguous terms, inconsistent reveal that several characteristics for the quality and incomplete requirements. The common problems and determination of textual requirements are rarely an analogue inverse description can be found for clear and investigated. Generally, we do not find approaches, which concise, consistent and complete in [11] and [17]. take into consideration all characteristics from [11]. Therefore, to address this research gap, we enable the In [7], the authors present desirable properties and rating of the whole set of characteristics. indicators used to evaluate and measure quality in textual requirements. A tool (RQA) displays requirements Our work takes an additional approach from the together with automatically detected indicators and an mentioned literature: we use experts’ knowledge to rate overall perceived quality score. These indicators are based requirements but also to collect their inputs about different on following characteristics: atomicity, precision, characteristics. Working closely with an automotive completeness, consistency, understandability, unambiguity, engineering company allows us to work within an traceability, abstraction, validability, verifiability and industrial environment and with experts from the modifiability. Abstraction is defined as to “tell what the automotive industry. In this chapter, we present the application must do without telling how it must do it” [7, p. preparation, the execution and the results of the 28]. The description of implementation independent is requirements rating sessions. similar and “states what is required, not how the requirement should be met” [11, p. 11]. Same applies for A. Preparation atomicity where a requirement is “clearly determined” [7, The preparation phase includes information about how p. 28] and singular where a “statement includes only one requirements data is collected and prepared for the usage in requirement” [11, p. 11]. the rating tool. We collect English and German text data from 83 software requirement specifications of different Reference [36] describes a method to classify the automotive development projects. These software projects quality of textual requirements by using machine-learning aim at developing advanced driver assistance systems, techniques. The authors emphasize desirable properties of such as lane assist, collision avoidance functions and other a requirement: correct, consistent and complete. The safety systems. The data initially consists of 57.801 ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 56 objects, including 9.365 headings and 10.775 objects TABLE II. COEFFICIENT OF DETERMINATION (R²) FOR OVERALL PERCEIVED RATING marked as information. The object types heading and information are used to structure the specification R² documents and do not contain relevant requirement information in general. We remove objects belonging to complete 0.63 these types, as we only want to consider requirement clear and concise 0.62 objects for the rating. verifiable 0.45 In further steps, we reduce the database by English feasible 0.44 requirements, as our focus group of experts has German as mother language. We are convinced that once we prove the necessary 0.44 feasibility for German requirements the approach can consistent 0.38 easily be adapted to English requirements as well. At the end of the data cleansing steps, the dataset consists of traceable 0.34 14.341 unique software requirements in German language. singular 0.27 We then randomly select 766 unique requirements and implementation independent 0.24 import this dataset twice (1532 requirements) in a self- developed rating tool. This tool allows saving the rating results during the sessions as it is linked with a pre-defined Compared with the analysis of related work in the database. We import the dataset twice as we want to have previous chapter (cf. Table I), the research focus on clear two evaluations for each requirement from different and concise and complete is comprehensible. However, the experts. characteristic consistent, also occurs often in the related In the beginning of the survey, we ask general work, has less coherence with the overall perceived rating information about the expert’s role and years of of a requirement. Moreover, the characteristics verifiable, experience. The experts then rate random requirements feasible and necessary are worth to investigate in further according to the characteristics defined in [11] and [17]: research about quality determination of textual clear and concise, complete, consistent, feasible, requirements. Especially the two latter characteristics are implementation independent, necessary, singular, almost completely neglected in the current research. traceable and verifiable. Each characteristic can be rated Lastly, singular and implementation independent are less between 1 (very bad) and 5 (very good). If an expert coherent with the overall perceived rating. cannot rate a characteristic for a requirement, the answer In the second area of investigation, the ratability of a no rating possible (NRP) is possible to select. After the requirement is analyzed. The ratability describes if experts rating of each characteristic, the experts conduct the are able to rate characteristics of a requirement and overall perceived rating of the requirement between 1 identifies possible challenges for the rating. In this analysis (very bad) and 5 (very good) including the option no rating we follow the research question from our introduction: possible (NRP). The experts confirm the rating and the how reliable and consistent human experts can rate the next requirement is displayed. quality of software requirements. B. Execution and Results Every requirement is rated twice by different experts. Disagreements in the rating of characteristic are presented During May and October 2018, nine female and 83 in Table III. The first row “Average Variance” regards the male experts from the automotive industry rated 1532 variance in the evaluation of a single requirement. If the software requirements (766 unique requirements). On value is low, the individual rating of the experts average, the experts have five years of experience in their corresponds. A high value implies discrepancies between respective role. We identify nine different expert roles: test the evaluations. engineer, failure manager, systems integrator, function owner, function developer, requirements manager, The value for the average variance range between 0.47 requirements engineer, software developer, software for feasible and 0.62 for clear and concise and complete. architect. For clear and concise and complete the average variance value above 0.60 is comparable high and indicates minor We aggregate the rating results in two areas of challenges regarding a consistent rating between experts. investigation: influence and ratability. In the first area of Feasible has the lowest variance among the ratings and can investigation, we determine the individual coherence of be evaluated more consistently. The remaining each characteristic with the overall perceived rating of a characteristics have medium values between 0.52 and 0.60 requirement through the coefficient of determination. for the average variance. Table II shows the coefficient of determination Further, we analyze the percentage of “NRP” (no rating according to the overall perceived rating of a requirement possible) for each characteristic. If an expert cannot rate a based on univariate regression. Singular and characteristic for a requirement, this option is possible to implementation independent are less coherent with the select. With the interpretation of this value, we can derive overall perceived rating with a result less than 0.30 additional information about the ratability and reveal compared to other characteristics. The R²-value for clear difficulties in the rating. The characteristic consistent, for and concise and complete is greater than 0.60 and implies example, could not be rated for 17% of all requirements. a strong coherence with the overall perceived rating. The R²-value for the remaining characteristics range between 0.34 for traceable and 0.45 for verifiable. ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 57 TABLE III. EVALUATION DISCREPANCIES Several relevant approaches use parts of these quality Average criteria and enable a qualitative analysis of textual % NRP requirements. However, we do not find approaches that use Variance the whole set of proposed characteristics. perceived overall rating 0.52 2% With the identification of the research gap, we develop complete 0.62 5% a rating tool that enables experts to rate the quality of clear and concise 0.62 <1% textual requirements. The rating is based on the characteristics from the modified ISO standard [11, 17]. verifiable 0.56 4% The tool let experts carry out ratings on textual feasible 0.47 13% requirements from industrial projects of the automotive industry. The results from the rating sessions help us to necessary 0.57 16% reveal relations between individual characteristics and the overall perceived rating of a requirement. We derive consistent 0.56 17% following insights: traceable 0.55 35%  Influence. We reveal coherences from individual singular 0.54 2% characteristics with the overall perceived rating based on univariate regression. We identify clear and concise implement. independent 0.60 4% and complete as coherent characteristics with the overall perceived rating of a requirement. Research about the quality of textual requirements based on these The experts also have difficulties in rating traceable, characteristics is represented well in the related work. feasible and necessary. An explanation for the rating We also recognize singular and implementation difficulties is that further information is not available for independent to be less coherent with the overall several individual requirements. In addition, the individual perceived rating. Further, the characteristics feasible requirement does not offer sufficient indications to rate the and necessary, barely investigated in the current characteristic. Another reason could be the expert’s research about quality determination of textual background and level of experience that leads to different requirements, have a higher coherence with the overall forms of implicit knowledge and the ability to rate perceived rating compared to the characteristic requirements despite less information value. A deeper consistent. We identify a lack of research regarding the analysis is part of the future research. On the other hand, coherence of characteristics with the overall perceived for clear and concise, singular and the perceived overall rating and existing literature. Moreover, we do not find rating the experts are able to rate almost every a comprehensive approach considering all presented requirement. For complete, verifiable and implementation characteristics. independent, the rating is less difficult as for about 5% of  Ratability. The ratability enables us to identify the requirements a rating through the experts was not challenges and difficulties in the rating of possible. requirements. We find characteristics with similar Clear and concise is a characteristic well considered in variance values that indicates a common understanding the literature about the quality determination of textual and rating between experts. Based on a comprehensive requirements. Thus, the coherence with the overall analysis we derive several results regarding the perceived rating of a requirement is not surprising at all as ratability of characteristics. For experts the well as the ratability of this characteristic. However, the characteristics singular, clear and concise and the experts rating for clear and concise is not corresponding as perceived overall rating is possible to rate. For some compared to other characteristics. Same applies for characteristics the experts have difficulties in rating and complete. Other characteristics, such as feasible and providing consistent answers. This implies especially necessary, reveal coherence with the overall perceived for traceable, where more than a third of the rating of a requirement as well. The variance for feasible is requirements could not be rated. comparable low. However, experts take the option “no This research has also some limitations. In particular, rating possible” more often. Approaches in the literature our approach is based on the assumption, that requirements especially for feasible and necessary are barely available. can be handled as natural language despite their high Regarding these facts it is crucial to consider these technical and specific vocabulary. As only 1532 characteristics in the future research about the quality requirements (766 unique requirements) are rated until determination of textual requirements. today, the results comprise only a first step towards a comprehensive approach. V. CONCLUSIONS AND LIMITATIONS As depicted in Table I of the paper, several This paper sets up to explore and analyze quality characteristics are neglected in the current research about approaches of textual requirements. We reveal that the quality determination of textual requirements. literature on requirements quality largely falls into three Although the evaluation of some characteristics does not categories: Assistance, Transformation and Classification. seem to be conducive at the first sight, we do not exclude Several attempts to define requirements quality are characteristics in our research and in the rating sessions available. We identify the modified ISO/IEC/IEEE with experts. Thus, the observed difficulties for rating e.g. 29148:2011 standard [11, 17] as the relevant one. The traceable and consistent stem from the evaluation standard describes how requirements quality is defined and approach: As each participant only rates a single, random offers relevant characteristics to determine quality as well. requirement, the expert is not able to have a full view on ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 58 all requirements even not at the very end of the rating [13] IEEE Standard Glossary of Software Engineering Terminology, session process. In order to depict the resulting end system The Institute of Electrical and Electronics Engineers, IEEE Std. 610.12-1990, New York, 1990. or to check the dependencies to other requirements – in [14] C. Rupp, Requirements-Engineering und -Management: Aus der terms of consistency or horizontal traceability – full Praxis von klassisch bis agil, 6th ed., Carl Hanser Verlag, knowledge of the whole set of requirements would be München, 2014. helpful. Though, we are convinced that experts can [15] S. Robertson and J. Robertson, Mastering the requirements evaluate a single requirement as well. When rating e.g. the process: Getting requirements right, 3rd ed., Addison-Wesley, characteristic consistent, the expert only concentrates on 2013. the individual requirement and focus the consistency [16] G. Kotonya and I. Sommerville, Requirements engineering: within its statement(s). processes and techniques, 1st ed., Wiley Publishing, 1998. [17] G. Fanmuy, J. Marvin, and H. Ronald, Deployment Package - Further limitations include implicit knowledge as an Needs and Requirements Engineering - Systems Engineering Basic important fact that strongly correlates with the expert’s Profile. Technical report, International Council on Systems level of experience. It stresses the point that the creation of Engineering (INCOSE), 2014. requirements should follow closely defined quality criteria [18] F.P.J. Brooks, “No silver bullet-essence and accidents of software suggested by standards like ISO/IEC/IEEE 29148:2011 engineering,” in Proceedings of IFIP Tenth World Computer Conference, pp. 1069–1076, 1986. [11] to allow a better understanding for persons with less [19] S. Boyd, D. Zowghi, and A. Farroukh, “Measuring the implicit knowledge. expressiveness of a constrained natural language: an empirical study,” 13th IEEE Internationl Conference on Requirements Based on these results, further research questions arise Engineering, pp. 339–349, 2005. for future research. The quality of textual requirements can [20] D.M. Berry, E. Kamsties, and M.M. Krieger, From contract now be determined by using characteristics from the drafting to software specification: linguistic sources of ambiguity, modified ISO standard. The next step is to investigate, how Handbook, 2003. to measure the quality of textual requirements by using [21] O. Ormandjieva, I. Hussain, and L. Kosseim, “Toward a text quality characteristics. Is it possible to derive proven classification system for the quality assessment of software relations between quality characteristics and indicators? As requirements written in natural language,” in Fourth international quality characteristics are qualitative, they can only be workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting, pp. 39–45, 2007. judged and not be measured. Thus, we need to identify [22] A.J. Tyznik, S. Verma, Q. Wang, M. Kronenberg, and C.A. indicators that can be measured quantitatively and Benedict, “Distinct requirements for activation of NKT and NK represent defined quality characteristics as well. cells during viral infection,” in The Journal of Immunology, vol. 192, no. 8, pp. 3676–3685, 2014. REFERENCES [23] International Organization for Standardization, International Standard 26262: Road vehicles - Functional safety, International [1] D. Mohr et al., “The road to 2020 and beyond: What’s driving the Standard, 1st ed., 2011. global automotive industry?” McKinsey & Company, 2013. [24] L. Mich and R. Garigliano, “Ambiguity measures in requirement [2] F. Sagstetter et al., “Security challenges in automotive engineering,” in Proceedings of the International Conference on hardware/software architecture design,” in Proceedings of the Software Theory and Practice, pp. 39–48, 2000. Conference on Design, Automation and Test in Europe, pp. 458– [25] F. Fabbrini, M. Fusani, S. Gnesi, and G. Lami, “The linguistic 463, 2013. approach to the natural language requirements quality: benefit of [3] H. Lan, C. Zhang, and H. Li, “An open design methodology for the use of an automatic tool,” in Proceedings of the 26th Annual automotive electrical/electronic system based on quantum NASA Goddard Software Engineering Work, 2001. platform,” in Advances in Engineering Software, vol. 39, pp. 526– [26] A. Fantechi, S. Gnesi, G. Lami, and A. Maccari, “Application of 534, 2008. linguistic techniques for use case analysis,” in Requirements [4] M. Wolf, Security Engineering for Vehicular IT Systems. Engineering, vol.8, pp. 161–170, 2002. Vieweg+Teubner, 2009. [27] M. Ilieva and O. Ormandjieva, “Automatic transition of natural [5] M.I. Kamata and T. Tamai, “How does requirements quality relate language software requirements specification into formal to project success or failure?” in 15th IEEE International presentation,” in Natural Language Processing and Information Requirements Engineering Conference, pp. 69–78, 2007. Systems, vol. 3513, pp. 427–434, 2005. [6] E. Knauss, C. El Boustani, and T. Flohr, “Investigating the impact [28] D.M. Berry, A. Bucchiarone, S. Gnesi, G. Lami, and G. Trentanni, of software requirements specification quality on project success,” “A new quality model for natural language requirements in International Conference on Product-Focused Software Process specifications,” in Proceedings of the International Workshop on Improvement, pp. 28–42, 2009. Requirements Engineering: Foundation of Software Quality, [7] G. Génova, J.M. Fuentes, J. Llorens, O. Hurtado, and V. Moreno, REFSQ, pp. 1–12, 2006. “A framework to measure and improve the quality of textual [29] H. Kaiya and M. Saeki, “Using domain ontology as domain requirements,” in Requirements Engineering, vol. 18, pp. 25–41, knowledge for requirements elicitation,” in Requirements 2013. Engineering, 14th IEEE International Conference, pp. 189–198, [8] W.M. Wilson, L.H. Rosenberg, and L.E. Hyatt, “Automated 2006. analysis of requirement specifications,” in International Conference [30] K. Verma and A. Kass, “Requirements analysis tool: A tool for on Software Engineering, pp. 161–171, 1997. automatically analyzing software requirements documents,” in [9] D.M. Berry, “Ambiguity in natural language requirements International Semantic Web Conference, pp. 751–763, 2008. documents,” in Monterey Workshop, Springer, Berlin, Heidelberg, [31] J. Holtmann, J. Meyer, and M. von Detten, “Automatic validation pp. 1–7, 2007. and correction of formalized, textual requirements,” in Proceedings [10] J. Kasser, “The First Requirements Elucidator Demonstration of the 4th IEEE International Conference on Software Testing, (FRED) tool,” in Systems Engineering, vol. 7, pp 243–256, 2004. Verification and Validation Workshops, pp. 486–495, 2011. [11] ISO/IEC/IEEE 29148:2011, Systems and software engineering - [32] H. Yang, A. De Roeck, V. Gervasi, A. Willis, and B. Nuseibeh, Life cycle processes - Requirements engineering, Geneva, “Speculative requirements: automatic detection of uncertainty in Switzerland, 2011. natural language requirements,” in 20th IEEE International [12] Merriam-Webster, Definition of requirement, https://bit.ly/2st8p1c Requirements Engineering Conference (RE), pp. 11–20, 2012. (Accessed: 11.01.2019). ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 59 [33] C. Huertas and R. Juárez-Ramírez, “Towards assessing the quality [36] E. Parra, C. Dimou, J. Llorens, V. Moreno, and A. Fraga, “A of functional requirements using English/Spanish controlled methodology for the classification of quality of requirements using languages and context free grammar,” in The Third International machine learning techniques,” in Information and Software Conference on Digital Information and Communication Technology, vol. 67, pp. 180–195, 2015. Technology and its Applications (DICTAP2013), pp. 234–241, [37] S. Ghosh, N. Shankar, P. Lincoln, D. Elenius, and W. Li, 2013. “Automatic requirements specification extraction from natural [34] M. Soeken et al., “Quality assessment for requirements based on language (ARSENAL),” SRI International Menlo Park CA, pp. 1– natural language,” Special Session at the Forum on specification & 14, 2016. Design Languages (FDL), Munich, 2014. [38] IEEE Std 830-1993 - Recommended Practice for Software [35] A. Arellano, E. Carney, L. Martin, C. Park, and M.A. Austin, Requirements Specifications, 1993. “Frameworks for natural language processing of textual [39] INCOSE: Guide for writing requirements. Technical report, requirements,” in International Journal On Advances in Systems Requirements Working Group, 2015. and Measurements, vol. 8, pp. 230–240, 2015. ASE 2019: 16th Workshop on Automotive Software Engineering @ SE19, Stuttgart, Germany 60