- Requirement of Must-Be Quality for software has changed over time, and the awareness of software quality has become increasingly strong. Therefore, an objective and easyto-understand method for analyzing software quality is necessary. By utilizing SQuaRE (ISO/IEC 25000 series) in software development projects, it is possible to proceed with software development with comprehensive quality in mind. However, in order to utilize these quality characteristics, the project is needed to have traceability of quality in various phases or sprints. Therefore, analyzing the quality using by various evidence under development project provides an effective means of explanation of the quality logically and enables us to judge the quality under standard rules. In this paper, we propose a quality coverage analysis method using quality characteristics for software within a general software lifecycle. (Abstract) quality software engineer, quality characteristics,SQuaRE (key words)
The Kano Model[
In the world of software, I feel that this Kano model of quality has been broadly interpreted and used in the same way as minimum requirement of quality. However, we believe that this quality requirement is changing with the times like below bullet. The quality characteristics in parentheses at the end indicates those that match the product quality.
• Quality that was commonplace 10 years ago -Functions work correctly as required (Functional Suitability) -No response problems (Performance Efficiency) • Quality that is commonplace today -Safe to use (Security) • Quality as a matter of course in the future Hiroshi Ishikawa -Quality with the user in mind (Effectiveness Quality in Use-) -Safe and secure to use (Freedom for risk – Quality in Use-) *Quality characteristics in parentheses
Ten years ago, Must-Be Quality indicated that a function
worked exactly as required and that there were no
performance issues. However, this alone is worrisome in a
world where a variety of devices are connected to the Internet.
Every week, numerous cyber incidents are distributed by the
CERT Coordination Center[
ISO/IEC 25010[
The quality model consists of characteristics and subcharacteristics that make up the characteristics. The product quality model classifies the quality of a software or system product into eight characteristics. This product quality provides the 'quality of things'. It means the quality of a thing because it is a quality that the product itself has.
On the other hand, quality in use is the quality that people receive or feel when they use the product. It is called as "user quality" and it is a quality that is perceived mainly by the user. Quality in use is closely related to product quality, and quality in use will not improve unless product quality is exhaustive.
SQuaRE consists of five divisions and defines the means to achieve quality-conscious software development, Figure.1. Utilizing SQuaRE in your development projects will enable you to develop software with quality characteristics in mind.
However, to make full use of quality characteristics,
SQuaRE can be used from the quality requirement phase of a
development project, or all test work must have test cases or
test criteria that are mapped to quality characteristics to
classify the ensured quality into quality characteristics. Since
quality characteristics provides essentially a classification of
quality, they are effective in measuring the coverage of the
quality of the final product. Therefore, we developed a
method to achieve coverage analysis of quality with
classification of the quality of software during a development
project with ISO/IEC/IEEE 12207 [
This section proceeds with the quality coverage analysis from the following artifacts created in a typical software development project.
• Project plan and test plan • Various documents: requirement spec, design spec, test cases and test procedures • Bug information during the project • Review results and Test analysis report • Software and test data The project plan generally describes goal of the quality requirements. Also, the document indicates how to fulfill the quality requirement even though the project use waterfall development process or agile process. The project plan includes test process to list the test types to be conducted and will contain the idea of the criteria for each test type.
Since the quality requirements written in the project plan are the quality goals of the final deliverable software, it is possible to define the required quality by classifying the quality requirements by quality characteristics. It is necessary to analyze both the process and the deliverables to see if the developed software has the quality that is the goal.
The first step to find out if the process was a
qualitybuilding process is to find out which quality characteristics
the various activities during the project have an effect on.
Although it is very time-consuming to do this task from
scratch, ISO/IEC 30130[
Table 1 shows the results of mapping common test types to quality characteristics. You can refer to this table to map the quality characteristics of the test types written in the test plan.
From the above approach, it will be possible to summarize which quality characteristics the development project is performing for from the activities listed in the project plan or sprint plan and the test plan.
REQUIREMENTS
The next step is to map the quality requirements to quality characteristics to identify the quality that the final product, the software, will require. If the quality requirements are not clear, it is a good idea to categorize them into functional requirements, performance and load requirements, user interface requirements, etc., and map each requirement to a quality characteristic. If there is a service specification, the listed items can also be classified as quality characteristics.
Maintainability requirements are generally not included in requirement definitions. Of course, You can refer to the project plan or project rules to determine how software maintenance will be performed. The quality characteristic probably contains coding style rules for programming or testing way for using some stab or debugging technique.
Ideally, the requirements derived from each requirement should also be classified as a quality characteristic. Typically, requirement items are mapped to a single quality characteristic so that the requirements required by the final product, the software, can be counted for each quality characteristic. Since the number of man-hours of classification work increases proportionally with the size of the software, requirements can be sampled or, at worst, only requirements can be classified into quality characteristics.
CREATING A COVERAGE ANALYSIS OF QUALITY
Quality Requirement Mapping to Activity Design/Manufacture/Test Activity during project Mapping to PQ Mapping to PQ Review Criteria Review Results Bug Reports Reference Reference
This work completes the classification of quality characteristics of the quality required for the software.
IV.
Then, classify all the review evidence over the life of the project, e.g., design reviews, code reviews, test design reviews, etc., into quality attributes. Rather than simply mapping activities to quality attributes, we classify criteria into quality attributes. For example, in the case of API design, the criteria and review reports show whether the reviewers check not only the implemented functionality itself, but also error sequencing and recovery methods, performance and load considerations, authentication methods, and vulnerability responses and so on.
The findings that are identified and corrected during the review are also classified by quality characteristics. Corrected bug information is important evidence to ensure the quality of the corresponding quality characteristic.
Finally, we refer to the criteria for each test type performed to see if the quality characteristics mapped to the test type are ensured by having met the criteria. For example, suppose the criteria are set for performance testing, Table 2. If the criteria are too vague to map the quality characteristics of the criteria, the test cases are checked and classified. In addition, bugs found and fixed in the test as well as in the review report are classified according to each quality characteristic to check whether the quality characteristics are maintained or not. Software
Checking of PQ
Mapping to PQ Test Criteria Test Reports
Once all the classification work for completeness analysis using quality characteristics is completed, the results of categorizing the quality requirements are compared with the results of categorizing the final deliverable, the software, to ensure that the quality is on target. The broad analysis is determined by using a radar chart as like Figure 3 to visualize the differences between requirement and results.
The areas that have a large difference between requirements and results indicate possibility of the unclear qualities, which can be pursued by checking the specific classified results and investigating the causes.
In general, functional conformance requirements are often fulfilled. But reliability is not ensured if the scope of impact of a bug fix is not adequately checked or if the functional conformance test is not re-run even though effect of fixed some bug found in a test related to performance efficiency has an impact on functionality. It is also a good idea to analyze whether their impact on relevant quality characteristics has been verified or not, when the bugs which are classified as non-functional conformance have been fixed.
If there is a large difference between the two, you can run some of the tests and check the results of the analysis with your team members to improve the validity of the results. Coverage Analysis
Functional Suitability 20 15 10 5 0 Usability
Results Maintenability
Security Requirement
Performance Efficiency Compatibility Portability Reliability
Quality characteristics enable us to objectively and logically classify the quality of software product, and conduct an exhaustive analysis of whether the quality requirements are being met or not. Although it is most desirable to use SQuaRE from the beginning of a development project, quality analysis using quality characteristics can be performed even after the project is over by using the method described here. In addition, it is possible to identify the quality that is considered to be unsecured from the analysis results, and therefore, it is also possible to consider policies for strengthening weaknesses.
Thus, quality comprehensiveness analysis using quality characteristics not only visualizes the quality of software, but it is also effective as a means to improve quality.
It is also possible to classify test techniques, process transfer criteria and test reports in sprints by quality characteristics. In the case of derived development, it is possible to judge the appropriateness of quality quantitatively for each quality characteristic by using quality data from past development and quality characteristic quantities defined in ISO/IEC 25022. We intend to further improve the quality comprehensiveness analysis method using these quality characteristics so that it can be used as a quality monitoring method that enables objective visualization of quality.
SUMMARY OF CAPABILITIES WITH CHARACTERISTICS IN ISO/IEC 30130 Granularity Input for Code Test Test Quality Test Ver if code analysis plan asset record completio ication analysis report n report and validation
Test Functional Reliability Usability Performance Maintain Portability Compatib Security Smallest Intermedia Largest status suitability efficiency ability ility unit te units unit report 〇 〇 〇 〇
〇 〇 〇 〇 〇 〇 〇 〇