The actual task is to develop the defect-free medical software by establishing the presence of residual defects, and the purpose of this study is to develop an intelligent method & system of developing the defect-free medical software by establishing the presence of residual defects. The paper develops a method of developing the defect-free medical software by establishing the presence of residual defects, the essence of which is to identify the set of defects of different levels of criticality and analysis of this set for the presence or absence of residual defects. The method differs from the known ones in that the input information about the results of the basic testing is processed by an artificial neural network. The proposed system of developing the defect-free medical software by establishing the presence of residual defects allows the user to obtain a conclusion about establishing the presence or absence of residual defects (indicating the level(s) of criticality of residual defects in case of their presence) based on the processing of information on the number and types of defects detected during the basic testing contained in the basic test report. On the basis of heuristic estimates, the threshold values of the number of defects of each criticality level are defined, after exceeding which a conclusion is made about establishing the presence or absence of residual defects (indicating the criticality level(s) of residual defects in case of their presence). In general, the proposed system development of defect-free software by detecting residual defects in medical software after the basic testing, thereby increasing the veracity of testing and, accordingly, increasing the quality of medical software.
2021 Copyright for this paper by its authors. From Till software developers and testers, the impact of their inherent subjective shortcomings, etc.) factors [1619]. Such defects are referred to as residual defects.
< 2 K
On the one hand, the annual increase in the number of lines of program code [25], and on the other – the increase in the number of news about software defects [26], gives the right to argue the potential increase in the number of residual software defects.
Even more residual defects remain in multidisciplinary medical software, which is developed at
the intersection of the medical domain and software engineering domain, when there is a different
understanding of the context of information, the need to consider both software development
standards and standards of the medical field [
Since both the veracity of testing and the quality of software depend on the number of defects found in it, including residual, the increase of the veracity of testing and, accordingly, the quality of software can also be achieved by developing the defect-free software by detecting residual defects in software after basic testing.
Then actual task is to develop the defect-free software by establishing the presence of residual defects, and the purpose of this study is to develop an intelligent method & system of developing the defect-free software by establishing the presence of residual defects. 2. Concept of Developing the Defect-Free Medical Software by Establishing the Presence of Residual Defects
Let's clarify the distribution of the type of software defects to form a conclusion about the presence or absence of residual software defects. All software defects are divided into minor, moderate, serious and catastrophic.
Minor software defects will be such defects that do not affect the user's actions, the software with their presence is suitable for use.
Moderate software defects are considered to be such defects that affect the user's actions, but the software with their presence is still suitable for use with partial loss of functionality.
Serious software defects will be considered such defects that lead to erroneous results, as a result of which the software is already unusable.
Catastrophic software defects will be considered such defects that lead to distortion of information (data), as a result of which the software is unusable and an attempt to use it leads to system failure.
Minor defects are assigned the lowest criticality level – the first. Moderate defects are assigned, respectively, the 2nd criticality level; serious – the 3rd criticality level. The highest criticality level is assigned to catastrophic defects – 4th level.
The accumulation of a certain number of minor defects leads to the appearance of moderate defects, the accumulation of a certain number of which, in turn, leads to the appearance of serious defects, the accumulation of a certain number of which, accordingly, leads to the appearance of catastrophic defects. This is justified by the fact that the longer the defect is in the software development cycle, the more it penetrates into all software components and the more damage it causes at all stages to all components. Thus, defects of one level of criticality can be the cause of residual defects not only of the next level of criticality, but also of residual defects of higher levels of criticality. In this case, we introduce a threshold of the allowable number of defects, above which it is necessary to conclude that the presence of residual defects of the next level of criticality.
The initial data for forming a conclusion about the presence or absence of residual defects of a certain level of criticality is information about the number and types of defects detected during the basic testing, i.e. the report on basic software testing.
Finding as many residual defects as possible will increase the veracity of the testing process and, accordingly, the quality of the software, which, in turn, will make it possible to develop the defectfree software.
The concept of developing the defect-free medical software by establishing the presence of residual defects using an artificial neural network (for solving the difficult-to-formalize problem of determining the importance and criticality of software defects and their mutual influence, vagueness of initial data about existing defects, etc.) is presented in Figure 2. 3. Method & System of Developing the Defect-Free Medical Software by Establishing the Presence of Residual Defects
For describing the rules of forming a conclusion about establishing the presence of residual defects, we enter the threshold ri R ( R { r j | j 1..k } , where r j – the threshold of the allowable number of defects, exceeding of which leads to the conclusion is formed about the presence of residual defects, j – the number of criticality types varying from 1 to k , k – the number of criticality levels of defects (i.e. now k 4 )).
The general rule. If the ratio of the total value of defects of the i -th criticality level to the total number of defects detected during the basic testing (or the total value of defects of the i -th criticality level) exceeds the threshold ri , it is concluded about establishing the presence of residual defects ( i 1) -th (next) criticality level.
The following specific rules of forming the conclusion about establishing the presence of residual defects follow from the general rule:
1. If the ratio of the total value of defects of the 1st level of criticality (minor defects) to the total number of defects detected during the basic testing is equal to or exceeds the threshold r1 , it is concluded about establishing the presence of residual defects of the 2nd criticality level (moderate defects).
2. If the ratio of the total value of defects of the 2nd level of criticality (moderate defects) to the total number of defects detected during the basic testing is equal to or exceeds the threshold r2 , it is concluded about establishing the presence of residual defects of the 3rd criticality level (serious defects).
3. If the total value of defects of the 3rd criticality level (serious defects) exceeds the threshold r3 , the conclusion is made about establishing the presence of residual defects of the 4th criticality level (catastrophic defects).
4. If the total value of defects of the 4th criticality level (catastrophic defects) is equal to or exceeds the threshold r4 , then a conclusion is made about the unsuitability of the software and the possible failure of the system.
Method of developing the defect-free medical software by establishing the presence of residual defects consists of the following steps:
1. Based on the results of the artificial neural network (ANN) the set D { di | i 1..4 } is formed, where d i – the total values of the defects of each of the criticality levels. d 2. Based on the set D the set DN { dni | i 1..4 } of ratio dni i is formed, where n – the total n number of software defects detected during the basic testing.
3. All the rules of forming the conclusion about establishing the presence of residual defects are analyzed, and based on known facts (elements d 3 and d 4 of the set D; elements dn1 and dn2 of the set DN) conclusions are sought, which follow from these facts. The analysis of the rules of forming the conclusion is as follows. In the above set of rules the search for a rule for elements d 3 and d 4 of the set D and elements dn1 and dn2 of the set DN is conducted. If the value of the elements satisfies the condition of the left part of the rule, then this rule, in fact, becomes one of (or the only) conclusions of the method. If none of the rules worked, then a conclusion is made about establishing the absence of residual defects.
Based on the developed rules and method the structure of system of developing the defect-free medical software by establishing the presence of residual defects is developed (Figure 3).
System of developing the defect-free medical software by establishing the presence of residual defects functions as follows. The user of the system submits a report on basic testing of the medical software. Block of semantic parsing of the report and preparation of data for ANN performs semantic parsing of the report, selecting from it information about software defects found during the basic testing, and then converts this information from the linguistic form of presentation to quantitative form using data tables of the knowledge base. Knowledge base contains tables for assigning numbers to the types of detected software defects. ANN processes all obtained quantitative information about software defects detected during the basic testing, and then provides information about the levels of criticality of detected defects (0 or 1 for each of the 4 levels, where 1 means that the defect belongs to this level of criticality). The results of ANN are recorded in the data table of the knowledge base. The knowledge base also contains the above rules for forming the conclusion about establishing the presence of residual defects. Further, according to the above method of developing the defect-free software by establishing the presence of residual defects, a conclusion about establishing the presence or absence of residual defects is formed (indicating the criticality level(s) of residual defects in case of their presence) and issued to the user of the system.
Therefore, the proposed system of developing the defect-free medical software by establishing the presence of residual defects allows the user, based on the report on the results of the basic testing, to obtain a conclusion about establishing the presence or absence of residual defects (indicating the criticality level(s) of residual defects in case of their presence).
Since the clarification of the distribution of the type of software defects to form a conclusion about the presence of residual software defects was carried out in this study, the threshold values of the number of defects of each criticality level, exceeding which the conclusion is made about the presence of the residual defects (indicating the criticality level(s) of residual defects in the case of establishing their presence), are not described in known literature references.
For the establishment of these thresholds, a study of the number of the medical software defects was conducted, which was developed by software companies in Khmelnytskyi and consisted of a different number of lines of code, taking into account and without taking into account the impact of defects of one criticality level on the occurrence of the defects of the next criticality level. The results of this study are shown in Table 1. As a result of the analysis of Table 1, the following conclusions can be drawn: 1. For the medical software source code from 100 operators, the number of minor defects is 80% (more than 75%) of the total number of detected defects without taking into account the interaction of defects of one criticality level on the occurrence of defects of the next criticality level, and because of this there 3 moderate defects appearanced; the number of moderate defects is 50% of the total number of detected defects without taking into account the interaction of defects of one criticality level on the occurrence of defects of the next criticality level, and because of this there 2 serious defects appearanced that caused 1 catastrophic defect;
2. For the medical software source code from 1000 operators, the number of minor defects is 76% (more than 75%) of the total number of detected defects without taking into account the interaction of defects of one criticality level on the occurrence of defects of the next criticality level, and because of this there 9 moderate defects appearanced; the number of moderate defects is 56% (more than 50%) of the total number of detected defects without taking into account the interaction of defects of one criticality level on the occurrence of defects of the next criticality level, and because of this there 1 serious defects, 2 serious defects appearanced, that caused 1 catastrophic defect; 3. For the medical software source code from 10000 operators, the number of minor defects is 75% of the total number of detected defects without taking into account the interaction of defects of one criticality level on the occurrence of defects of the next criticality level, and because of this there 17 moderate defects appearanced; the number of moderate defects is 49% (not more than 50%) of the total number of detected defects without taking into account the interaction of defects of one criticality level on the occurrence of defects of the next criticality level, therefore, serious defects due to the accumulation of moderate defects did not occur; the same serious defect did not cause a catastrophic defect.
In this case, the threshold values of the number of defects of each criticality level, in excess of which the conclusion is made about the presence or absence of residual defects (indicating the criticality level(s) of residual defects in case of their presence), are entered on the basis of heuristic estimates (Table 1) as follows:
1. If the ratio dn1 of the total value of defects of the 1st level of criticality (minor defects) to the total number of defects detected during the basic testing is equal to or exceeds 75%, it is concluded about establishing the presence of residual defects of the 2nd criticality level (moderate defects).
2. If the ratio dn2 of the total value of defects of the 2nd level of criticality (moderate defects) to the total number of defects detected during the basic testing is equal to or exceeds 50%, it is concluded about establishing the presence of residual defects of the 3rd criticality level (serious defects).
3. If the total value d 3 of defects of the 3rd criticality level (serious defects) exceeds 2, the conclusion is made about establishing the presence of residual defects of the 4th criticality level (catastrophic defects).
4. If the total value d 4 of defects of the 4th criticality level (catastrophic defects) is equal to or exceeds 1, then a conclusion is made about the unsuitability of the software and the possible failure of the system.
So, elements of the set R (thresholds) have the following values: r1 0.75 ; r2 0.5 ; r3 2 ; r4 1 .
The analysis of Table 1 shows that in excess of such values there are residual defects of higher criticality levels, so these values are used as thresholds in forming a conclusion about establishing the presence or absence of residual defects (indicating the level(s) of critical residual defects in case of establishment of their presence).
Several experiments were conducted with the proposed system of developing the defect-free medical software by establishing the presence of residual defects. During the first experiment, the user of the system submitted a report on basic testing of the software module for automation of medical processes of the family medicine outpatient clinic of Ozerna microdistrict (Khmelnytskyi). Block of semantic parsing of the report and preparation of data for ANN performed semantic parsing of the report, selecting from it information about software defects found during the basic testing, and then converted this information from a linguistic form of presentation to a quantitative form using data tables of the knowledge base. ANN processed information about software defects detected during the basic testing, and then provided the following information: {[0;0;1;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0] [0;1;0;0] [1;0;0;0] [1;0;0;0]}
After deciphering these data, it becomes clear that one defect of the 3rd level of criticality (serious), three defects of the 2nd level of criticality (moderate) and four defects of the 1st level of criticality (minor) were diagnosed.
Further, according to the developed method of developing the defect-free software by establishing the presence of residual defects, sets D1 { 4;3;1;0 } and DN1 { 0.5;0.375;0.125;0 } are formed. All the rules of forming a conclusion about establishing the presence of residual defects are analyzed, and conclusions are found on the known facts, which of these facts follow: dn1 r1 , i.e. rule 1 does not work; dn2 r2 , i.e. rule 2 does not work; d3 r3 , i.e. rule 3 does not work; d 4 r4 , i.e. rule 4 does not work too. Therefore, none of the rules worked, so the conclusion is made about establishing the absence of residual defects for the 1st experiment.
During the second experiment, the user of the system submitted a report on basic testing of the software module "Medical card of an outpatient" of the family medicine outpatient clinic of Ozerna microdistrict (Khmelnytskyi). Block of semantic parsing of the report and preparation of data for ANN performed semantic parsing of the report, selecting from it information about software defects found during the basic testing, and then converted this information from a linguistic form of presentation to a quantitative form using data tables of the knowledge base. ANN processed information about software defects detected during the basic testing, and then provided the following information:
{[0;0;1;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [0;0;1;0] [1;0;0;0] [0;0;0;1] [0;1;0;0] [0;1;0;0] [0;1;0;0] [1;0;0;0] [0;0;0;1] [0;0;1;0] [1;0;0;0] [1;0;0;0] [0;0;0;1] [1;0;0;0] [0;1;0;0] [0;1;0;0] [0;0;0;1]}
After deciphering these data, it becomes clear that four defects of the 4th level of criticality (catastrophic), three defects of the 3rd level of criticality (serious), five defects of the 2nd level of criticality (moderate) and eight defects of the 1st criticality level (minor) have been diagnosed.
Further, according to the developed method of developing the defect-free software by establishing the presence of residual defects, sets D2 { 8;5;3;4 } and DN 2 { 0.4;0.25;0.15;0.2 } are formed. All the rules of forming a conclusion about establishing the presence of residual defects are analyzed, and conclusions are found on the known facts, which of these facts follow: dn1 r1 , i.e. rule 1 does not work; dn2 r2 , i.e. rule 2 does not work; d3 r3 , i.e. rule 3 works; d 4 r4 , i.e. rule 4 works too. Thus, the developed system forms a conclusion about establishing the presence of residual defects of the 4th level of criticality (catastrophic defects) and about the unsuitability of the software and the possible failure of the software module.
During the third experiment, the user of the system submitted a report on basic testing of the software module "Laboratory" of the family medicine outpatient clinic of Ozerna microdistrict (Khmelnytskyi). Block of semantic parsing of the report and preparation of data for ANN performed semantic parsing of the report, selecting from it information about software defects found during the basic testing, and then converted this information from a linguistic form of presentation to a quantitative form using data tables of the knowledge base. ANN processed information about software defects detected during the basic testing, and then provided the following information: {[1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [0;0;1;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0] [1;0;0;0]}
After deciphering this data, it becomes clear that one defect of the 3rd level of criticality (serious), five defects of the 2nd level of criticality (moderate) and twenty-four defects of the 1st level of criticality (minor) have been diagnosed.
Further, according to the developed method of developing the defect-free software by establishing the presence of residual defects, sets D3 { 24;5;1;0 } and DN3 { 0.8;0.17;0.03;0 } are formed. All the rules of forming a conclusion about establishing the presence of residual defects are analyzed, and conclusions are found on the known facts, which of these facts follow: dn1 r1 , i.e. rule 1 works; dn2 r2 , i.e. rule 2 does not work; d3 r3 , i.e. rule 3 does not work; d 4 r4 , i.e. rule 4 does not work too. Thus, the developed system forms a conclusion about establishing the presence of residual defects of the 2nd level of criticality (moderate).
During the fourth experiment, the user of the system submitted report on basic testing of the software module "Accounting of the functional diagnostics room" of the family medicine outpatient clinic of Ozerna microdistrict (Khmelnytskyi). Block of semantic parsing of the report and preparation of data for ANN performed semantic parsing of the report, selecting from it information about software defects found during the basic testing, and then converted this information from a linguistic form of presentation to a quantitative form using data tables of the knowledge base. ANN processed information about software defects detected during the basic testing, and then provided the following information:
{[0;1;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0] [0;1;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0]}
After deciphering this data, it becomes clear that nine defects of the 2nd level of criticality (moderate) and six defects of the 1st level of criticality (minor) have been diagnosed.
Further, according to the developed method of developing the defect-free software by establishing the presence of residual defects, sets D4 { 6;9;0;0 } and DN 4 { 0.4;0.6;0;0 } are formed. All the rules of forming a conclusion about establishing the presence of residual defects are analyzed, and conclusions are found on the known facts, which of these facts follow: dn1 r1 , i.e. rule 1 does not work; dn2 r2 , i.e. rule 2 works; d3 r3 , i.e. rule 3 does not work; d 4 r4 , i.e. rule 4 does not work too. Thus, the developed system forms a conclusion about establishing the presence of residual defects of the 3rd level of criticality (serious).
During the fifh experiment, the user of the system submitted report on basic testing of the software module "Accounting of the office of radiological, fluorographic studies and magnetic resonance imaging" of the family medicine clinic of Ozerna district (Khmelnytskyi). Block of semantic parsing of the report and preparation of data for ANN performed semantic parsing of the report, selecting from it information about software defects found during the basic testing, and then converted this information from a linguistic form of presentation to a quantitative form using data tables of the knowledge base. ANN processed information about software defects detected during the basic testing, and then provided the following information: {[1;0;0;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0] [0;1;0;0] [1;0;0;0] [1;0;0;0] [0;1;0;0] [0;1;0;0]} After deciphering these data, it becomes clear that five defects of the 2nd level of criticality (moderate) and five defects of the 1st level of criticality (minor) were diagnosed.
Further, according to the developed method of developing the defect-free software by establishing the presence of residual defects, sets D5 { 5;5;0;0 } and DN5 { 0.5;0.5;0;0 } are formed. All the rules of forming a conclusion about establishing the presence of residual defects are analyzed, and conclusions are found on the known facts, which of these facts follow: dn1 r1 , i.e. rule 1 does not work; dn2 r2 , i.e. rule 2 works; d3 r3 , i.e. rule 3 does not work; d 4 r4 , i.e. rule 4 does not work too. Thus, the developed system forms a conclusion about establishing the presence of residual defects of the 3rd level of criticality (serious).
During the sixth experiment, the user of the system submitted report on basic testing of the software module "Accounting for the endoscopic office" of the family medicine outpatient clinic in the Ozerna district (Khmelnytskyi). Block of semantic parsing of the report and preparation of data for ANN performed semantic parsing of the report, selecting from it information about software defects found during the basic testing, and then converted this information from a linguistic form of presentation to a quantitative form using data tables of the knowledge base. ANN processed information about software defects detected during the basic testing, and then provided the following information: {[0;0;1;0] [0;0;1;0] [0;0;1;0] [0;0;0;1] [0;0;0;1]}
After deciphering these data, it becomes clear that three defects of the 3r level of criticality (serious) and two defects of the 4th level of criticality (catastrophic) were diagnosed.
Further, according to the developed method of developing the defect-free software by establishing the presence of residual defects, sets D6 { 0;0;3;2 } and DN 6 { 0;0;0.6;0.4 } are formed. All the rules of forming a conclusion about establishing the presence of residual defects are analyzed, and conclusions are found on the known facts, which of these facts follow: dn1 r1 , i.e. rule 1 does not work; dn2 r2 , i.e. rule 2 does not work; d3 r3 , i.e. rule 3 works; d 4 r4 , i.e. rule 4 works too. Thus, the developed system forms a conclusion about establishing the presence of residual defects of the 4th level of criticality (catastrophic defects) and about the unsuitability of the software and the possible failure of the software module.
The results of all 6 experiments will be presented in the form of a diagram of the detected medical software defects of different levels of criticality – Figure 4.
Thus, as shown by the conducted experiments, the proposed system of developing the defect-free medical software by establishing the presence of residual defects as a result of its operation issues a conclusion about establishing the presence or absence of residual defects (indicating the level(s) of critical residual defects in case of their presence) based on the processing of information on the number and types of defects detected during the basic testing, which is contained in the report of the basic testing. Minor software defects Moderate software defects Serious software defects Catastrophic software defects Total software defects
Therefore, the proposed system allows developing the defect-free medical software by detecting the residual defects in the software after the basic testing, thereby increasing the reliability of testing and, accordingly, increasing the quality of the medical software.
On the one hand, the annual increase in the number of lines of software source code, and on the other – the increase in the number of news about software defects, give the right to rightly affirm a potential increase in the number of residual software defects.
The paper develops the method of developing the defect-free medical software by establishing the presence of residual defects, the essence of which is to identify the set of defects of different levels of criticality and analysis of this set for the presence or absence of residual defects. The method differs from the known ones in that the input information about the results of the main testing is processed by an ANN.
The structure of the system of developing the defect-free medical software by establishing the presence of residual defects is proposed, which allows the user on the basis of the report on the results of the basic testing to get a conclusion about establishing the presence or absence of residual defects (indicating the level(s) of critical residual defects in case of their presence). In general, the proposed system allows the development of defect-free medical software by detecting residual defects in software after the basic testing, thereby increasing the reliability of testing and, accordingly, increasing the quality of the medical software. 6. References