The article deals the task of comparing of agree of the results of expert evaluation conducted by several independent groups of experts of the software of teacher' rating of higher educational institutions. The resulting evaluation of the expert group calculated by the modified method of expert evaluation of software systems based on interval data analysis. The resulting evaluation of expert groups will can improve the "weak" aspects of the software system and can help conduct analyze the expert assessments too.
One of the important tasks of the present is the research of
one of the most important characteristics of software systems
- quality. Under the quality of the software system, we
understand the set of the software product properties, which
characterizes its ability to meet the established or predicted
needs of the customer, which he expressed in the form of user
requirements for software in the early stages of its
development [
In view of the above, the task of studying agree of expert’ evaluations of different groups in estimating software systems of any practical or theoretical direction is actuality.
The evaluation of the software system quality begins from
the early stages of development, including the definition of
the specification requirements for the software system,
analysis, design, etc. The quality estimation process at these
stages usually involves design engineers who themselves
evaluate some part of the system they are developing. After
that, the results of their estimations are integrating and
averaging. Of course, the resulting evaluation in this case will
be overestimated [
Approaches and recommendations for the process of expert
evaluation of the software systems quality in various
scientific schools, studies by individual scientist cover
various aspects of this issue. Some works try to formulate
recommendations for the evaluation process. In the work [
One of the important drawbacks of the method is the exact
formalization of the expert's opinion when evaluating a
particular criterion [
To take into account the possible questionable expert
evaluations according to certain criteria, the method of expert
evaluation of the software systems quality to be realization
using the methods of interval data analysis are proposed
[
Interval evaluation of the software system based on the set
up of the upper and lower limits [
The resulting expert evaluation has the form: [ X k ] = ∑[xmin ; xmax ] ⋅ cm ,
m where [ X k ] = [ X k− ; X k+ ] - interval evaluation of the expert of the area, which takes into account the percentage of the expert's rating deviation from the nominal value; cm percentage indicator of importance of the criterion of software evaluation, ∑ cm = 1 .
m
The resulting evaluation of the software system quality in this case will take the form:
∑[ X k ] ⋅ qk [ X ] = k
, ∑ qk k where qk - the weight indicator of the individual group; [ X ] = [ X e−x ; X e+x ] - the resulting evaluation of all groups of experts, which is the interval of confidence to the software developing.
The condition of the agree of the obtained interval
estimation of the software system quality to the admissible is
[
[ X e−x ; X e+x ] ⊂ [ X min ; X max ] , where [ X min ; X max ] - the established interval of software evaluation, which is guaranteed to satisfy software developers.
III. EXAMPLE OF THE EXPERT EVALUATION OF THE SOFTWARE OF TEACHER’S RATING OF HIGHER
In Fig. 1 schematically illustrated the process of expert evaluation the quality of the software system.
As we can see from the figure, the resulting evaluation of the software system quality depends on the estimates made by experts at the initial stage.
We will conduct a comparative analysis of evaluations put forward by different groups of experts on the example of the software of teacher’ rating of higher educational institute. Shortly about the rating system of teachers. The first version of this system developed by the Master of the Department of Computer Science in 2017. In Fig. 2 shows the main window of the system. (2) (3) (4) On Fig. 3 is provide the general rating of university teachers.
The work of the teacher evaluated in the context of such activities as research work, educational work, methodical work, organizational work, qualifications and additional criteria. On fig. 4 showed one of the windows for capability reporting. one in order to take into account the "doubts" of experts of the setting score.
To evaluate the quality of this software system in an objective way and the possibilities for its further perfection and improvement, it decided to conduct its evaluation. Leading departments of the Faculty of Computer Information Technologies were in the role of expert groups.
The software system evaluated according to the following
criteria [
1. The correctness of the work (the system must be isolated from external influences and the result of performing the functions should be correct in all conditions).
2. Protection from unauthorized access 3. Program reliability (the system must be resistant to various user-side influences) 4. Comfortable graphical user interface 5. Low cost of hardware resources (the system should not require high hardware costs of the computer)
6. Mobility (the system should have a small amount of memory, a small amount of processor time, etc., so that it can be used on any PC)
7. Scalability (improving the capabilities of the system by introducing a new functional) 8. Convenience of use 9. Speed 10. Completeness of functional requirements (the system must meet all of its functional requirements from the side of the subject area).
The software system was evaluated by the experts of the following categories: expert in the area (in the table - EA), business analyst (BA), software architect (SA) and expert of user interface (EI). Percentage coefficient importance of the criterion cm for each of the criteria listed above, accordingly 0,05; 0,05; 0,05; 0,1; 0,05; 0,05; 0,05; 0,1; 0,1; 0,4. System developers want to achieve system quality with a minimum threshold of 80%. Below in the tables 1-3 shows the upper limit for the expert's estimate. The lower limit of the evaluation will formed with a deviation 5% from the upper 1 9 9 2 9 9 3 9 8 4 8 8 5 9 9 6 8 7 7 8 8 8 9 9 9 9 8 10 9 8 Weight 7 8 indicator of each expert The resulting [8,35;8,8] [7,79;8,2] evaluation of a group of experts The resulting evaluation of the project
The paper is devoted to the problem of studying and evaluating the quality of software systems. The methods of expert evaluation are considered. It indicated that most of them carry only theoretical recommendations for improving the evaluation process. Some methods aimed at improving the visual presentation of the evaluation results. The results of the analysis showed the actuality and importance of paying special attention to the selection of experts who would evaluate software systems so that their opinion was objective and unbiased. It proved that, in order to avoid "doubts" regarding the evaluation of a particular criterion, use the methods of interval data analysis. The method of expert evaluation of software systems based on the analysis of interval data is developed, the result of which is to check the agree of the resulting interval estimation according to the project proposed by an independent group of experts and the set the interval of evaluation of the software system that satisfies the developers. On the example of the expert evaluation of the rating system of the university teachers, the diversity of opinions of different groups of experts shown.