In this article the problems of systems for assessing the quality of knowledge based on test control. The approaches to the development of intelligent systems for testing the quality of knowledge are examined, the functioning of which is based on the apparatus of fuzzy inference. A knowledge assessment model for fuzzy testing systems based on a four-point assessment system is proposed. Also presented are fuzzy systems of testing. In particular, adaptive systems, the advantages of using the fuzzy logic apparatus in building intelligent testing systems designed to improve the accuracy of testing and identifying the quality of knowledge by students. The method of complex assessment of students knowledge based on the Type 2 Tagaki-Sugeno fuzzy model are proposed.
- systems with a combination of open and closed questions.
with full-time examinations, have a number of disadvantages: the examiner's inability to apply an individual approach to each student, a fixed list of questions, difficulties in choosing the next difficulty level and suitable question options, and the main question is how to use the teacher’s experience directly during testing.
Considering the fact that in each specific field of knowledge, the teacher’s experience and skills
can be quite narrow and specialized, we can talk about the need to model their expert reasoning,
which, in conditions of incomplete and inaccurate answers to open questions by students, entailed the
creation of various intelligent fuzzy systems testing, designed to reproduce the train of thought of the
teacher in assessing the knowledge of students [
The aim of this work is to consider the principles of building intelligent testing systems using a fuzzy logic apparatus, as well as based on fuzzy logic inference algorithms in order to improve the quality of knowledge identification and assessment.
Fuzzy inference systems (FS) include the following main stages of their work [
2.1.
In general, a fuzzy system can be expressed as follows [
=1
=1 ⋃ ((⋂ = , ) → = ) , = 1̅̅,̅̅̅, where – number of rules in the IF-THEN fuzzy rule block, – fuzzy rule conclusion, = , – correspondence of a variable to a fuzzy , , ⋃⋂– fuzzy disjunction (conjunction) operations.
Then the conclusion of the fuzzy inference according to the Mamdani algorithm [
The knowledge bases of Takagi-Sugeno fuzzy inference systems contain blocks of fuzzy if-then
rules [
Conventional Takagi-Sugeno fuzzy systems (first order) operate on the basis of if-then fuzzy rules of the form: = ( 1, 2, … , ), = 1,2, … . , , where is the function in the consequent fuzzy rule: ( 1, 2, … , )= 0 + 1 1 + 2 2 + ⋯ + consists in the form of a linear functional dependence on a set of non-fuzzy values of the input variables, – the number of fuzzy rules.
The conclusion of the Takagi-Sugeno fuzzy system (which is a numerical value) is calculate (4): = ∑ =1
min =1… ∑ =1∙ =1… min
( ) ( ) . finding the minimum is used as a conjunction. where ( ) – membership functions in the antecedent of a fuzzy rule, where the operation of The discrete second type fuzzy sets are presented accordingly: [∫ ( )]
, ̃ = ∑ ̃ ( ) = ∑ =1 [∑ = 1 ( )] (3) (4) (5) (6)
Takagi-Sugeno systems of Type 2 (T2) [
The Karnik-Mendel algorithm of fuzzy inference for Takagi-Sugeno T2 systems based on fuzzy
sets with interval secondary functions was developed in [
The continuous T2 fuzzy sets have the form:
∈ ⊆ ∈ [ ̃ ( ), ̃ ( )] ⊆ [
Takagi-Sugeno T2 FS assumes the use of interval T2 fuzzy sets [
̃ ℎ ( )
= 0 + 1 1 + ⋯ + , where ̃ 1 … ̃ – interval T2 fuzzy sets, is the number of the rule. Interval T2 fuzzy sets have the form:
,
= {( , ): ∈ [ ̃ ( ), ̃ ( )]} ⊆ [
Iterative Karnik-Mendel algorithm for Takagi-Sugeno Type 2 fuzzy inference system
The Karnik-Mendel algorithm [
The next steps of the algorithm are the operation of lowering the type and finding the interval
output of the fuzzy system according to formulas (11–13). In [
(x)= [ (x), (x)], (x)= (x)=
min ( )∈[ ( ),
( )] =1,…,
max =1,…, ( )∈[ ( ), ( )] ∑ =1
(x) (x) ∑ =1 (x)
∑ =1
(x) (x) ∑ =1 (x) ,
Thus, multiple calculation of fuzzy rule consequents on the interval [ ( ), ( )] is performed, since the obtained values may differ. The final output value of the Takagi-Sugeno FS T2 is calculated according to (14): (x)= 1⁄2 ( (x)+ (x)).
Figure 2 shows the structure of a Type 2 fuzzy system model, where x = ( 1, 2, … , )– vector of crisp input values, x̃ = ( ̃1, ̃ 2, … , ̃ )– a set of fuzzy input variables obtained as a fuzzyfication result. 3. Adaptive fuzzy inference systems for testing students based on fuzzy selection of the next level of complexity
One of the problems associated with the development of testing systems is the intellectualization of the algorithm for choosing the next level of difficulty when passing a multi-level test by a student. x = ( 1, 2, … , )
x̃ = ( ̃1, ̃ 2, … , ̃ )
(x)
The approach proposed in [
The input data vector is information about the last level of difficulty of the test task passed by the student, the success of his passage, and the average assessment of passing all the tests at the previous stages can also be taken into account.
The output is the value of the selected difficulty level at the next stage of testing. Figure 3, 4 show the linguistic variables “complexity” and “correctness” (relative to the last test performed) by type 2 triangular membership functions with interval secondary membership functions (the numerical scale of difficulty levels depends on the specific system and is not given here):
The knowledge base can be represented by the following block of fuzzy rules (fragment): 1: 1 = 2: 1 = : 1 = 2 =
…
ℎ
1 =
4. Adaptive fuzzy testing systems based on fuzzy processing of student's
answers
evaluating results [
Another approach to creating adaptive testing systems is the intellectualization of the process of inference algorithms when processing student test answers and deriving the final knowledge score.
options for sub-connections. Where the number n corresponds to the number of test tasks, N – to the set of possible options for evaluating the completed task:
N = {“Unsatisfactory”, “Satisfactory”, “Good”, “Excellent”}.
The input vector X x1, x2 ,..., xn is the set of results of answers to many test questions. The membership functions of the input data to a particular fuzzy term for a 100-point scale can be similar to the membership functions shown in Figure 4.
The conclusion of the fuzzy system is a qualitative indicator of a student’s knowledge of the set of N linguistic terms (which, however, can, if necessary, be reduced to a quantitative form). The fuzzy inference algorithm can be selected depending on the way the fuzzy rules consequents are presented.
The knowledge base of the intellectual testing system is presented in the form of fuzzy predicate rules IF-THEN, in which such assessment rules can be displayed that are inherent to a particular teacher, taking into account the field of knowledge.
The fragment of a block of fuzzy rules is presented below: 1: 1 =
2: 3 =
1 = 2 = …
= : 1 =
It is worth noting that in this model the consequents of fuzzy rules are also fuzzy values.
The output value of the fuzzy Mamdani system for estimates on a four-point scale is written as follows:
= { 1, 2, 3, 4}; 1 =
2 = ⋃ ⋃ ⋃ 3 =
⋃ ( ); 4 = ( ).
( ); ( );
There ⋃are determines the fuzzy conjunction operation when performing the accumulation operation of the subclauses of fuzzy Mamdani rules. Сonclusion of the fuzzy system will be produced by defuzzing the output variable Y according to (1).
In the case of using the fuzzy Takagi-Sugeno knowledge base, the ⋃operation is accordingly replaced by the summation operation = ∑
( ), and the output value is calculated according to (4).
The output linguistic variable is shown in Figure 6. We can observe a rather high degree of vagueness of the “Good” and “Excellent” ratings in order to increase the objectivity of knowledge control.
Figure 6 shows the result of the accumulation of the three fuzzy rules presented above and finding the final student grade by the center of gravity method.
This approach to building knowledge quality control systems helps to a large extent to bring the process of evaluating test results closer to that in which pedagogical experience and the methodology for testing the quality of knowledge by an expert teacher are involved. 5. The method of complex assessment of students' knowledge based on the
A method of fuzzy assessment of the quality of knowledge has been developed to obtain a comprehensive characteristic of a student for a training course (module). The Takagi-Sugeno fuzzy inference model with interval fuzzy membership functions of type 2 was taken as a basis. This model allows one to take into account the vague nature of the boundaries of linguistic estimates. Thus, giving at the output a more objective characteristic of knowledge (using the Karnik-Mendel fuzzy inference algorithm according to formulas (11-14).
The Takagi-Sugeno fuzzy model, the fuzzy rule consequents of which are presented in the form of functional dependencies, was not chosen by chance. Since this model allows you to form an expert opinion based on the numerical rating points given to the student during the course.
Figure 7 schematically shows the organization of the fuzzy rule calculations when using the proposed method. Every fuzzy rule of rule block = { 1, 2, … , } as input parameters = { 1, 2, … , } in the antecedent are accepts the evaluation for each lesson (topic), where m is number of lessons.
The block of rules for fuzzy inference is drawn up by an expert teacher and can take into account the nonlinear dependencies of a student's knowledge for individual lessons (topics). This may take into account the incompleteness of the student's knowledge, as well as the subjective methodology of teaching and assessing certain academic disciplines.
The linguistic nature of fuzzy mathematics, which makes it possible to operate with qualitative quantities, makes it possible to introduce fuzzy characteristics into the system. This helps the teacher to more accurately formulate and evaluate the requirements regarding the complexity of the tasks, as well as improve the interaction of the system with students during the tests.
The introduction of a 100-point rating scale did not solve the problem of the accuracy and objectivity of the knowledge assessment system, but rather made it more fragile and vulnerable. Indeed, the verge of transition of the Satisfactory score (74 points) to the Good score (75 points) is 1 point.
Fuzzy systems can eliminate this drawback by establishing a varying degree of fuzzy transitions from one estimate to another. And also introducing additional quality indicators, such as “pretty good”, “almost satisfactory”, “not very good”, “brilliant” and others.
Sometimes a useful feature of fuzzy testing systems is the adjustment of the severity of evaluating
the results, with the ability for students to give fuzzy-logical answers when passing through the testing
procedure used in [
( 1) ( ) 1 0 1 0
…
TS Type 2 Fuzzy Rulek
̃ ( ) = 0 + 1 1 + ⋯ +
The apparatus of fuzzy logic in the design of testing systems allows you to more accurately identify gaps in student knowledge. And at the same time, given the incompleteness of answers during the tests, to identify quantitative and qualitative indicators of existing knowledge, without requiring the student to give knowingly false answers in the absence of them.
The paper considers testing systems for identifying and checking the quality of students' knowledge, operating on the basis of the fuzzy inference methods. The method of complex assessment of students' knowledge based on the Type 2 Takagi-Sugeno fuzzy model are proposed. A knowledge assessment model for fuzzy testing systems based on a four-point assessment system are proposed.