The Construction of the IRT Profiles Using Fractures that Store the Average

Abstract

IRT profiles scheme using average interpolating polygons. The article deals with the construction characteristics of the aggregate quality of tests using average interpolating linear splines. It was found that the use of splines with free node allows to build an integral characteristic quality of compilation of tests task.

Keywords

• IRT profiles of tests task
• complexity of the task
• differential ability
• characteristic curves
• splines that interpolate on average
• automatic testing

-

1.1

Introduction

Item Response Theory (IRT)

Student knowledge and skills control is one of the main elements of the learning process. The effectiveness of managing educational work and the quality of the training of specialists depends on the correct organization of the control. Through control, a "feedback" is established between the teacher and the student, which allows assessing the dynamics of learning the learning material, the actual level of knowledge, skills and abilities, and, accordingly, makes appropriate changes to the organization of the learning process. Testing is an important part of knowledge control methods. The testing system is a versatile tool for identifying students' knowledge at all stages of the learning process. In modern conditions, knowledge of testing techniques and the creation of test-bench bases is a necessary component of the teacher's work.

The use of tests as a tool for measuring knowledge implies the presence of certain quality characteristics arising from the theory of test control [3,6]. The theoretical basis for test control is the classical theory of tests and the modern theory of Item Response Theory (IRT). These theories began to emerge in the studies of the late 19th and early 20th centuries in the scientific works of F. Galton, J. Cattell, A. Binet, T. Simon, , E. Thorndike, C. Spearman, H. Gulliksen, L. Guttman, L. Crocker, J. Algina, G. Rasch, A. Birnbaum and others. The steady increase in the number of publications seeking and improving the IRT model indicates the relevance of choosing these models and their widespread use.

Classic model for profile questions (the probability of a respondent with a level of knowledge θ and a correct answer to question with the complexity no higher βj) is considered a two-parameter model of Birnbaum: e Da j i  j  P( i , j )  1  e Da j i  j  , where D=1.7 is constant, is item discrimination parameter, which determines the slope of characteristic curve. The disadvantage of the model in its practical application is its non-linear dependence on the parameters, and limited "flexibility". IRT is based on mathematical models that differ in visible function P( i , j ) . Based on these models, profiles of the complexity of the questions and the level of students' preparedness are constructed - characteristic curves. Characteristic curves of the test are the main source of information in the IRT, since all other test's scores are derived from them. Characteristic profile of the task are inherent:

1. The complexity of the task, which is determined by the student's preparedness scale at the level of probability of the correct answer P( )  0.5 . So the complexity of the task is the median distribution of the probability of the correct answer.

2. Differential ability, which shows how good the task can distinguish students of different levels of knowledge. Differential ability is estimated by the values of the lower and upper limits. The boundaries are determined by the profile: the bottom is at the level of P( )  0.25 and the upper is P( )  0.75 . This property is the level of inclination of the characteristic curve of the task in the middle part. Therefore, the higher the inclination, the better the task of the test will be able to distinguish pupils' knowledge levels.

The number of mathematical models in the IRT is constantly increasing, their review appears in scientific periodicals. The reason for this is, first of all, considerable interest in the issues of assessing the quality and reliability of tests in education, as well as the need for the most accurate, reliable and easy to use model. Justifying the disadvantages of parametric models, J. Ramsey, M. Abrahamovich, S. Winsberg, D. Thyssen and G. Weiner (J.O. Ramsay, M. Abrahamowicz, S. Winsberg, D. Thissen, H. Wainer) proposed methods for evaluating characteristic curves that based on the use of spline model [1, 2]. It should be noted that the use of interpolation splines does not always correctly reflect the real characteristics, therefore, it would be advisable to consider spline models based on other approximation models, as in works I. Shelevitsky proposed to use spline regression models [4, 5]. In this paper, as an IRT model, it is suggested to use splines that store the mean of functions, that is, interpolate on average. 2 2.1

Methodological Approach

Preliminaries

Consider the supporting results that are needed in the future.

We denote by Sr (n ) the set of all polynomial splines of the order r of the minimum defect by partition of n  0  t0  t1  ...  tn  T , that is, the set of all functions with a continuous r 1 - th derivative that coincide on each of the intervals ti1 , ti  with an algebraic polynomial of degree not higher r .

If for a continuous function x(t) such that there exist x( ) zz  0,T ,  0,1,...,(r  1) / 2 the spline sr x, n , t  Sr n  is such that for odd r

sr x, n , ti   xti , i  0,1,...,n , sr( ) x, n , z  x( ) zz  0,T ,  0,1,...,(r  1) / 2, and for paired r , if ti1/ 2  (ti  ti1) / 2, i  0,1,...,n  1 fulfills the conditions of

sr x, n , ti1/ 2   xti1/ 2 , i  0,1,...,n  1 , sr( ) x, n , z  x( ) zz  0,T ,  0,1,...,(r  2) / 2, then it is assumed that the spline interpolates the function x(t) in the nodes of the partition  n , if the equality 1 ti ~sr x,  n , t dt  1 ti xt dt hi ti  1 hi ti  1 is performed, such a spline interpolates on average or is one that preserves the average value of the function x(t) .

Theorem 1. Let x(t) be a function that integrates on R1 and X (t ) , the antiderivative x(t) is such that X (0)  0. In addition, let s( X ,t) be a spline that interacts X (t ) in nodes ti 0  t0  t1  ...  tn1  tn , that is

X (ti )  s( X , ti )i  0,1,...,n ( 1 ) Then spline s( X ,t) preserves the average value of the function x(t) on the [ti , ti1 ] interval.

Proving. Consider the relationship ti1 ti1 ti1   x(t)dt   s( X , t)dt   x(t)  s( X , t)dt   x(t)  s( X , t) t, ti , ti1 dt, ti ti ti  ( 2 ) where

1, t [a, b],  t, a, b  

0, t [a, b], Heaviside step function. Apply to ( 1 ) integration by parts b b  udv  uv ba  vdu, a a let then u(t)   t,ti ,ti1  and dv(t)  x(t)  s( X , t)dt,

du(t)   t  ti   t  ti1 dt Where  t  delta function of Dirac and

t t t v(t)   x( )  s( X , )d  x( )d   s( X , )d  X (t)  sX , t .

0 0 0 Given that ti1 ti1  x(t)dt   s( X , t)dt  X (t)  sX , t  t, ti , ti1 ti  ti1  ti ti

ti1   X (t)  sX , t  t  ti   t  ti1 dt.

ti then from condition ( 1 ) we have In addition, because X (t)  sX , t  t, ti , ti1  ttii1  X (ti1 )  sX , ti1   X (ti )  sX , ti  X (t)  sX , t  t, ti , ti1 ti

 ti1  0 . ti1  X (t)  sX , t  t  ti dt  X (ti )  sX , ti  ti and ti1  X (t)  sX , t  t  ti1 dt  X (ti1)  sX , ti1  ti then from the condition of interpolation ( 1 ) we have ti1  X (t)  sX , t  t  ti   t  ti1 dt  0.

ti ti1 ti1 Thus,  x(t)dt   s( X , t)dt, that is, spline s( X ,t) preserves the average value of ti ti the function x(t) in the interval [ti ,ti1 ] in other words, is interpolated on average.

From the results of work [7] it is not difficult to get the next result.

Theorem A. Let   2 / 7,  0.2  3 , then for an arbitrary function x  C 4 , the sequence *n n1  ti*,n in0 n1 defined by the conditions

 ti*,n x(t)  1  dt  i T  x(t)  1  dti  0,1,...,n 0  n  n 0  n  will be asymptotically optimal for interpolation parabolic splines x  s2 x, *n  2  innf x  s2 x, *n  2 1  o( 1 )  ( 3 )  n3 x  1  o( 1 ),    10  t 2 dt 1/ 2  t   1 t 2 (1  t) 2 .

24

In the case when x can equal zero to only the finite number of segments (which is quite a natural condition for many real tasks, including for the purpose of our study), the conditions for choosing nodes can be simplified ti*,n x(t)  dt  i T x(t)  dti  0,1,...,n.

0 n 0 Using the theorem and theory 1, we immediately get the following relation.

Theorem 2. Let x  C 3 be such that x can equal zero to only a finite number of segments and a sequence *n n1  ti*,n in0 n1 , defined by  ti*,n x(t)  dt  i T x(t)  dti  0,1,...,n 0 n 0 ( 4 ) and s2 ( X , *n ) as a parabolic interpolation spline for X (t) , where X (t) is the initial x(t) such that X (0)  0. Then ~s1(x, *n )  s2 ( X , *n ) is a broken (spline of a minimal defect of order 1), which preserves the mean value of the function x(t) , and the partition *n is asymptotically optimal. 2.2

Construction of IRT Spline Profiles

Let's turn to the main results of this research. Consider the process of forming respondent responses to test questions. In this case, we have two a priori unknown values that characterize the test question and the respondent, namely, the level of difficulty of the question and the level of knowledge of the respondent. If one of the parameters  or  is locked, the evaluation task P(, ) reduces to the determination of dependence on one of the parameters: P() - question profile, or P( ) - profile of the respondent. Let's assume that  and  in the experiment process (testing) remain unchanged, then one can find probability estimates associated with  and  . Assume that the result of the response of the i-th respondent to the j-th task is equal to ri,j, where ri,j=1, if the answer is correct (but we can use a weighted estimate of ri,j>0), in the opposite case, is 0.

ˆi  1 M

 ri, j ,

M j 1 and the assessment of the difficulty level of the test is equal 1 N ˆi  1   ri, j ,

N i1 where M is the number of test tasks and N is the number of respondents.

Given that a respondent with a higher level of knowledge is correct on probabilities with probability no less than a respondent with a lower level of knowledge, we have

P(i) P(k), if i>k.

This implies the non-declining nature of the P(i) dependence for a fixed level of complexity of the question  . That is, P(i, ) is the characteristic profile of the task. Proceeding from this, P(i, ) is a cumulative probability curve, each point of which corresponds to the probability that the respondent with a knowledge level not greater than  gives the correct answer to the question with the level of complexity . Thus, the estimates of the characteristic curve of the question are defined as 1 Pj ( )  ri, j  i   .

N Since we have only a set of Nˆ ratings , we obtain Nˆ empirical points 1 Pˆj ( k )  ˆ  ri, j  i   k , k  1,...,Nˆ .

N ,6 ,4 ,2 0 0 0 ,5 ,5 1 0 1

0 0 5

1 0 Fig.4.Behavior Pˆj ( k ) for various test tasks I and II.

0 5 0 1 0

Here is an algorithm for constructing an IRT spline profile, which stores an average value.

Let there be a plural Pˆj ( k ), k  1,...,Nˆ . We denote by

I Pˆj ( k )  1 k

 Pˆj ( ), k  1,...,Nˆ

Nˆ  0 the discrete analogue of the antiderivative Pˆj ( k ) .

We denote by SI Pˆj ( k ), t the parabolic spline of the minimal defect with two free nodes t1, t 2  ( 0,1 ) , such that it is determined by the conditions

SI Pˆj ( k ),0 0, SI Pˆj ( k ),0 0, and the nodes t1, t2 are determined by the conditions ( 4 ), where X  Pˆj ( k ) .

According to Theorem 2, the derivative SI Pˆj ( k ), t s a linear spline, which preserves the mean value.

The nodes of the optimal partition t1, SI Pˆj ( k ), t1 and t2 , SI Pˆj ( k ), t2  characterize the behavior of the IRT profile. The smaller the value of t2  t1 and more SI Pˆj ( k ), t2  SI Pˆj ( k ), t1  , the better the differential function of the problem, the higher the value of , the greater the complexity of the task. The ideal task meets the conditions 1t t1  t2  0.75 and SI Pˆj ( k ) , t1  0, SI Pˆj ( k ) , t2  1.   So for Pˆj ( k ), k  1,...,Nˆ we have

For the test task I (see Fig. 4-I) for the task II (see Fig. 4- ІІ)

Thus, an aggregated characteristic is obtained, by which it is possible to automate the process of assessing the quality of test tasks and to analyze the complexity of a particular test task (as in the example given in Fig. 4, task I is compiled rather qualitatively, and the task II is too simple). 3

Conclusion

Using an IRT model based on splines that interpolate on average allows to obtain an aggregate characteristic of the assessment of the quality of test tasks, which allows for automatic testing of test quality.

References

1. Abrahamowicz M.: Multicategorical spline model for item response . Psychometrika 57 , 5 - 27 ( 1992 ).

2. Winsberg , S. , Thissen , D. , & Wainer , H.: Fitting item characteristic curves with spline functions . Educational Testing Service , 84 - 52 ( 1984 ).

3. Linda , C. , and A. James , T. : Introduction to classical and modern test theory . Logos ( 2010 ).

4. Duban , R. , Shelevytskyy , I. : Spline models in test measuring knowledge . Mother School 7 -8 ( 967 - 968 ), 11 - 14 ( 2010 ).

5. Duban , R. , Shelevytskyy , I. : Comlpex issues and respondent knowledge profile spline models in knowledge testing . Management Information System and Devices 156 , 71 - 77 ( 2011 ).

6. Chelyshkova , M. : Theory and Practice of Designing Pedagogical Tests . Logos 432 ( 2002 ).

7. Shumeiko , A. : On the choice of nodes for interpolation parabolic splines . High schools Mat. 4 , 67 - 71 ( 1990 ).

8. Shumeyko , A , Knurenko , V. : Building IRT profiles by using splines that retain Average value . Mathematical modeling 1 ( 32 ), 28 - 32 ( 2015 ).

9. Shumeiko , A , Iskandarova , A. : The contruction of the IRT profiles by using piecewise linear regression whit free nodes . Modeling in industry, transport and education 11 , ( 2016 ).