=Paper=
{{Paper
|id=Vol-2105/10000357
|storemode=property
|title=The Construction of the IRT Profiles Using Fractures that Store the
Average
|pdfUrl=https://ceur-ws.org/Vol-2105/10000357.pdf
|volume=Vol-2105
|authors=Oleksandr Shumeyko,Anastasiia Iskandarova-Mala
|dblpUrl=https://dblp.org/rec/conf/icteri/ShumeykoI18
}}
==The Construction of the IRT Profiles Using Fractures that Store the
Average==
https://ceur-ws.org/Vol-2105/10000357.pdf
The Construction of the IRT Profiles Using Fractures
that Store the Average
Shumeyko Oleksandr1 and Iskandarova-Mala Anastasiia2
1 Dniprovsk State Technical University, Dniprobudivska 2, Kamensk, 51900, Ukraine
shumeiko_a@ukr.net
2 Dniprovsk State Technical University, Dniprobudivska 2, Kamensk, 51900, Ukraine
anastasia.iskandarova@gmail.com
Abstract. IRT profiles scheme using average interpolating polygons. The arti-
cle deals with the construction characteristics of the aggregate quality of tests
using average interpolating linear splines. It was found that the use of splines
with free node allows to build an integral characteristic quality of compilation
of tests task.
Keywords: IRT profiles of tests task, complexity of the task, differential abil-
ity, characteristic curves, splines that interpolate on average, automatic testing.
1 Introduction
1.1 Item Response Theory (IRT)
Student knowledge and skills control is one of the main elements of the learning pro-
cess. The effectiveness of managing educational work and the quality of the training
of specialists depends on the correct organization of the control. Through control, a
"feedback" is established between the teacher and the student, which allows assessing
the dynamics of learning the learning material, the actual level of knowledge, skills
and abilities, and, accordingly, makes appropriate changes to the organization of the
learning process. Testing is an important part of knowledge control methods. The
testing system is a versatile tool for identifying students' knowledge at all stages of
the learning process. In modern conditions, knowledge of testing techniques and the
creation of test-bench bases is a necessary component of the teacher's work.
The use of tests as a tool for measuring knowledge implies the presence of certain
quality characteristics arising from the theory of test control [3,6]. The theoretical
basis for test control is the classical theory of tests and the modern theory of Item
Response Theory (IRT). These theories began to emerge in the studies of the late 19th
and early 20th centuries in the scientific works of F. Galton, J. Cattell, A. Binet, T.
Simon, , E. Thorndike, C. Spearman, H. Gulliksen, L. Guttman, L. Crocker, J. Algina,
G. Rasch, A. Birnbaum and others. The steady increase in the number of publications
seeking and improving the IRT model indicates the relevance of choosing these mod-
els and their widespread use.
Classic model for profile questions (the probability of a respondent with a level of
knowledge θ and a correct answer to question with the complexity no higher βj) is
considered a two-parameter model of Birnbaum:
Da
e j i j
P( i , j ) Da
,
1 e j i j
where D=1.7 is constant, is item discrimination parameter, which determines the
slope of characteristic curve. The disadvantage of the model in its practical applica-
tion is its non-linear dependence on the parameters, and limited "flexibility". IRT is
based on mathematical models that differ in visible function P(i , j ) . Based on these
models, profiles of the complexity of the questions and the level of students' prepar-
edness are constructed - characteristic curves. Characteristic curves of the test are the
main source of information in the IRT, since all other test's scores are derived from
them. Characteristic profile of the task are inherent:
1. The complexity of the task, which is determined by the student's preparedness
scale at the level of probability of the correct answer P( ) 0.5 . So the complexity
of the task is the median distribution of the probability of the correct answer.
2. Differential ability, which shows how good the task can distinguish students of
different levels of knowledge. Differential ability is estimated by the values of the lower
and upper limits. The boundaries are determined by the profile: the bottom is at the
level of P( ) 0.25 and the upper is P( ) 0.75 . This property is the level of
inclination of the characteristic curve of the task in the middle part. Therefore, the
higher the inclination, the better the task of the test will be able to distinguish pupils'
knowledge levels.
Fig.1. Characteristic curve tasks with the same differential ability, but with different levels of
complexity
Fig.2. Characteristic curves of tasks with the same level of difficulty, but with
Fig. 3. Characteristic of the task with ideal differential ability
The number of mathematical models in the IRT is constantly increasing, their re-
view appears in scientific periodicals. The reason for this is, first of all, considerable
interest in the issues of assessing the quality and reliability of tests in education, as well
as the need for the most accurate, reliable and easy to use model. Justifying the disad-
vantages of parametric models, J. Ramsey, M. Abrahamovich, S. Winsberg, D.
Thyssen and G. Weiner (J.O. Ramsay, M. Abrahamowicz, S. Winsberg, D. Thissen, H.
Wainer) proposed methods for evaluating characteristic curves that based on the use of
spline model [1, 2]. It should be noted that the use of interpolation splines does not
always correctly reflect the real characteristics, therefore, it would be advisable to
consider spline models based on other approximation models, as in works I. Shelevitsky
proposed to use spline regression models [4, 5]. In this paper, as an IRT model, it is
suggested to use splines that store the mean of functions, that is, interpolate on average.
2 Methodological Approach
2.1 Preliminaries
Consider the supporting results that are needed in the future.
We denote by S r ( n ) the set of all polynomial splines of the order r of the
minimum defect by partition of n 0 t0 t1 ... t n T , that is, the set of all
functions with a continuous r 1 - th derivative that coincide on each of the inter-
vals t i 1 , t i with an algebraic polynomial of degree not higher r .
If for a continuous function x(t ) such that there exist
x ( ) z z 0,T , 0,1,...,(r 1) / 2 the spline sr x, n , t S r n is such that for
odd r
sr x, n , ti xti , i 0,1,...,n ,
s r x, n , z x z z 0, T , 0,1,...,(r 1) / 2, and for paired
( ) ( )
r, if
ti 1/ 2 (ti ti 1 ) / 2, i 0,1,...,n 1 fulfills the conditions of
sr x, n , ti 1/ 2 xti 1/ 2 , i 0,1,...,n 1 ,
sr x, n , z x z z 0, T , 0,1,...,(r 2) / 2, then it is assumed that the spline
( ) ( )
interpolates the function x(t ) in the nodes of the partition n , if the equality
t t
1 i ~
h t
rs x ,
n
,t dt
1 i
h t
xt dt
i i 1 i i 1
is performed, such a spline interpolates on average or is one that preserves the av-
erage value of the function x(t ) .
Theorem 1. Let x(t ) be a function that integrates on R1 and X (t ) , the antideriva-
tive x(t ) is such that X (0) 0. In addition, let s( X , t ) be a spline that interacts
X (t ) in nodes t i 0 t0 t1 ... t n1 t n , that is
X (ti ) s( X , ti )i 0,1,...,n (1)
Then spline s ( X , t ) preserves the average value of the function x(t ) on the
[t i , t i 1 ] interval.
Proving. Consider the relationship
ti 1 ti 1 ti 1
ti
x(t )dt s( X , t )dt
ti
x(t ) s( X , t )dt x(t ) s( X , t ) t, ti , ti1 dt ,
ti
(2)
where
1, t [a, b],
t , a, b
0, t [a, b],
Heaviside step function. Apply to (1) integration by parts
b b
udv uv a vdu,
b
a a
let
u (t ) t , ti , ti 1 and dv (t ) x(t ) s( X , t ) dt ,
then
du (t ) t ti t ti1 dt
Where t delta function of Dirac and
t t t
v(t ) x( ) s( X , )d x( )d s( X , )d X (t ) s X , t .
0 0 0
Then
t i 1 t i 1
x(t )dt s( X , t )dt X (t ) s X , t t, ti , ti1 t
i 1 t
i
ti ti
ti 1
X (t ) s X , t t t i t t i 1 dt.
ti
Given that
X (t ) s X , t t, ti , ti1 tt i 1
i
X (ti 1 ) s X , ti 1 X (ti ) s X , ti
then from condition (1) we have
X (t ) s X , t t, ti , ti1 tt i 1
i
0.
In addition, because
ti 1
X (t ) s X , t t t dt X (t ) s X , t
ti
i i i
and
ti 1
X (t ) s X , t t t dt X (t ) s X , t
ti
i 1 i 1 i 1
then from the condition of interpolation (1) we have
ti 1
X (t ) s X , t t t t t dt 0.
ti
i i 1
ti 1 ti 1
Thus, x(t )dt s( X , t )dt , that is, spline s( X , t ) preserves the average value of
ti ti
the function x(t ) in the interval [t i , t i 1 ] in other words, is interpolated on average.
From the results of work [7] it is not difficult to get the next result.
Theorem A. Let 2 / 7, 0.2 3 , then for an arbitrary function x C 4 , the
n
sequence *n n1 t i*,n i 0 n1 defined by the conditions
ti*, n T
1 i 1
0 x(t ) n dt n 0 x(t ) n dt i 0,1,...,n (3)
will be asymptotically optimal for interpolation parabolic splines
x s 2 x, *n inf x s x, 1 o(1)
2 n
2
*
n
2
1/ 2
1
x 1 o(1) , t dt
2
,
n 3
0
t
1 2
t (1 t ) 2 .
24
In the case when x can equal zero to only the finite number of segments (which
is quite a natural condition for many real tasks, including for the purpose of our
study), the conditions for choosing nodes can be simplified
ti*, n T
x (t ) dt i 0,1,...,n .
i
0
x (t ) dt
n0
Using the theorem and theory 1, we immediately get the following relation.
Theorem 2. Let x C 3 be such that x can equal zero to only a finite number of
segments and a sequence *n n1 t i*,n i 0 n1 , defined by
n
t i*, n T
x(t ) dt i 0,1,..., n
i
0
x(t ) dt
n0 (4)
and s2 ( X , *n ) as a parabolic interpolation spline for X (t ) , where X (t ) is the initial
x(t ) such that X (0) 0. Then ~ s ( x, * ) s ( X , * ) is a broken (spline of a minimal
1 n 2 n
defect of order 1), which preserves the mean value of the function x(t ) , and the parti-
tion *n is asymptotically optimal.
2.2 Construction of IRT Spline Profiles
Let's turn to the main results of this research. Consider the process of forming re-
spondent responses to test questions. In this case, we have two a priori unknown val-
ues that characterize the test question and the respondent, namely, the level of diffi-
culty of the question and the level of knowledge of the respondent. If one of the pa-
rameters or is locked, the evaluation task P(,) reduces to the determination of
dependence on one of the parameters: P() - question profile, or P() - profile of the
respondent. Let's assume that and in the experiment process (testing) remain un-
changed, then one can find probability estimates associated with and . Assume that
the result of the response of the i-th respondent to the j-th task is equal to ri,j, where
ri,j=1, if the answer is correct (but we can use a weighted estimate of ri,j>0), in the
opposite case, is 0.
1 M
ˆi ri, j ,
M j 1
and the assessment of the difficulty level of the test is equal
1 N
ˆi 1 ri , j ,
N i 1
where M is the number of test tasks and N is the number of respondents.
Given that a respondent with a higher level of knowledge is correct on probabilities
with probability no less than a respondent with a lower level of knowledge, we have
P(i) P(k), if i>k.
This implies the non-declining nature of the P(i) dependence for a fixed level of
complexity of the question . That is, P(i, ) is the characteristic profile of the task.
Proceeding from this, P(i,) is a cumulative probability curve, each point of which
corresponds to the probability that the respondent with a knowledge level not greater
than gives the correct answer to the question with the level of complexity . Thus,
the estimates of the characteristic curve of the question are defined as
Pj ( ) ri , j i .
1
N
Since we have only a set of N̂ ratings , we obtain N̂ empirical points
Pˆ j ( k ) ri , j i k , k 1,..., Nˆ .
1
Nˆ
0 1
,6 ,5
0 1
,4
0
0 0 5 1 ,5 0 5 1
,2 0 0
Fig.4.Behavior Pˆ j ( k ) for various test tasks I and II.
0
0
Here is an algorithm for constructing an IRT spline profile, which stores an aver-
age value.
Let there be a plural Pˆj ( k ), k 1,..., Nˆ . We denote by
N1ˆ Pˆ ( ), k 1,..., Nˆ
k
I Pˆ j ( k ) j
0
the discrete analogue of the antiderivative Pˆ j ( k ) .
We denote by S I Pˆ j ( k ) , t the parabolic spline of the minimal defect with two
free nodes t 1 , t 2 (0,1) , such that it is determined by the conditions
S I Pˆ ( ) ,0 0, S I Pˆ ( ) ,0 0,
j k j k
and the nodes t1 , t 2 are determined by the conditions (4), where X Pˆj ( k ) .
According to Theorem 2, the derivative S I Pˆj ( k ) , t s a linear spline, which pre-
serves the mean value.
The nodes of the optimal partition t1 , S I Pˆj ( k ) , t1 and t , SI Pˆ ( ), t
2 j k 2
characterize the behavior of the IRT profile. The smaller the value of t 2 t1 and
more S I Pˆ j ( k ) , t2 S I Pˆ j ( k ) , t1 , the better the differential function of the
problem, the higher the value of , the greater the complexity of the task. The ideal task
meets the conditions 1t t1 t 2 0.75 and S I Pˆj ( k ) , t1 0, S I Pˆj ( k ) , t2 1.
So for Pˆj ( k ), k 1,..., Nˆ we have
For the test task I (see Fig. 4-I) for the task II (see Fig. 4- ІІ)
Thus, an aggregated characteristic is obtained, by which it is possible to automate
the process of assessing the quality of test tasks and to analyze the complexity of a
particular test task (as in the example given in Fig. 4, task I is compiled rather qualita-
tively, and the task II is too simple).
3 Conclusion
Using an IRT model based on splines that interpolate on average allows to obtain an
aggregate characteristic of the assessment of the quality of test tasks, which allows for
automatic testing of test quality.
References
1. Abrahamowicz M.: Multicategorical spline model for item response. Psychometrika 57, 5-
27 (1992).
2. Winsberg, S., Thissen, D., & Wainer, H.: Fitting item characteristic curves with spline
functions. Educational Testing Service, 84-52 (1984).
3. Linda, C., and A. James, T.: Introduction to classical and modern test theory. Logos
(2010).
4. Duban, R., Shelevytskyy, I.: Spline models in test measuring knowledge. Mother School
7-8 (967-968), 11-14 (2010).
5. Duban, R., Shelevytskyy, I.: Comlpex issues and respondent knowledge profile spline
models in knowledge testing. Management Information System and Devices 156, 71-77
(2011).
6. Chelyshkova, M.: Theory and Practice of Designing Pedagogical Tests. Logos 432 (2002).
7. Shumeiko, A.: On the choice of nodes for interpolation parabolic splines. High schools
Mat. 4, 67-71 (1990).
8. Shumeyko, A, Knurenko, V.: Building IRT profiles by using splines that retain Average
value. Mathematical modeling 1 (32), 28-32 (2015).
9. Shumeiko, A, Iskandarova, A.: The contruction of the IRT profiles by using piecewise lin-
ear regression whit free nodes. Modeling in industry, transport and education 11, (2016).