Generation of Test Tasks in Systems of Computer
          Mathematics for Educational Purposes

                             Michael Lvov, Hanna Shmarova

                      Kherson State University, Uníversitets'ka, 27
                               73000, Kherson, Ukraine
                 lvov@ksu.ks.ua, hanna.shmarova@gmail.com



       Abstract. The study of exact sciences includes not only lectures but also the ac-
       tive forms of learning (practical classes, laboratory work and so on). Thus, it is
       necessary to control not only declarative knowledge but also procedural
       knowledge of the students (the knowledge of methods for solving problems).
       This article describes functional requirements, mathematical models, and algo-
       rithms for the development of the modules of procedural knowledge testing in
       the systems of computer mathematics for educational purposes. We present
       methods of generation of test tasks implemented in the web application “Test
       on Mathematics”.


       Keywords: Systems of computer mathematics for educational purposes, gener-
       ation of test tasks, procedural knowledge, computer software, template.


       Key Terms: Educational process, computer software.


1      Introduction

There are two approaches to solve the problem of the generation of specific test tasks:
saving a series of similar tests in the database and using algorithms for automatic
generation of tests implemented as system procedures. Each of these approaches has
its advantages and disadvantages. In the first case, it is necessary to spend a lot of
time to fill the database, and tests are repeated. In the second case, the large amount of
time is spent on the implementation of algorithms for automatic generation, but each
test is unique. The weighted approach is to:

1. Develop a single common model for each sufficiently wide class of tests, as well as
   models and algorithms for generating conditions and answers for this model.
2. Develop common CASE-technologies for description subclasses of test tasks based
   on a single common model.
3. Develop joint mechanisms for storing and calling of algorithms for generation of
   specific test tasks.
2       Test Tasks-Templates

The expression 𝐹(𝑥1 , … , 𝑥𝑚 ; 𝑎1 , … , 𝑎𝑘 ) in signature ∑𝑆𝐷 in subject domain 𝑆𝐷 is
called expression template.
   Variables 𝑥1 , … , 𝑥𝑚 are called variable templates or metavariables. The common
range of values for metavariables is set 𝑉𝑎𝑟 (𝑥𝑖 ∈ 𝑉𝑎𝑟). It follows that in each par-
ticular case, each of the variable templates represents one of the variables.
   Variables 𝑎1 , … , 𝑎𝑘 are called coefficient templates or parameters. The range of
values for parameters is numerical sets (𝑎𝑗 ∈ 𝑁𝑢𝑚). It follows that in each particular
case, each of the coefficient templates is a number.

Example 1
𝑎1 𝑥1 + 𝑎2 𝑥2 is expression template.
                                                            2
   Specific instances of this template: 2𝑎 + 3𝑥, − 𝑢 + 5𝑥 and so on. Suppose
                                                            3
𝐹(𝑥1 , … , 𝑥𝑚 ; 𝑎1 , … , 𝑎𝑘 ) is expression template and 𝑥𝑗 ∈ 𝑉𝑎𝑟, 𝑎𝑗 ∈ 𝑁𝑢𝑚, then
     𝑣 ,…,𝑣       𝑐 ,…,𝑐
𝑆𝑢𝑏𝑥11,…,𝑥𝑚
          𝑚
            𝑆𝑢𝑏𝑎11 ,…,𝑎𝑚𝑘 𝐹(𝑥1 , … , 𝑥𝑚 ; 𝑎1 , … , 𝑎𝑘 ) = 𝐹(𝑢1 , … , 𝑢𝑚 ; 𝑐1 , … , 𝑐𝑘 )
   is called specialization (a special case or instance) of 𝐹(𝑥1 , … , 𝑥𝑚 ; 𝑎1 , … , 𝑎𝑘 ).
   Expression templates define the common view of the model and common view of
the answer for test task. It follows that the basis of determining the test task 𝑇 is pair
< 𝐹𝑇𝑎𝑠𝑘 , 𝐹𝐴𝑛𝑠 >, where 𝐹𝑇𝑎𝑠𝑘 , 𝐹𝐴𝑛𝑠 are expressions templates (Fig.1), defined on
common lists of metavariables ( 𝑋𝑣𝑎𝑟 ) and parameters (𝐴𝐶𝑜𝑒𝑓 ).Then

              𝑇(𝑋𝑣𝑎𝑟 , 𝐴𝐶𝑜𝑒𝑓 ) =< 𝐹𝑇𝑎𝑠𝑘 (𝑋𝑣𝑎𝑟 , 𝐴𝐶𝑜𝑒𝑓 ), 𝐹𝐴𝑛𝑠 (𝑋𝑣𝑎𝑟 , 𝐴𝐶𝑜𝑒𝑓 ) >

  In tests on equivalent expressions transformation 𝐹𝑇𝑎𝑠𝑘 =𝑆𝐷 𝐹𝐴𝑛𝑠 , where " =𝑆𝐷 "
means semantic equality in domain 𝑆𝐷.




Fig. 1. Forms for displaying task and answer templates in the interface of web application “Test
on Mathematics”


Example 2
              𝑇(𝑥, 𝑦; 𝑚1 , 𝑛1 , 𝑚2 , 𝑛2 ) =< 𝑥 𝑚1 𝑦 𝑛1 ∙ 𝑥 𝑚2 𝑦 𝑛2 , 𝑥 𝑚1 +𝑚2 𝑦 𝑛1+𝑛2 >

   In addition to the signature of the domain system function 𝑉𝑎𝑙(𝐹) is used in the
definition of a template of test task. It is defined on the set of expressions of the sub-
ject domain. System interpreter of function 𝑉𝑎𝑙(𝐹) computes a special canonical form
of the expression. Function 𝑉𝑎𝑙(𝐹) is used in algorithms for generation and verifica-
tion of test task.


3       Descriptions of Variables and Coefficients in Samples

Set 𝑉𝑎𝑟 is common range of values for metavariables and basic set. Description of
metavariables conforms to the following rules:
       𝑉𝑎𝑟𝐷𝑒𝑠𝑐𝑟𝑖𝑝𝑡𝑖𝑜𝑛 ∷=< 𝑉𝑎𝑟𝐿𝑖𝑠𝑡 >∈< 𝑉𝑎𝑟𝑇𝑦𝑝𝑒 >|.𝑉𝑎𝑟𝐷𝑒𝑠𝑐𝑟𝑖𝑝𝑡𝑖𝑜𝑛;

                                < 𝑉𝑎𝑟𝐿𝑖st >∈< 𝑉𝑎𝑟𝑇𝑦𝑝𝑒 >
𝑉𝑎𝑟𝐿𝑖𝑠𝑡 is a list of variables separated by commas,

     𝑉𝑎𝑟𝑇𝑦𝑝𝑒 ∷= 𝑉𝑎𝑟|[𝑉𝑎𝑟𝐼𝐷. . 𝑉𝑎𝑟𝐼𝐷]|𝑉𝑎𝑟𝐼𝐷|{𝑉𝑎𝑟𝑆𝑒𝑡}|𝑉𝑎𝑟𝑇𝑦𝑝𝑒 ∪ 𝑉𝑎𝑟𝑇𝑦𝑝𝑒.

𝑉𝑎𝑟 is a basic set of variables;
𝑉𝑎𝑟𝐼𝐷 is one letter;
[𝑉𝑎𝑟𝐼𝐷. . 𝑉𝑎𝑟𝐼𝐷] is a segment with type 𝑉𝑎𝑟, initial and final value of segment are
various letters without indices, or different letters with the same index, or the
same letter with different indices (Fig.2);
{𝑉𝑎𝑟𝑆𝑒𝑡} is a set of variables separated by commas;
𝑉𝑎𝑟𝑇𝑦𝑝𝑒 ∪ 𝑉𝑎𝑟𝑇𝑦𝑝𝑒are two combining types of descriptions.
   Variables VarList (i.e., the left side of the definition) are metavariables. Variables
that are included only in 𝑉𝑎𝑟𝑇𝑦𝑝𝑒 are instances. Both metavariables and instances of
variables can be used in the template. Metavariables and instances of variables should
be denoted by different letters to avoid the complication of semantics.
   For example:
         𝑋, 𝑌 ∈ 𝑉𝑎𝑟 are metavariables are defined in the 𝑉𝑎𝑟;
         𝐴, 𝐵, 𝐶 ∈ [𝑎. . 𝑑] ∪ [𝑢. . 𝑧]      are      metavariables       take      value
             𝑎, 𝑏, 𝑐, 𝑑, 𝑢, 𝑣, 𝑤, 𝑥, 𝑦, 𝑧;
         𝐴 ∈ [𝑎. . 𝑑]; 𝐵 ∈ [𝑢. . 𝑧] are metavariables A and B have different do-
             mains;
         𝐹 = 𝑎𝑥 + 𝑏𝑦 + 𝑐; 𝑎, 𝑏, 𝑐 ∈ [𝐴. . 𝑅]. 𝑎, 𝑏, 𝑐 are metavariables, 𝑥, 𝑦 are in-
             stances of variables;
         𝑎1 , 𝑎2 , 𝑎3 , 𝑎4 , 𝑎5 , 𝑎6 ∈ {𝑢, 𝑣}; 𝑢, 𝑣 ∈ [𝑎. . 𝑧] are metavariables take
             value 𝑢 and 𝑣, that take as values small Latin letters.
   Numerical algebras 𝑁𝑎𝑡 ⊂ 𝐼𝑛𝑡 ⊂ 𝑅𝑎𝑡 are common ranges of values for parameters
and basics sets (Fig.3). Parameters with type 𝑅𝑎𝑡 are defined by the value of their
numerator and denominator:

      𝑟 ∈ 𝑅𝑎𝑡, 𝑁𝑢𝑚(𝑟) ∈ [𝑀𝑖𝑛𝑁𝑢𝑚, 𝑀𝑎𝑥𝑁𝑢𝑚], 𝐷𝑒𝑛(𝑟) ∈ [𝑀𝑖𝑛𝐷𝑒𝑛, 𝑀𝑎𝑥𝐷𝑒𝑛]

    with default constraints:

     𝑀𝑖𝑛𝑁𝑢𝑚 ≤ 𝑀𝑎𝑥𝑁𝑢𝑚, 𝑀𝑖𝑛𝐷𝑒𝑛 ≤ 𝑀𝑎𝑥𝐷𝑒𝑛, 𝑁𝑢𝑚(𝑟) ∈ 𝐼𝑛𝑡, 𝐷𝑒𝑛(𝑟) ∈ 𝑁𝑎𝑡,
                               𝐺𝐶𝐷(𝑁𝑢𝑚(𝑟), 𝐷𝑒𝑛(𝑟)) = 1.

   Descriptions for test tasks’ templates are used in a procedure for generation in-
stances of tests. In the simplest case, the procedure 𝐺𝑒𝑡𝑇𝑒𝑠𝑡 has specification:

                       𝑇𝑎𝑠𝑘𝑇𝑒𝑠𝑡 𝑃 ≔ 𝐺𝑒𝑡𝑇𝑒𝑠𝑡(𝑇𝑒𝑚𝑝𝑙𝑎𝑡𝑒𝑇𝑒𝑠𝑡 𝑇),
   where 𝑇 =< 𝐹𝑇𝑎𝑠𝑘 , 𝐹𝐴𝑛𝑠 >.




Fig. 2. Form for adding variable with a given range of values in the interface of web application
“Test on Mathematics”




Fig. 3. Form for adding parameter with a given range of values in the interface of web applica-
tion “Test on Mathematics”
Example 3. Apply the formulas of abridged multiplication:

                            𝑇 =< (𝑎 + 𝑏)2 , 𝑎2 + 2𝑎𝑏 + 𝑏 2 >

  Suppose the user describes following template for this test task:

                        𝑇1 =< (𝑋 + 𝑌)2 , 𝑋 2 + 2 ⋅ 𝑋 ⋅ 𝑌 + 𝑌 2 >,
                   𝑇2 =< (𝑎 ⋅ 𝑋 + 𝑏)2 , 𝑎2 ⋅ 𝑋 2 + 2 ⋅ 𝑎 ⋅ 𝑏 ⋅ 𝑋 + 𝑏 2 >,
            𝑇3 =< (𝑎 ⋅ 𝑋 + 𝑏 ⋅ 𝑌)2 , 𝑎2 ⋅ 𝑋 2 + 2 ⋅ 𝑎 ⋅ 𝑏 ⋅ 𝑋 ⋅ 𝑌 + 𝑏 2 ⋅ 𝑌 2 >,

   𝑇4 =< (𝑎 ⋅ 𝑋 𝑚 + 𝑏 ⋅ 𝑌 𝑛 )2 , 𝑎2⋅𝑚 ⋅ 𝑋 2⋅𝑚 + 2 ⋅ 𝑎𝑚 ⋅ 𝑏 𝑛 ⋅ 𝑋 𝑚 ⋅ 𝑌 𝑛 + 𝑏 2⋅𝑛 ⋅⋅ 𝑌 2⋅𝑛 >.

   𝑋, 𝑌 are metavariables, 𝑎, 𝑏, 𝑚, 𝑛 are parameters. Using conditions leads to a reduc-
tion of templates’ number and extension of test tasks’ classes. Thus, test task 𝑇4
describes all classes of test tasks with templates 𝑇1 , 𝑇2 , 𝑇3 in conjunction with addi-
tional conditions = 1, 𝑏 = 1, 𝑚 = 0 ∨ 𝑚 = 1, 𝑛 = 0 ∨ 𝑛 = 1. Description of the con-
ditions corresponds to the syntax:

                         𝐶𝑜𝑛𝑑𝐿𝑖𝑠𝑡 ∷= 𝐶𝑜𝑛𝑑; |𝐶𝑜𝑛𝑑, 𝐶𝑜𝑛𝑑𝐿𝑖𝑠𝑡;

         𝐶𝑜𝑛𝑑 ∷= 𝐴𝑡𝑜𝑚𝐶𝑜𝑛𝑑|𝐴𝑡𝑜𝑚𝐶𝑜𝑛𝑑 & (𝐶𝑜𝑛𝑑)|𝐴𝑡𝑜𝑚𝐶𝑜𝑛𝑑 ∨ (𝐶𝑜𝑛𝑑),

                  𝐴𝑡𝑜𝑚𝐶𝑜𝑛𝑑 ∷= 𝑉𝑎𝑟𝐼𝐷 = 𝐶𝑜𝑛𝑠𝑡|𝑉𝑎𝑟𝐼𝐷 = 𝑉𝑎𝑟𝐼𝐷.

  In this example condition has the form:

               Φ𝐶𝑜𝑛𝑑 = (𝑎 = 1, 𝑏 = 1, 𝑚 = 0 ∨ 𝑚 = 1, 𝑛 = 0 ∨ 𝑛 = 1).

  The semantics of condition description is:

 Each atomic condition divides all set of tests into two classes. First class is a class,
   where this condition is satisfied, second class is a class, where it fails (in this case
   corresponding value is randomly generated from the range of values, that are de-
   fined by parameter descriptions): 𝑉𝑎𝑟𝐼𝐷 ≔ 𝐶𝑜𝑛𝑠𝑡|𝑉𝑎𝑟𝐼𝐷 ≔ 𝑅𝑎𝑛𝑑𝑜𝑚().
 Independent conditions are listed with separation by commas. All particular cases
   are defined by all possible combinations of conditions.
 Conjunction unites independent conditions that describe the same particular case.
 Disjunction combines dependent conditions, each of which defines different indi-
   vidual cases.
   Test generation algorithm returns a single particular test in each of its call. There-
fore, the condition is reduced to canonical form

                              Φ𝐶𝑜𝑛𝑑 = Φ1 + Φ2 + . . . +Φ𝑁
   where Φ𝑗 defines one particular case. Let’s use logical variables that are corre-
sponding to atomic conditions:

     𝐴 = (𝑎 = 1), 𝐵 = (𝑏 = 1), 𝑀0 = (𝑚 = 0), 𝑀1 = (𝑚 = 1), 𝑁0 = (𝑛 = 0),
                                       𝑁1 = (𝑛 = 1).

   Then various tests are described via a Zhegalkin polynomial:

                   𝐺 = (𝐴 + 1)(𝐵 + 1)(𝑀0 + 𝑀1 + 1)(𝑁0 + 𝑁1 + 1)
    Each monomial describes one particular test. It is necessary to write the polynomi-
al in standard form 𝐺 for describing Φ𝐶𝑜𝑛𝑑 in canonical form:

          𝐺 = 𝐴𝐵𝑀0 𝑁0 + 𝐴𝐵𝑀0 𝑁1 + . . . + 𝐴 + 𝐵 + 𝑀0 + 𝑀1 + 𝑁0 + 𝑁1 + 1.

   Presentation of tests’ set for given template via polynomial 𝐺 is used in the classi-
fication of tests according to their computational complexity. The complexity of the
test is determined by a degree of the monomial (the number of boolean variables).
The smaller is the degree of monomial, the more complex is a case of the test. In giv-
en example, the simplest tests are tests 𝐴𝐵𝑀0 𝑁0 , 𝐴𝐵𝑀0 𝑁1 , 𝐴𝐵𝑀1 𝑁0 , 𝐴𝐵𝑀1 𝑁1 . The
most difficult test is test with all randomly generated parameters.
   Semantics for the 𝐺𝑒𝑡𝑇𝑒𝑠𝑡 procedure:
1. Randomly substitute one of the variables that are listed in the description of the
   metavariables into the 𝑇 template instead of each metavariables 𝑥𝑗 (Fig.4).
2. Randomly substitute one of the values that are listed in the description of the pa-
   rameters into the T template instead of each parameters 𝑎𝑗 (Fig.5).
3. Calculate the function 𝑉𝑎𝑙 in the resulting instance (Fig.6).
4. Return the result of calculations in the form 𝑃 =< 𝐹𝑇𝑎𝑠𝑘 , 𝐹𝐴𝑛𝑠 >.




Fig. 4. Table for displaying the description of the variables and their generated values in the
interface of web application “Test on Mathematics”




Fig. 5. Table for displaying the description of the parameters and their generated values in the
interface of web application “Test on Mathematics”
Fig. 6. An example of a specifically generated test task based on template in the interface of
web application “Test on Mathematics”


4      Conclusion

In this article, we describe functional requirements, mathematical models, and algo-
rithms for the development of the modules of procedural knowledge testing in the
systems of computer mathematics for educational purposes.
    Using of generation of test tasks saves time because there is no need to spend a lot
of time to fill a database with conditions and answers for each task. The generator
provides a large number of similar tasks based on a single template and each instance
of the test is unique.
    The testing system based on the results of this research is implemented in the web
application “Test on Mathematics”. The system of computer mathematics for educa-
tional purposes “Test on Mathematics” will reduce the burden on teachers via auto-
matic generation of the tests and verification of the answers. Thus, it is easy to control
procedural knowledge during the absence of student (during his illness, during quar-
antine, etc.).


References
 1. Lvov M.S. Proektirovanie logicheskogo vyvoda kak poshagovogo reshenija
    zadach v matematicheskih sistemah uchebnogo naznachenija / M. S. Lvov // Uprav-
    ljajushhie sistemy i mashiny. – 2008. –No1. – S.25–32.
 2. Lvov M.S. Koncepcija proghramnoji systemy pidtrymky matematychnoji di-
    jaljnosti./ M.S.Lvov // Komp'juterno-orijentovani systemy navchannja: Zb. nauk.
    praci/ K.:NPU im. M.P.Draghomanova.– Vyp. 7.–2003.– S.36–48.
 3. Lvov M.S. Koncepcija, arhitektura i funkcional'nost' gibkoj raspredelennoj pro-
    grammnoj sredy uchebnogo naznachenija dlja srednej shkoly. Rabochee mestometodista
    /M. Lvov //Upravljajushhie sistemy i mashiny. –2009. –No 6. –S. 71-78.
 4. Lvov M.S. Raspredelennye programmnye sredy uchebnogo naznachenija.
    Podsistema upravlenija uchebnym processom / M.S. Lvov // Upravljajushhie sistemy
    i mashiny. – 2010. –No1. – S. 66 – 71.
 5. Lvov M.S. Matematicheskie testy v sistemah komp'juternoj matematiki
    uchebnogo naznachenija. / M.S.Lvov // Upravljajushhie sistemy i mashiny. –2011. –
    No6. – S.60–67
6. Lvov M.S. Matematychni modeli ta metody pidtrymky khodu rozv'jazannja
   navchaljnykh zadach z analitychnoji gheometriji/ M.S.Lvov // Iskustvennyj intel-
   lekt. –No1. – 2010. –C. 86–92.