The paper presents an approach to building a system for testing procedural physical knowledge, i.e. knowledge of basic physical laws and the ability to use them. This approach consists of constructing mathematical models for each academic module in a physics course. The main constructive objects are test templates, which are mathematical models of test tasks, based on physical models of systems, processes and phenomena. The template of the class of physical tests for checking knowledge of the physical laws and abilities of transformations of a physical system is represented by a set of geometric drawings, diagrams, graphs of functional dependencies, a system of formulas for transforming physical values, templates of scenarios for changing the states of a physical system and a response template. Each such template can be used both in generating similarity algorithms for specific multiple tests, and in algorithms for automatically checking the correctness of answers. The proposed method allows describing a relatively simple class of specific tests. An important feature of the system is the ability to automatically check not only the final answer, but also the parameters of the intermediate states of the physical system. The implementation of a procedural physical knowledge testing system can be performed by creating software interactive multimedia objects using the methods of computer mathematics and algebraic programming technology.
The teaching process of the exact sciences in secondary school, especially physics,
includes both the lecture part of the lesson and the active teaching forms: practical
works, double-acting works, independent and test papers, etc. Thus, it is necessary to
control not only the declarative knowledge, but also procedural knowledge, that is,
knowledge of methods for solving physical problems. General methods of
constructing of computer mathematics systems for educational purposes, one of the subsystems
of which is the testing environment, are described in [
Physical tests, which will be discussed, are designed to control procedural knowledge of a physics course using the example of a physics course in the 7th-9th grades of the secondary school in Ukraine. Technologies for the control of procedural knowledge have not yet been studied and developed. Thus, the problem of research is relevant.
In [
In this paper, within the framework of this general approach, the features of the construction of physical knowledge testing systems are considered. It is assumed that a testing system containing procedural physical tests will be implemented as a module Physical Knowledge Testing Environment, which can be used both in computer-aided educational mathematics systems (CAEMS) in physics and for other purposes.
The problem of the present work can be formulated as a study of the functional requirements, mathematical models and algorithms for constructing of a system for testing procedural physical knowledge in CAEMS.
Separately, it can be highlighted the task of designing and creating software
environment testing of physical knowledge and skills. In this case, the actual need is to
develop a software module for physical knowledge testing and skills in distance
learning systems (DLS) [
The widespread use of distance learning in educational institutions in secondary
and higher education was increased demands on the quality of distance learning. One
of the main tasks is to improve the quality of distance courses, especially training
modules of a practical orientation. There are testing systems, simulators, laboratory
and practical works. The problem is that the international standards do not clearly
spell out the specifications for the structure and implementation of such training
modules. At the same time, an essential requirement for testing systems is the requirement
of compliance with existing international standards of distance learning, which
contributes to the integration of learning information resources. The problem of
standardization of methods and technologies for developing a system of physical knowledge
and skills testing is debatable and requires discussion of a wide range of specialists. In
this paper, we consider methods for solving the problem of modeling and
implementing of a module for physical knowledge and skills testing in distance courses based on
adaptive (laboratory) and template tests technology, which is consistent with
international standards IMS and SCORM [
The relevance of solving the problem of designing of a module for physical
knowledge and skills testing in distance courses is determined by the fact that the use
of test types based on choice currently allows only knowledge control [
Examples of formal models of physical processes The subject area (ontology) is represented by structural-logical schemes (SLS). These ontologies are represented by a three-level hierarchy:
“Discipline” – “Educational Module” – “Model of Phenomenon, Process”.
Our approach is the system of test tasks should be unified within the framework of
the discipline “Physics 7-9” [
PHYSICS 7 – PHYSICS 8 – PHYSICS 9 – ….
Before proposing a general definition of the pharma model of the physical process, we consider several examples. 2.1
Example 1. Ohm's law for subcircuit.
From the program Physics 7-9: Ohm’s law for subcircuit. Series and parallel connection of conductors.
The testing system should verify the application of the following physical laws: 1. The current strength I in the circuit is determined by the Ohm’s law formula: I =
U/R 2. With a series connection of resistors, their total resistance is equal to the sum of their resistances: R = R1 + R2 + R3 + … 3. With parallel connection of resistors, their total conductivity (inverse of resistance) is equal to the sum of their conductivities: 1/R = 1/R1 + 1/R2 + ...
Test task. The given scheme (Fig. 1)
R1
B A
R2 R3
C A UAD
D designation of connection point 1, designation of connection point 2; resistance model (resistor); current strength model; voltage model. model of the element (primitive) of the electrical circuit (EC), circuit model as a connection of elements.
Models of resistance, current strength, voltage are the models of physical quantities. A model of a physical quantity is a five (name, unit of measure, range of values, step of change, value).
The model of the EC element (electrical circuit) is a data set:
The EC model is defined as follows: for circuit elements σ1, σ2, … σk the scheme ∑ is a formula for the variables σ1, σ2, … σk in the signature <+, || >, where “+” is the sign of the serial connection operation, “||” is the sign of the parallel connection operation with consistent notations of beginnings and ends. For example, the formula of the circuit in Figure 1 is ∑ = σ1 + (σ2 || σ3). Here, the primitives are the following EC elements: σ1 = (A1, B1, R1, I1, U1), σ2 = (A2, B2, R2, I2, U2), σ3 = (A3, B3, R3, I3, U3), (1)
In the circuit model it is natural to distinguish subcircuits. In general, the relation “circuit – subcircuit” is determined through one (any) of the circuit construction operations: if F(x1, …, xn) is a circuit construction operation, and ∑1, …, ∑n is a circuits, then ∑ = F(∑1, …, ∑n) is a constructed circuit and ∑, ∑j are in a “circuit-subcircuit” relationship. In this example, three primitive subcircuits are defined:
σ1 = <_R1_>, σ2 = <_R2_>, σ3 = <_R3_>, as well as constructions ∑1 = <_ σ2 || σ3_>, ∑ = <_ σ1 + ∑1_> (2) (3)
For each subcircuit, as well as for primitive subcircuits, variables R, I, U (1) are defined, and also meta variables – the points of connection of subcircuits into circuits. In (2), (3) these points are marked with an underscore. So any scheme ∑ = (A∑, B∑, R∑, I∑, U∑). Connection points of subcircuits are denoted by letters – the values of metavariables. The data that is included in the sections Given and Find in Test task, is displayed using these symbols. The notation can be simplified, but we do not give a simplification mechanism (see Fig. 1).
Test generation method: The scheme construction mechanism is called as the test design template.
A set of subcircuits (for example, (2) and (3)) is called as instance of a test template.
The test task generation method is carried out in three stages (Fig. 2): 1. The test task generation begins with the use of a construction template to obtain a specific instance of the test template.
2. The construction of a set of test template instances can be done pre-bye (see the Difficulty of the test task)
3. The test task is generated from the test template instance by selecting the variable sections Given and Find, as well as randomly generating the values of variables of the section Given.
1 N
1 N
The test task generation is now carried out by selecting the test template instance schema, randomly selecting variables and their values for the Given section, as well as randomly selecting the names of other variables for the Find section.
Test’s Complexity. The complexity of the model and the test, including the level of computational complexity, is determined by the number of characters of operations in the formula of the test template instance. For the scheme of this level of complexity, the complexity of the actual test task is also determined based on the number of quantities in the Find section.
The presented example should be considered rather complicated, since the formula contains two signs of operations.
To limit the complexity, it is proposed to limit the maximum complexity of the circuit. In the example, this maximum can be schemes with three operation signs.
For each difficulty level, it is easy to write out all the schemes (test template instances) of a given difficulty level and a test exercise for this model can now be defined as a set of test tasks with a given distribution of difficulty levels. 2.2
Example 2. Mixing liquids’ model.
The elementary process of mixing liquids can be described as follows: two vessels contain liquids, the physical characteristics of which are given by the heat capacities c, the temperatures t of the masses m. It is required to determine the physical characteristics of their mixture contained in the third vessel (Fig. 3).
As in the first example, the formal model of a physical quantity is a five (name, unit, range of values, step of change, value).
In the definition of a fluid mixing model, formal models of such physical quantities are used: heat capacity, temperature, mass, heat.
Formal fluid model: heat capacity c, temperature t.
The formal model of a vessel with a liquid is an element of the scheme: a model of a liquid, mass m.
The formal model of mixing liquids uses formal models of vessels with liquids (c1, t1, m1), (c2, t2, m2), (c3, t3, m3).
The testing system should verify the application of the following physical laws: where m1 = … c1 = … t1 = … m2 = … c2 = … t2 = …
m = m1 + m2, Q = Q1 + Q2 (heat balance equation),
Q1 = c1m1(t1 – t), Q2 = c2m2(t2 – t)
In addition, for this particular example, the complexity of the test task is influenced by the fact whether two identical or two different liquids mix with different heat capacities. As in example 1, the complexity of the test task is determined by the complexity of the construction template, the amount of data in the Find section, taking into account the number of different mixable liquids. 3 3.1 3.1.1
Formal model of physical knowledge testing system
is represented by a hierarchy of dependence of the subject (training) modules (structural-logical scheme of the discipline).
In the training module, a list of the intrinsic physical quantities used in it is defined (Fig. 4). Each physical quantity is defined by a five: (name, basic physical dimension, type of value, range of values, step of change)
Relationship dependence determines the sequence of study of relevant topics of the training course (Fig. 5).
In addition to the attributes described in 4.1.1, the training module is represented by a set of formal models of physical phenomena and processes (Fig. 6).
Training module Name, List of models of physical quantities
…
The formal model of a physical phenomenon, a process The formal model of a physical phenomenon, a process is a scheme (formula) in the signature of the operations of constructing schemes, a list of types of elements of the scheme, a set of interpreters of operations for constructing a scheme (Fig. 7).
Physical value is defined as the five attributes: (Name, type of value, physical dimensionality, range of values, step of change) The element of the scheme (primitive scheme) is a set of physical quantities. Thus, if is an element of the, and a1,…,al are physical quantities, then = (a1,…,al). The schema element has its own type: Type() = (type(a1),…, type(al)). If x1,…,xl – variables are physical values, then a structured variable = (x1,…,xl) is a scheme element.
Scheme construction operations. Each scheme is composed of scheme elements and other schemes using construction operations. The set of names and arity of construction operations is called the schema signature: OPΣ = < 1( 1), … , ( ) >.
The construction operations are, generally speaking, multi-sorted. If = ( 1, … , ) ∈ OPΣ, then the types of variables , 1, … , belong TPΣ and, generally speaking, are different. The types of operations for constructing schemes are determined by the types of variables y.
Test schemes (templates) are constructed from the elements of schemes and other schemes using construction operations: = ( 1, … , ) ∈. Thus, the test template Σ is the formula in the signature OPΣ. All the arguments of the schema Σ are the variables — the schema elements. Thus, the template (formula scheme) has the form Σ(x1,…,xl).
The complexity of the scheme is the number of symbols of functions in the formula Σ. Since the value of any function for a given variable requires computation, the concept of the computational complexity of a scheme is closely related to the concept of the complexity of a scheme.
The test task is obtained as a schema specification with the values of some subset of the set (x1,…,xl). The test task has the form Σ(a1,…,ak,xk+1,…,xl). Values a1,…,ak form Given section of the test task. In the simple case, Find section contains the names of one or more variables from the set (xk+1,…,xl).
The test template interpreter is a software function that calculates the value of a scheme given the values of its arguments. Let (x1,…,xl) be an any element of the scheme. Variables x1,…,xl are related by relationships Ф(x1,…,xl) – physical laws whose knowledge is being tested. Therefore, all values (xk+1,…,xl) of the Find set variables can be calculated by values a1,…,ak. Thus, for any type scheme f(Σ1,…, Σl), its value y = Value(f(Σ1,…, Σl)) can be calculated from bottom to top: a1 = Value(Σ1),…,al = Value(Σl), y = Int f(a1,…,al), where Intf is the interpreter of signature operation f.
Thus, the interpreter of the scheme calculates not only the values of the variables of the set (xk+1,…,xl), but also all the physical values of any subscheme of the test template.
Thus, the number of test case variables includes not only variables x1,…,xl, but also all intermediate variables of a form y = f(x1,…,xl) for any subschem of a type f(Σ1,…, Σl). 4 4.1
Methods for generating and validating test items In principle, there are two approaches to solving the problem of choosing a test pattern. First, for each model of the physical process, the phenomena are created manually by the table of templates (see tab.2), which are stored in the database. With this approach, together with the table of templates, it makes sense to keep a table of the distribution of probabilities for choosing this template. Secondly, it is possible to implement an algorithm for automatically generating circuits of a given complexity. In this case, first with a given probability, the level of complexity is selected, and then a circuit of this complexity is generated. Each of these approaches has its own advantages and disadvantages. In the first case, you need to spend a lot of time filling the database with template tables. In the second case, time is spent on the implementation of an algorithm for automatically generating a circuit of a given complexity.
Another technological challenge is to implement a graphic image of the physical process, the phenomenon. The test window contains the fields Picture, Given, Find. The Picture field displays the scheme in which variables are marked. Fields Given and Find are filled in at the stage of test task generation. The uniqueness of the model leads to the fact that each drawing scheme is programmed separately. Let be the test pattern template. Consider a set of schemes of a given circuit containing all elements and all composite schemes Σ = < Σ1 = 1,…, Σk = k, Σk+1 = l >. Each element of this set depends on the variables - physical values: Σ1(x11,…,x1n),…, Σl(xl1,…,xln). As already mentioned, for each subscheme Σj(xj1,…,xjn) the following property takes place: if the values of any m variables from the set (xj1,…,xjn) are known, the values of the other n – m variables can be calculated. Therefore, the process of calculating the data values of a particular test task is carried out from the bottom up to the structure of the circuit.
Our approach to growing the practical knowledge testing system is as follows: 1. On the basis of algorithmic analysis, define a specific subject area as a set of models of physical quantities, models of elements of schemes and schemes as formulas in a given signature. 2. Develop a unified general model of this class (test pattern design), as well as test pattern templates as an implementation of the test pattern design 3. Implement the algorithms for generating conditions and responses to this model pattern, thereby determining the algorithm for generating a test task and verifying it 4. Develop CASE-technology for describing subclasses of test items based on a single common model for users of the computer-aided mathematics system for educational purposes. 5. Develop common mechanisms for storing and invoking algorithms for generating specific test items. 6. Develop methods for generating and validating test items. 5
Test template selection
IMS QTI standard specifications allow you to develop a model and create software
for testing physical knowledge and skills in the DLS. For this purpose, specifications
of Adaptive Items and Item Templates [
Adaptive test items
The software module for creating and executing test tasks for testing physical
knowledge and skills can be based on the technology of developing adaptive tests.
The IMS QTI version 2.2 specification provides support for adaptive tests in distance
learning systems. In accordance with this standard, adaptive test questions may
contain software object modules that provide interactive user interaction with them. For
such questions, we introduce into consideration a new type of test question — an
object type, and tests containing object-type questions will be called adaptive
(laboratory) tests. Below is the specification and schema of the attributes of the object type
question of the adaptive (laboratory) test (Fig. 8) [
Each type of question separately has its own characteristics in the specification. This is due to differences in the parameters of these types.
The response of the test is processed in the Response Processing module. Evaluation of the answer in the module can occur in two ways: a differentiated assessment of the entire question and the accumulation of evaluation of the response options.
According to the IMS QTI Specification, when working out questions of the
adaptive test, there is a feedback with the test person, determining the response adjustment
at each stage and thus forming the variability of the response. In this question there
may be additional parameters that are not specified by the standard. An example of
the implementation of such an object question is a controlled interactive Flash
animation in which a specific task is programmed [
Question entry
Question body
Attributes question type question title adaptability depending on time used template
question text addaitional question parameters Answer variants and their identifier
Identifier of correct answers Fig. 8. The question attributes scheme of the object type of the adaptive test and its specification.
The result of answering a question of a laboratory (adaptive) type can be determined in the test passing object, taking into account the value of the maximum mark for correct passing the test, and used in automatic (software) assessment. Alternatively, the score can be determined (changed) by the tutor during the check.
Due to the lack of a separate specification of the IMS standard for laboratory (adaptive) tests, we consider the method of their simulation, which consists of the following. Two object modules are developed that are interconnected by a special format data transfer interface. The first module (the test passing object) is designed to pass a laboratory (adaptive) type test, and the second (answer display object) is intended to reflect the test passing results during the test. The scheme of interaction between the objects of the testing module and the database is shown in Fig. 9.
Test task (question) )
Object of the task Input parameters of the object for its initialization before starting the execution of the task Result (answer) as a special format string
Result (answ(er) Result (response) display
object Input parameters of the object for its initialization before the start of displaying the response or demonstrating the result
( фор format string
Database Object modules can be Java applets, COM objects (for example, Adobe Flash) or HTML 5 program objects, which upon initialization receive the values of input parameters (attributes) and are set up in a working state. For an object passing the test, such parameters may be the value of the maximum mark for correct passing the test, the value of the time limit for passing the test, other values of the object initialization parameters; for the response display object, the values of the input parameters that ensure the initial state of the object, corresponding to the final state of the test object.
Thus, the combination of two object modules forms a closed system for passing, checking and evaluating tests of the adaptive (laboratory) type. The proposed method of creating adaptive (laboratory) type tests meets the specifications of the IMS standard and allows packaging for portability in other distance learning systems. At developing of the testing module, we will understand the testing process as a sequence of calls of operators with a certain behavior in the context of a learning model.
The process of automatic test assembly in the DLS testing module is a workflow in which certain logic is implemented with the possibility of automation. It is important to begin by defining an abstract model that contains all the necessary data describing the purpose of training and explaining the correlation between objects of specific instances of this model in order to automate the compilation of test questions as a workflow.
Suppose Item Templates is an instance of a test task generated by this template with the automatic test generation process described above.
Automatic generation of Item Templates will help to significantly simplify the preparation of practical tasks, reducing the time for their creation and ensuring the unique nature of each generated task. It will also provide an opportunity to automate the evaluation of test results without the negative impact of the human factor.
We define the test as a set of learning tasks set and test settings: ST = <S(S1, …, SL), T(T1, …, TN)>, where S(S1, …, SL) – a set of test settings, defined by a user (a test compiler), and T(T1, …, TN) – a set of learning tasks. To build the abstract model of the learning task T we use its signature of following view:
T = <M, P, Q>, where M – a set of the task input parameters, and is described as М(X1, …, Xn), where X1, …, Xn – the task parameters; P – a set of the task conditions, described as P(P1, …, Pk) , where P1, …, Pk – task conditions; Q – the task result model, described as Q(Q1, …, Qm), where Q1, …, Qm – sets of metadata, defining the results of the task. We introduce the following software modules to describe the Item Templates object: LrnTaskTemplate (abbreviated from Learning Task Template) for T; LrnTaskModel (abbreviated from Learning Task Model) for M; LrnTaskValidator (abbreviated from Learning Task Validator) for P; LrnTaskModel for Q.
The abstract generator model is a generalized model of math tests. The scheme of interrelations of the objects described above is shown in Figure 10.
According to this scheme, ISmartTestModule is a library of learning task generator
templates for compiling a test task. The template library of the instructional task
generator module is a program class (LrnTaskTemplate), which contains all the functions
necessary for building learning task instances, validating and solving them using a
named model. The functions of the task generator template use the common
(TestSettings) and local (LrnTaskTemplateSettings) user-defined settings (the test compiler)
as input parameters. These settings can provide the template with parameters for
generating and checking specified learning tasks in the form of: instructions for using
preferred question types [
The proposed approach to the process of physical knowledge and skills testing can be used to build a model and develop software for a testing module in a distance learning system in accordance with the requirements of international standards IMS and SCORM.