On the basis of the developed facet dynamic adaptive testssimulators, there were obtained protocols of educational actions numerical evaluation on facet classification of English language tenses, for a random sample of 150 students. Testees are taught the facet classification of puzzles with sentences in English in a randomized puzzle electronic problem environment, in the conditions of numerical reinforcements of the actions of a testee: + 1 correct; - 1 incorrect actions. A self-consistent change in the relative frequency of reinforcements helps to adapt and increase the level of autonomy of testees. The qualification approach to the dynamic evaluation of the educational activity protocols made it possible to specify three groups of testees: those who have achieved the state of autonomous educational activity; approaching the state of autonomous educational activities; with insufficient training in foreign languages or with "trained" helplessness to bilingual activities * The research was carried out within the state assignment of Ministry of Science and Higher Education of the Russian Federation (theme No. FSRZ-2020-0011). Supported by the Krasnoyarsk Territory Science Fund, grant No. 12/19.
The problem of the qualimetry of the dynamic evaluation of the procedural
characteristics of educational activities is topical and still not completely solved [
As stated in work [
Despite the complexity and hardness of computer simulations of mediator
activities, electronic systems for dynamic assessment of the learning process are
developed in various subject areas [8; 9]. The most successful is the attempt of
computerized dynamic assessment development of teaching a second foreign
language [
In our study, we used the ideas of reinforcement learning [
Learning of facet classification, based on ideas of reinforcement learning, is fundamentally different from the traditional approach of learning with a teacher. Feedback on reinforcement learning is evaluative. That is, the actions of the student receive reinforcements in the form of a numerical assessment. The evaluative nature of feedback determines the search activity of trainees in an electronic problem environment. Receiving reinforcements from the problem environment for his actions, the testee must learn to perform tasks. The complexity of the problem environment is determined by the changing uncertainty of tasks and reinforcements. A testee adapts to the problem environment in the course of educational activity when learning how to solve problems based on the experience of his interaction with the problem environment. In the process of adapting a student to a problematic environment, autonomous educational activity is formed.
In the process of learning how to solve problems, two components of the
educational activity of the subjects can be distinguished: the first is the study of the
problem states space; the second is the application of the obtained information about
the task state space. As a result, two distinctive characteristics of the interaction of the
testee with the electronic problem environment are determined - the ability to conduct
search by trial and error and delayed incentives [
The learning with evaluation feedback includes the interaction of the student with the problem environment, which contains uncertainty, undefined factors. The testee searches for a solution to the problem, despite the presence of uncertainty in the training environment. He makes the decision to choose the action himself. At the same time, he can evaluate the nature of the progress towards the desired goal, based on a direct perception of the current state of the task. The student uses the accumulated experience to improve his characteristics over time. The knowledge that students bring to the task at the beginning of the process of its solving has an impact on how useful or simple the process of self-teaching will be. But the decisive factor that ensures the adjustment of the student's behavior taking into account the specific features of the problem being solved is his interaction with the problem environment based on evaluative feedback.
According to the law of influence of Thorndike [
For self-management of educational activities, it is important for a student to have
alternative options for finding solutions to problems and multitude of actions
(operations). Since the information, according to the definition of Quastler H, " is a
random and memorable choice of one option from several possible and equal ones,"
the student should have the opportunity to independently make decisions about
choosing an option from the formed set [
In dynamic evaluation computer systems, conditions must be created so that the
testee can show creative independence in converting the original information into new
messages. The factor of semiotic diversity is a prerequisite for adaptation of the
subject to educational activities in electronic problem environments. Semiotic
diversity allows the subject to realize the potential of learning, develops the ability to
operate with numerical and sign symbols, as well as formal structures, structures of
relations and connections [
As noted above, learning with reinforcements in dynamic learning potential
assessment systems is based on the fact that the testee attempts to maximize the
remuneration received by operating in an electronic problem environment with a high
level of uncertainty. Uncertainty of problem environment is defined by changing
conditions of solving problems, introduction of elements of randomness of parameters
of problem environment and problems, as well as setting of many alternative options
for selection of training actions and ways of finding solution of problems. Uncertainty
causes an imbalance between cognitive needs and cognitive capabilities, initiating
bifurcation [
The qualimetry complexity of the procedural characteristics of the educational activity is due to the fact that they are usually qualitative characteristics of the ability of the student to produce and absorb new information during the training process. Search of a solution to the problems of classifying sentences in English by facets of English tenses consists in recognition of mismatch between the current and target states of solving the problem and execution of actions that reduce this mismatch to zero.
Sentences in English are located on the puzzles, which a testee must relate to facets. If the action is performed correctly, then a testee will receive positive reinforcement in the form of a numerical estimate of + 1. If the action is incorrect, then the testee receives reinforcements in the form of a numerical estimate - 1.
The qualimetric approach in the dynamic evaluation of the sentences classification process in English by English language tenses in facet dynamic adaptive testssimulators [19] is implemented by: - tracking and recording of educational activities of the trainee in real time; - recognition of the mismatch value between the current and target status of the task solution and its correction through local feedback mechanisms in the form of information about the "distance to the target" in on-line mode;
- self-consistent adjustment of relative frequency of learning actions reinforcements during the current task performance depending on relative frequency of correct actions during performance of the previous task.
The procedural characteristics diagnostics qualimetry is based on a numerical assessment of the educational actions of a testee, which play the role of reinforcements. Computer tracking of target task state search is performed in on-line mode. For example, the sequence of single rewards and fines is: 1; 1; 1; 1; -1; -1; 1; 1; 1; 1; 1; 1; -1; -1; -1; 1; 1. The total win is + 5. The total gain is equal to reducing the distance (in actions) to the target. Sequences of educational actions and reinforcements form time series of events recorded in the protocol. In zero approximation, educational actions have three numerical estimates of 1, 0. -1. The correct action reduces the mismatch between the current state and the target state, and has a numerical estimate - reward + 1. An educational activity that does not explicitly change the mismatch between the current and target status of the task (e.g., browsing, listening, etc.) has a numerical value of 0. The wrong action has a numerical rating - fine - 1. It increases the mismatch between the current and target states.
The time series of events - educational actions presented in the protocols (see Figure 1) form the qualimetric basis for the procedural characteristics of training activities. Figure 2 shows a graph of the dependence of the level number n of the educational activity independence of a testee on the task number i. The graph analysis shows that the level of independence increased monotonically to level 10, which corresponded to the autonomous activity of a testee. However, a testee was not ready for autonomous educational activities and as a result of this bifurcation, there was a regression of educational activities, and a corresponding decrease from the 10 to the 7th level of independence. The actiogram shows that when solving the problem, a testee first uses the trial and error method, and spends a lot of time on each action, but this does not give a result. Then, a testee proceeds to a systematic search, which consists in sorting out all options until there is the right action. This leads to a loss of the time pace of the task and an increase in labor intensity. A testee spent more than 300 seconds to find a solution to the problem. 3
Quantitative processing of the time series of educational actions allows making a numerical assessment of the information received by a testee from the reinforcements of his actions. By searching for a solution to the problem, the student receives after each action reinforcements that carries information I . A numerical evaluation I can be obtained using the Shannon-Claude formula I 1 Н 1 n1 log 2 nn1 n2 log n2 (1)
n n n Where
n n1 n2 - the total number of actions taken to solve the current task , n1 number of right actions, n2 number of wrong actions. In formula (1), instead of the probability of choosing the right and wrong actions, the relative frequency of the corresponding actions is used. Figure 4 shows graphs of entropy H dependence on time t.
a) b) c) Fig. 4. Dependence of entropy H of educational activity on time t for three
levels of the independence of students: low – а); middle – b); high - c) 4
The maximum entropy value is 1 and the minimum entropy value is 0. The graph in Figure 4 (a) shows that the entropy of the weak student's activities fluctuates steadily near 1. Of the sample, the proportion of such testees is about 27%. They usually use random or systematic search for the right solutions. Without flexibility of thinking and sufficient operative memory, a testee cannot solve the problems of facet classification of English language tenses without constant reliance on external information reinforcements from the problem environment. Such testees may be expected to lack foreign language education [21].
Testees who successfully solve the problems of facet classification of English language tenses, but cannot switch to autonomous activity mode, are represented by a characteristic graph of entropy dependence on time in Figure 4 (b). For such testees, the conditions of educational activity are more energetically comfortable, when it is possible to make mistakes and compensate for them with external reinforcements. The proportion of such testees in this sample is 41%.
Testees who reach zero entropy of educational activities (see Figure 4 c) have abilities for autonomous educational activities. They are characterized by thinking flexibility, developed operational and long-term memory. The percentage of such testees is 32%. They achieve the state of autonomous educational activity, moving from finding a solution by trial and error to intelligent decision-making.
Thus, the qualimetric approach to the dynamic evaluation of learning to solve the problems of facet classification of English language tenses allows you to obtain meaningful diagnostic information about the procedural characteristics of the educational activities of the testees. 19. Dyachuk, P.P., Shadrin, I.V., Peregudova, I.P.: Dynamic computer test simulator "English Language Tenses" Certificate of Registration of the Program for Computers RU 2020663077, 12.10.2020. Application No. 2020662147 dated 22. 10. 2020 20. Dyachuk, P.P., Shadrin, I.V., Kudryavtsev, V.S.: Actiogram method in control systems and diagnostics of human activity [in Russian]. B. Sb.: Systematic analysis and information technologies "SAIT-2013" works of the V. International Conference. Department of Nanotechnology and Information Technologies of the Russian Academy of Sciences and others 121-128 (2013) https://www.elibrary.ru/item.asp?id=23546242 21. Rotenberg, V.S., Arshansky, V.V.: Search activity and adaptation. Moscow: Nauka (1982)