The integration of human intellectual capabilities into the process of building a machine learning model is the most promising area. The advantage of this area is to effectively combine the capabilities of both human and machine through the use of visual analytics. Visual analytics combines machine learning, data transformation, and data visualization, that enables people to understand big and complex data. Using this approach, a human can form a mental model of a decision-making mechanism based on data analysis. To enable the machine to use this model, it is necessary to transform it into the form used by the machine. The paper proposes an information technology for transforming a model from the domain of human understanding into machine representation through formalization. The practical application of this technology is presented using the data classification method as an example. Data is visualized by lowering the dimension of the feature space. Using visual analytics, a human forms a classification model that is transformed into machine form through formalization. This research allows us to demonstrate the effectiveness of humanmachine interaction in the process of model building and the model transformation technique.
The use of human analytical abilities in machine learning significantly improves and expands the possibilities of the practical application of artificial intelligence. A human can understand the information content of data, relations, and structure. This circumstance is the main reason for involvement and integration into machine learning-based decision-making systems. However, the process of effective integration of a human, namely, his intellectual capabilities, is complicated from the point of view of developing information technologies that are embodied in practical tools for use. The related works section describes solutions that, in the current development of machine learning, are presented in research areas. The purpose of this paper is to efficiently transform data into various, applied areas for classification based on hyperplanes. This is justified by the fact that the effectiveness of technology largely depends on the correct conversion of data with no loss of information content. And this in turn depends on the application area. The main prerequisites for information technology and the process of obtaining decisions using a model built by a human are presented in the section on the concept of integrating the intellectual capabilities of a human on the basis of the construction of a mental model. 2
The interaction between a computer and a human is very important, and sometimes
crucial. Interaction involves the exchange of certain information, while the
presentation of this information must have the property of understanding. That is, information
must be transformed and presented in such a way that the recipient and user of this
information can understand and interpret it. The purpose of obtaining information
may be ultimate for the consumer. An example of such use is the various types of
tables, graphs, charts that are prepared for the human. Humans most fully and
comprehensively use the visual representation of information. Therefore, for a human,
various types of visualization are important. Often, the graphics that the machine
prepares are the final product for the human. For a machine, a human presents
information in the form of numbers which are forming data volumes. However, a human,
providing the machine with data, wants to get the result of working on the data in the
form of an informational presentation that is convenient for use. Based on the
calculation results, a person can change or perform any actions on data or algorithms,
providing feedback for the machine and improving the results of the machine. Thus, a
human becomes involved in the process of obtaining the necessary calculated results. A
human is cyclically involved in the computational process and becomes its necessary
part. Such a direction in the development of human-machine interaction was called
“human-in-the-loop” [
Improving the process of visual analysis allows shortening the way of cognition. In
this direction, the process of interaction between machine and a human is being
improved. Interface interaction is the main part of visual analytics, as it is the only
connecting link. This link plays a key role in the transfer of information between a human
and machine. An important issue is the quality of interaction. Since the interaction is
process-oriented, it is difficult to assess how much knowledge has expanded and, as a
consequence, to evaluate the quality of the interaction itself. In the general case, in
our opinion, a qualitative interaction should reduce the number of cyclic interactions,
for example, the number of “sensemaking loops” [
The development of the process of human – machine interaction is the knowledge
generation model for visual analytics [
This process is also human-oriented, but allows specifying the result of using visualization. The knowledge acquired by a human is the result of the use of visual analytics and can be more formalized than focusing on the process of cognition. The final product in this case is knowledge.
Effective visualization is to show as much information as possible in its simplest form. This aspect is very important because it allows expanding the use of visual analytics for analysts with different qualifications. Thus, the trends in the use of visual analytics indicate an increasing involvement of a human in the process of extracting knowledge from data and developing visual analytics workflow. 3
Like the previous approach, visual analysis is used for the end user - human.
However, it should be noted that the machine may also be the final consumer of the product
of visual analytics. In this case, the human acts as a necessary integrated part of the
system for obtaining the final product that will be used by the machine. Such areas as
Interactive machine learning [
Thus, a human makes a valuable contribution and helps the machine to achieve the
development goal, which is the creation of a model. This model, by definition in [
Two forms of the model are determined: formal, for machine use and mental [
According to the above graduation, the mimic expressions of emotions are presented as follows (Table 2). lips brows
Joy normal stretched raised
Grief normal compressed lowered
Fear open normal raised
Anger narrowed normal lowered
Delight normal stretched lowered The representation of mimic display in the context of emotional states shown in Table 2 serves as a basis for the subsequent synthesis of the model by which detection will be carried out. Empirically defined features are formally submitted as follows: – x1 – the sign of facial expressions of the eyes area; – x2 – the sign of facial expressions of the lips area; – x3 – the sign of facial expressions of the eyebrows area.
x1, x2 , x3 ∈ [
The validity of the proposed model is verified by the data obtained in the study
[
According to the images of relevant emotions from the above study, according to Table 2, the features are formed at relevant intervals. The generated input is visualized in two-dimensional space by the proposed approach (Fig. 2).
As the Fig. 2 shows, the synthesized data are grouped by emotion, which confirms the ability of the proposed model to be used to classify emotional states. Further, following the steps of the proposed approach, piecewise linear dividers for the classes that correspond to emotional states are indicated. Going further, according to the suggested approach, the hyperplane parameters are obtained. Using the obtained parameters of the hyperplanes, a solution tree was constructed for the hyperplane classification of mimic expressions of emotional states. As can be seen in Fig. 2, the classes of emotions “anger” and “grief” can be visually divided into two groups each. For the emotion of “grief”, this is explained by the fact that some of respondents in the photo have eyelids twisted ( x1 ∈ [0,0.2] ), and the rest of respondents have the eyelids in normal condition ( x1 ∈ [0.4,0.6] ) (Table 3). 4.2
The application of the proposed approach to the task of marking the
electrocardiographic (ECG) signal is considered. When analyzing the ECG signal, an important
step is to break it into gaps containing QRS complexes (Fig. 3).
For the emotion "Anger", some of the respondents in the photo have narrowed eyelids
x1 ∈ [0,0.2] , and others have expanded ones ( x1 ∈ [0.8,1] ) (Table 3).
The QRS complex is the combination of three of the graphical deflections seen on a
typical electrocardiogram (EKG or ECG) [
The Q, R, and S waves occur in rapid succession and reflect a single event and thus are usually considered together. A Q wave is any downward deflection immediately following the P wave. An R wave follows as an upward deflection, and the S wave is any downward deflection after the R wave. The T wave follows the S wave, and in some cases, an additional U wave follows the T wave.
For realization of partition the training ECG signal is divided into two classes: intervals containing QRS complexes (positive samples) and intervals without QRS complexes (negative samples), or with their partial presence (Fig. 4). The resulting intervals are reduced to one dimension (a given number of values).
Positive samples
Negative samples For the training sample, we apply the proposed approach and obtain the parameters of a hyperplane that divides the given two classes. Passing a window of a given size of an ECG signal, we form a gap and check it for belonging to one of the classes. If the gap contains a QRS complex, then we mark the signal (Fig. 5).
The training and testing of the proposed approach was carried out on the data from
[
The proposed approach has the potential to solve the problem of classification of test
tasks in the process of adaptive testing of the level of knowledge. Adaptive methods
are widespread in modern e-education, in particular to determine the level of
knowledge [
For the automatic selection of the next test task, those tasks from the initial set that have not been used in the testing process (relevant) are analyzed. The analysis is performed on many parameters of the test tasks. The following parameters include: 1. The classic parameters provided by the test task for any test algorithm – both adaptive and classic. These include: the type of test task; the number of correct answers; the level of difficulty; the number of characters in the task and the answers; maximum response time, etc. 2. The semantic parameters that are required for adaptive testing and relate to each test task with the elements of the semantic structure of the educational course. These include: current heading of educational material; a key term knowledge of which is tested; a snippet of the content of the educational material that was used to create the test assignment.
When classifying current test tasks by each parameter, some classes may not contain samples, or samples in classes may be unevenly distributed. The cumulative set of parameters forms a polycube of relevant test tasks, which must be compared with a polycube that forms irrelevant (already used) test tasks. The result is the choice of the next test task for the user, which by parameters is as different as possible from the irrelevant test tasks. This process is repeated each time the user answers the next test task. Elements of the model of the given multidimensional representation of parameters thus are updated every time.
The proposed approach to defining the boundaries of classes and determining the
need for their transformation makes it possible to automatically provide the required
interactivity and dynamism of the classification system. The set of test tasks, obtained
as a result by the user, will be balanced in their parameters, representative of the
semantic structure of the educational course and corresponding to the chosen algorithm
of adaptive testing [
The use of visual analytics must be focused on the formation of the final product in the form of a model. This direction is the most promising as it allows finding not only solutions, but creating a mechanism for making decisions. In this way, both a mental model for a human and a formal model for use by a machine are formed. The received models are the result of effective interaction between human and machine during the productive exploitation of advantages. The orientation of the use of visual analytics not to the process of cognition, but to obtaining a model allows expanding its use.
Based on his analytical abilities, a human himself determines the ability and measure of data separation based on visualization. One of the necessary circumstances is to minimize losses and identify relationships between data. The visual representation of space should not distort or reduce information connections. The use of data from diverse application areas has shown effective visualization with the right transformation. The data presented in visual form are aggregated into groups that distance themselves. This suggests the possibility of widespread use of the approach of transforming the mental model into machine-executed space.