Career guidance testing assumes the presence of several types of individuals, and in the case when the output data of testing are images that characterize certain qualities of the subjects - several types (classes) of images. The image classes includes of selected searchable elements that determine whether an image belongs to a particular selected type (class). There is a ColourUnique M software module that allows you to automate the process of testing and saving test forms. The functions of the classifier are still performed by an expert (teacher or psychologist), which implies errors in evaluating the result due to individual characteristics of human perception, which can negatively affect the reliability of the classification. The paper considers two algorithms for evaluating images (the made test forms), one of which is a neural network, and the second is a filtering algorithm with hard - defined areas for determining the desired elements. During the implementing of these algorithms, a number of problems arose. The classifier is created in order to improve the accuracy of classification, both in comparison with expert assessment and with the first experimental data obtained. For achieve of the most reliable classification results, the authors consider the possibility of implementing a hybrid classifier for career guidance tasks.
Today, there are many different methods for
recognizing the desired objects in an image. The choice is
determined by the features of the object and the goals of
the recognition process set by the developer. Properties of
the desired object are often set without strict
mathematical parameters. In this case, you need to
formulate the properties of the desired object (or objects)
and develop a stable method for its (their) detection. To
solve this problem, it is necessary to find, generalize and
formulate empirical observations in mathematical terms.
In other words, formalize the parameters of the desired
object [
When the entire object classes are searched for, where parameter sets differ, or not all parameters are found in all classes, formalization leads to a decrease in the accuracy of the estimation. And if we are talking about such an area of activity as career guidance, then the error with determining the object class will be critical.
This paper offers a solution to the problem of inaccuracy in defining one of the classes in the process of implementing a neural network algorithm. the authors develop an alternative filtering algorithm with the possibility of implementing a hybrid classifier. The goal is to consider both algorithms and compare their effectiveness. based on the obtained data, we can create a combined method for evaluating and classifying images, which contains a neural network algorithm and a filtering algorithm.
The classification of individuals proposed for
automation is similar to the Holland classification [
Since the F type is the only one that uses the «cut» tool when working with the test form (Fig. 1), it is determined already in the process of working with the test form and does not pass processing through the neural network algorithm and filtering algorithm. This leaves 5 types of individuals to classify.
The other A, B, C, D, and E types differ mainly in the
distribution of dark and light pixels within the planigon
[
At the moment, the object of research and testing of algorithms are scans of the made test forms to avoid distortion of cells (Fig. 2). For the user, the test form has the form of a quasi-space (the effect of perceptual distortion), but in reality it is a matrix.
The A type characterized by a predominant (50% or more, respectively 100% – pure type) presence of elements «horizontal lines» (Fig. 3). Lines (from 3 cells in length) can be either the colors of exactly the same coordinates, or the colors of different coordinates, but related to the same tone, for example, «green», as in Fig.3c. assumed that there will be fewer bands of colors from different coordinates, but there will be less than one tone.
Another condition for the presence of the A type is a small number of shades, namely-no more than one tone with a number of shades of 5 or more.
There are the following signs of dominant A type: 1. 50% or more of the scan is filled with the desired «horizontal line» elements (from three cells of the same color or tone);
2. A small number of shades, namely no more than one tone with a number of shades of 5 or more; 3. The fewer shades applied by the recipient, the more pronounced type considered.
The B type characterized by the presence (3 elements per scan or more) of the «vertical lines» elements (from three cells in height) with identical coordinates (Fig. 4c, 4d). The required elements have at least 3 cells in height, they can be up to 6, respectively, the definition area can reach 6x1 (Fig. 4e).
The total number of shades can be very different, the main thing – 4 times or more the presence of an element from 3 cells in height of exactly the same color (Fig. 4e).
There are the following signs of dominant B type: 1. The presence of 3 or more elements of the «vertical lines».
The C type characterized by the presence of the elements «horizontal lines» (Fig. 5c) and «wide» or «narrow» gradients in the definition areas 3×3 and 2×3 (for wide) and 3×1,4×1, 5×1, 6×1 (for narrow ones) (Fig. 5d), the line elements (from 3 cells in length) can consist of colors of exactly the same coordinates, or colors of different coordinates, but related to the same tone (Fig. 4d, 4e, 5d, 5e).
Tone – is a range of colors that are indicated in the color circle as «yellow», «red», «orange», and so on. It is also the tone that gives the color its name.
In the current selection, the A type elements are mostly represented by colors of the same coordinates.it is
The «wide gradients» – it’s a bands of colors with exactly the same coordinates in width and colors of the same tone range in length, for example, green and orange gradients in the definition areas 3×3 and 2×3 in figure 5d.
The total number of shades may vary, but there are generally 4-5 shades for 2-3 tones. For example, in Fig. 5 d, the scan shows from 3 shades for red, green, purple and orange tones. There are the following signs of dominant C type: 1. Simultaneous presence of elements of «wide» or «narrow» gradients and «lines»;
2. The minimum number of elements for determining the type of 2 lines and one wide gradient 3×3 and 2×3, or 2 lines and two narrow gradients; 3. Relatively high variety and number of shades.
The D type characterized by the presence of elements «chess» or « chess-like» (Fig. 6c, Fig. 7). Chess and chess-like elements are arranged by a special arrangement of dark and light cells in the area of definition 2×2. Moreover, chess elements can contain any color.
The total number of shades is usually small.
There are the following signs of dominant D type: 1. The presence of «chess» elements on 50 or more percent of the scan area;
2. A large number of cells with repeated colors of the same coordinates, especially black, white, yellow, red and blue.
3. Using a relatively larger number of colors from the «basic» section of the ColourUnique M software module (Fig. 8) compared to other types.
The E type is mainly characterized by the presence of a large number of shades of the selected colors, namely – from 5 shades for at least two tones (Fig. 9). Аs a rule, there are much more of them. Some representatives of this type are distinguished by the presence of mainly «narrow» gradients, less often «wide» (Fig. 5e, Fig. 9с).
There are the following signs of dominant E type: 1. The presence of a large number of shades, namely – from 5 shades for at least two tones;
2. The predominant presence of «narrow», sometimes «wide» gradients. the total data is cut off (15% goes for validation, 15% for testing). There are 3 folders with data for training, validation and testing in each folder, images are divided into classes (types).
Previously untrained neural networks were used to
classify individual types using the «Associative color
space» © testing method. Three architectures were used:
MobileNet [
Training process. The training consists of periods. At the moment, the most accurate network is Inception_v3, showing a result of about 65%, trained on 40 epochs. The number of epochs is set manually based on the results. However, too many epochs can lead to retraining (when the network recognizes local features rather than General ones, for example, it is linked to the color, presence or absence of a particular detail, etc.).
During the training process, the network counts the error at each step, then changes the weights to reduce the number of errors. For this purpose, the stochastic gradient descent (SGD) optimizer is used, since at the moment, when working with the network for career guidance, it has proved to be the most effective.
Both
statistical estimation
and
machine learning
consider the problem of minimizing an objective function
that has the form of a sum:
where the parameter w that minimizes Q(w) should be
evaluated [
The neural network diagram for classifying images (ready-made test forms) is shown in Look at the graphs of accuracy and loss on training and validation data (Fig. 11). As can be seen from the graph in Fig. 11a, there was a discrepancy in accuracy in validation and training around the 12th epoch.
The network began to retrain and Fig. 11b shows that the network is being it.
To improve the network performance and further experiments with sample expansion, you will need to change the optimizer, create new classification layers, and possibly reduce the step of weight changes, since this form of the validation curve may indicate that the values change too much at each epoch.
Validation process. The training sample is divided into iterations. The network looks at a certain number of images per iteration and selects weights. Then it evaluates the error. Then, with the same weights, the network passes to the validation sample (15% of the data from the training sample) and applies them to images in the validation sample.
Reducing the error both in training and validation indicates the success of network training. If the error in training falls, but increases in validation, it means that there is a re-training.
Testing process. The networks are tested on data that they haven't seen before (15% of the training sample). In other words, the network does not know which class is on a particular image and tries to determine it independently. The received responses are collected and the percentage of correct definition is output.
Library. The Keras library [
Augmentation process. For the first test of the neural network, images were used, divided by type (class). A-1, A-2... A-70; B-1, B-2 ... B-70, and so on up to E-70. The total amount of: 350 images. Due to the fact that there were not enough images for training, it was decided to apply augmentation. Due to the specifics of the task, some augmentation methods, such as stretching, rotations by degrees other than 180, warping, fnd so on were not used.
The methods of augmentation, used in the problem, presented on Fig. 12, there are: 1. 180° rotation (for such images, 180 is added to the title); 2. Vertical reflection (added to the top_bottom name for such images); 3. Horizontal reflection (added to the left_right name for such images).
The results. Consider the performance of the neural network for each of the desired types (classes) of images at the end of the first experiment. In the experimental test sample, the network showed the highest accuracy in classes B, C, E, and the lowest-D (Fig. 13).
In this sample, the network was presented with both bright and mixed samples, which contained signs of several types. First, the sample was classified as an expert, then as a neural network classifier. And, after repeated expert classification, the expert found that in some cases the correct answer was still given by the neural network (Fig. 13).
This fact is explained by individual features of human visual perception.
However, the network currently shows a low degree of confidence in the result in class D (D type).
The filtering algorithm is developed in parallel with the neural network algorithm.in the future, both classifiers will pass comparative tests and the main algorithm will be selected based on their results, while the second one will be a supporting or alternative class, where they were shown the best result.
Consider a filter developed for the desired class D, where at the moment the neural network does not classify images reliably.
The filtering process based on the principle of comparing regions 2x2 of the planigon cell with samples of «chess» and «chess-like» combinations consisting of cells of 4 conditions (Fig. 14).
For count the number of combinations, we used permutation schemes with and without repetitions. = ! ,
( + +...+ )! ,
( , , … , ) = !∙ !∙…∙ !
where n – the number of elements, Pn – the number of
permutations, and P(n1,...,nk) – the number of
permutations with multiplicities n1,...,nk [
Received: 1. 24 combinations for cases of using cells of all 4 conditions for the 2×2 definition area without repetitions; 2. 144 combination for the use cases of the cells 3 conditions with a single repeat of one condition (0, 1, 2 or 3); 3. 24 combinations for cases where 2 condition cells are used for the definition area 2×2 with a single repeat of two conditions (0, 1, 2 or 3) Finally, there is 192 combinations.
However, not all of the resulting combinations are «chess» or «chess-like». For example, in Fig. 13 it can be seen that most combinations form "bands" in areas 1×2 of the cell due to the fact that cells of the same States fall into this area.
Now, out of 12 combinations, there are 4, which means only 48 for cases of using cells of 3 States with a single repeat of one state (0, 1, 2 or 3).
Translation of an image in grayscale is due to the fact that, unlike the desired elements of other types (classes), the elements of the desired type (class) of images D can contain cells of all possible colors of the palette provided by the ColourUnique M program. Since «chess» or «chess-like» structures form alternating «dark» and «light» cells, converting the image to grayscale simplifies classification.
At the moment, the main problem with implementing
this filter is selecting the correct mode for translating
images in grayscale, since colors that have similar
saturation levels but different tones merge. The output
can be a channel translation, depending on which colors
are used in each case [
The paper explores the possibilities of different algorithms for creating a final classifier for career guidance tasks using the «Associative color space» © testing method. The efficiency of the neural network classifier in each of the classes is analyzed. The filtering algorithm is considered as an alternative or second evaluation method for creating a combined classifier, where either the most effective classifier or both will be used for each class of images.
The purpose of the work was to evaluate the first test results and analyze the experimental output data. This goal was achieved.
When performing the work, the following was performed:
1. The most efficient neural network algorithm for developing the classifier;
2. A filtering algorithm is proposed as a second component for creating a final classifier, in particular for class D images that are currently classified by the neural network with unsatisfactory accuracy.
3. The problems and errors in the implementation of the proposed algorithms were found, and ways to eliminate them were developed.
4. The results obtained can be applied to improve the accuracy of image classification and adjust the work of neural network algorithms, which will increase the accuracy of evaluating and predicting the processes of professional guidance of an individual.
The paper was performed with the support by RFBR, Grant № 19-07-00455.
Tarasova Iuliia S., PhD of Sciences degree seeking applicant, senior lecturer of Department of Industrial design, Nizhny Novgorod State University of Architecture and Civil Engineering. E-mail: tar06@list.ru
Andreev Vyacheslav V., Head of the Department «Nuclear reactors and power plants», Grand PhD of Sciences in technology, associate professor, Nizhny Novgorod state technical university n.a. R.E.Alekseev. E-mail: vyach.andreev@mail.ru
Ainbinder Roman M., PhD in Physico-mathematical sciences, associate professor of the Department «Mathematics», Nizhny Novgorod State University of Architecture and Civil Engineering; senior lecturer of Department of Information technologies in Humanities research, Lobachevsky State University of Nizhny Novgorod – National Research University. Е-mail: romain@inbox.ru
Toskin Denis V., master’s Degree student of IRIT Nizhny Novgorod state technical university n. a. R. E. Alekseev. Еmail: toskin.dv@gmail.com