=Paper=
{{Paper
|id=Vol-3101/Paper17
|storemode=property
|title=The use of a neural network to minimize the risks of positive assessment of academically dishonest students
|pdfUrl=https://ceur-ws.org/Vol-3101/Paper17.pdf
|volume=Vol-3101
|authors=Volodymyr Sherstjuk,Libor Dostálek,Raisa Zakharchenko,Leonid Zakharchenko,Olena Shtutsa,Kostiantyn Sokol
|dblpUrl=https://dblp.org/rec/conf/citrisk/SherstjukDZZSS21
}}
==The use of a neural network to minimize the risks of positive assessment of academically dishonest students==
https://ceur-ws.org/Vol-3101/Paper17.pdf
The Use of a Neural Network to Minimize the Risks of
Positive Assessment of Academically Dishonest Students
Volodymyr Sherstjuk1, Libor Dostálek2, Raisa Zakharchenko1, Leonid Zakharchenko1,
Olena Shtutsa1 and Kostiantyn Sokol1
1Kherson National Technical University, 24 Beryslavske shose, Kherson, 73000, Ukraine
2University of West Bohemia, Univerzitní 2732/8, Pilsen, 301 00, Czech Republic
Abstract
The issue of neural network learning and research in this area are important criteria for measuring the
technical level of research institutions, educational establishments, or enterprises. The possibilities of
using neural networks have not been exhaustively studied so far. For many years to come, they will be
a means for information technology development and will require highly qualified IT specialists. In
Ukraine, a whole set of methods for assessing academic achievements, both in full-time and part-time
studies, is being actively introduced into the educational process. Quality management is a key element
of any modern educational system which requires effective means for objective control of students’
achievements and the elimination of academic dishonesty.
The paper reviews and summarizes publications on neural network learning. It is proposed to use neural
networks to identify dishonesty cases in examinations and tests and minimize the risks of incorrect de-
termination of the attainment level of students.
Keywords 1
Neural network, convolutional neural network, classification, intelligent systems, academic dishonesty
risk minimization.
1. Introduction
One of the directions for the development of artificial intelligence is creating computer intelligent
systems capable of performing functions that are traditionally considered to be intelligent. The
tasks of classification of various types of information are of particular relevance nowadays. In
particular, trainable convolutional neural networks are used to classify images. The use of neural
networks in the tasks of determining risks in the information environment is an example of the
accuracy of decisions. Such tasks aim to obtain the optimal result by minimizing the risk of error.
Nowadays, special attention is paid to the study of issues related to neural networks in assessing
the level of knowledge online. Quarantine is not a reason to stop the learning process, it should
CITRisk’2021: 2nd International Workshop on Computational & Information Technologies for Risk-Informed Systems, September
16–17, 2021, Kherson, Ukraine
EMAIL: vgsherstyuk@gmail.com (V.Sherstjuk); dostalek@prf.jcu.cz (L.Dostálek); zraissa2@gmail.com (R.Zakharchenko); leoni-
dzaharchenko@icloud.com (L.Zakharchenko); shtutsaelena79@gmail.com (O.Shtutsa); kostya13sokol@gmail.com (K.Sokol)
ORCID: 0000-0002-9096-2582 (V.Sherstjuk); 0000-0002-1613-2644 (L.Dostálek); 0000-0003-4650-3095 (R.Zakharchenko); 0000-
0002-0000-0065 (L.Zakharchenko); 0000-0001-8817-3800 (O.Shtutsa); 0000-0001-5155-7202 (K.Sokol).
© 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
not be interrupted. Ukrainian teachers were recommended to switch to the online learning format.
When assessing the academic achievements of students, teachers constantly need to confront the
dishonesty of some of them. In recent years, high-tech deception has supplanted ordinary cheat-
ing. Students are becoming more technologically advanced, and online tutorial videos detailing
cunning methods of cheating appear on the Internet almost daily.
In order to minimize the risks of academic dishonesty in determining the attainment level of
students, it is necessary to apply intelligent systems.
2. Literature review
The solution to the problem of neural network learning is reflected in the results of the following
studies.
The work [1] describes deep neural network learning, which is complicated by the fact that the
distribution of the input data of each layer changes in the learning process, since the parameters of
the previous layers change. It is noted that this slows down learning, requiring a lower learning
rate and careful initialization of parameters, and makes it difficult to train models with saturating
nonlinearities. The authors call this phenomenon an internal covariate shift and solve the problem
by normalizing the input data.
The paper [2] describes a method for continuous learning of a neural network. The network
grows similarly to a tree to accommodate new classes of data without losing the ability to identify
previously trained classes. The authors propose a hierarchical deep neural network with CNN.
The work [3] provides an analysis of the Viola-Jones algorithm for solving the problem of
object detection in a static image.
The paper [4] presents a new threading architecture for running QNN on FPGAs. The proposed
architecture scales allow taking advantage of multiple FPGA systems better than alternatives. It
describes the implementation of support for gated connections that are used in modern networks.
It is shown that the proposed architecture allows adding such connections almost free of charge.
In [5], the authors describe the max pooling, i.e. sample-based discretization process. The goal
is to reduce the sampling rate of the input representation (image, output matrix of the hidden layer,
etc.). This is carried out to facilitate over-fitting by providing an abstracted form of representation.
Moreover, it reduces computational costs by reducing the number of parameters to learn and pro-
vides basic translation invariance for the internal representation.
Max pooling is done by applying a max filter to (usually) non-overlapping subregions of the
initial representation.
The paper [6] provides a brief overview of the achievements in the field of deep learning (DL),
starting with the deep neural network (DNN). A convolutional neural network (CNN), a recurrent
neural network (RNN), including long short-term memory (LSTM) and gated recurrent units
(GRU), autoencoders (AE), and deep belief networks (DBN) are considered.
In [7] the methods and algorithms of neural network learning are described. It is also proposed
to organize the step-by-step learning of a neural network, based on adaptive and genetic methods,
which has been successfully applied to the convolutional neural network to solve the problem of
object classification in the image.
Papers [8-9] give an overview of the current state and prospects for the development of research
on machine intelligence and consider both classical and modern models of deep learning.
In [10-12] the values of academic integrity in the educational sphere are described.
For further development of the theory of neural network learning and based on the above
analysis of the literature sources, it is proposed to use its capabilities and minimize the risks of
positive assessment of academically dishonest students in determining their attainment level.
Achieving the desired accuracy will be possible by creating several models that differ in param-
eters, characteristics, assessment quality, etc.
3. Problem statement
The pandemic has become a challenge for the entire education system in Ukraine. Distance
learning systems make it possible to track student achievements by creating online courses or
virtual classes accessible at any time and anywhere in the world where the Internet is available.
The main advantage of distance learning over full-time training is, first of all, its convenience. The
disadvantages include the occurrence of academic cheating, which is common in our country.
There exist the following ways of academic cheating: passing the procedures of knowledge
assessment by fictitious persons; submission or representation of works containing a result of
educational or scientific activity with the same content by different persons; writing someone else's
variants of tasks at control events; use of a system of hidden signals (sound, gesture, etc.) during
group knowledge assessment, tests, etc. providing for the same variants; receiving unauthorized
assistance during the performance of those tasks that involve independent work [10].
The main purpose of this study is to create a system that would detect manifestations of dis-
honesty in the assessment of academic achievements. When organizing distance learning, the de-
vice that provides contact between the student and the teacher is a web camera. Images transmitted
from the camera can be classified in terms of academic integrity, analyzing the emotions, language,
and body language of the student. For minimizing the risk of academic dishonesty in exams, it is
possible to use video to monitor their behavior. When people exhibit dishonesty, they are in a
stressful, uncomfortable state. To calm themselves, they begin to change positions, perform vari-
ous movements, etc. Emotions are subjective. It is hard to comment on audio. Specially trained
staff is required to listen to, analyze, and comment on entire audio recordings. Further, these com-
ments have to be assessed by a lot of other people because the judgments are subjective. It is very
difficult to collect data. Therefore, to detect academic cheating, it is proposed to use the capabilities
of the neural network aimed at classifying images concerning academic integrity. There are several
criteria for assessing the quality of this work. Most criteria are based on the so-called prediction
matrix, denoted by сij, the diagonal containing the number of correct predictions [1, 2]. Assuming
that 𝑡𝑡𝑖𝑖 = ∑𝑘𝑘𝑗𝑗=1 𝐶𝐶𝑖𝑖𝑖𝑖 is a set of training samples for the class і, the most generalized quality criterion
is the accuracy expressed by the following formula:
∑𝑘𝑘𝑗𝑗=1 𝐶𝐶𝑖𝑖𝑖𝑖
𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎(𝑐𝑐) = 𝑘𝑘 𝜖𝜖[0,1] (1)
∑𝑗𝑗=1 𝑡𝑡𝑖𝑖
One of the problems of accuracy as a quality criterion is the deviation from the class assess-
ment. If one class is more generalized than others, then the simplest way to evaluate any other
class with a high score will be its constant classification as generalized [2, 5]. To solve this
problem, the average value of accuracy can be used. However, in addition to the classification
accuracy criterion, in practice, other quality criteria are also important, namely the speed of
assessment and analysis of new images provided to the network, delay of training time, stability,
dimension of network architecture, etc.
4. Development of a system to minimize the risk of positive assess-
ment of academically dishonest students
To detect cheating in determining the attainment level of students the following algorithm was
used:
1. Converting a frame to a black and white image.
2. Selecting a face for analysis.
3. Preparing an image for processing by the neural network.
4. Classifying student’s behavior.
The experiment involved 10 groups of students. The input was a color frame from the student's
webcam, which turned into a black and white image. Converting a frame to a black and white
image is necessary to eliminate redundant data (color images contain three components (RGB),
and black and white pictures have only one).
The facial selection was performed using the Viola-Jones method. The method is based on
sliding window technology. That is, a frame that is smaller than the original moves with a certain
step on the image, and using a cascade of weak classifiers determines whether there is a face in
the window. During image preparation, it is reformatted so that it can be used by a neural net-
work [3].
A convolutional neural network was applied for classification. It uses only a limited matrix of
small weights in the convolution operation, which is "moved" throughout the processed layer (at
the beginning - directly on the input image), forming after each shift the activation signal for the
neuron of the next layer with a similar position. Therefore, for different neurons of the source
layer, the same weight matrix is used, which is also called the convolution nucleus. It is interpreted
as a graphical encoding of any feature. The next layer, resulting from the operation of convolution
of such a weight matrix, shows the presence of the given feature in the processed layer and its
coordinates, forming a so-called feature map. Naturally, a convolutional neural network has not
only one set of weights but a whole range that encodes the elements of the image (such as lines
and arcs at different angles). In this case, such convolution cores are not laid in advance but are
formed independently by teaching the network the classical method of inverse uncertainty propa-
gation. The passage of each weight set forms its copy of the feature map making the neural network
multi-channel. It should also be noted that when searching the layer with a weight matrix, it is
usually moved not a full step, but a short distance. Thus, for example, when the dimension of the
weight matrix is 5 × 5, it is shifted by one or two neurons (pixels) instead of five, so as not to "step
over" the searched feature [4].
Figure 1: Neural network model
Max-pooling reduces the dimensionality of the formed feature maps. In this network architecture,
it is considered that information about the presence of the searched feature is more important than
accurate knowledge of its coordinates, therefore, the maximum neuron is selected from several
neighboring neurons of the feature map and it is taken as one neuron of the compacted feature map
of smaller dimensions. Due to this operation, in addition to accelerating further calculations, the
network becomes more invariant to the scale of the input image [5].
Figure 2: Pooling process
The network consists of a large number of layers. After the initial layer (input image), the signal
passes through a series of convolution layers, in which the actual convolution and pooling
alternate. The alternation of layers allows compiling "feature maps", on each subsequent layer the
map decreases in size, but the number of channels increases. In practice, this means the ability to
recognize complex hierarchies of features. Normally, after passing several layers, the feature map
degenerates into a vector or even a scalar, but hundreds of such feature maps emerge. At the output
of the convolution layers of the network, several layers of the neural network (perceptron) are
additionally installed, the final feature maps being input [6].
Figure 3: Layers of convolutional neural network
To create a system that will control academic integrity, it is necessary that the neural network not
only determines the type of emotion through facial expressions but also considers atypical,
unnatural human behavior, to the fact that a student is hiding something. For this task, it is
proposed to use a role-playing intellectual game, Mafia, to train the neural network. According to
the rules, before the start, the participants of the game randomly receive cards that determine their
roles: six Civilians and one Sheriff form a peace team, one Mafia Don, and two ordinary Mafia
form a mafia team. The goal of the peace team is to find and expel the mafia. Civilians do not have
any additional information, do not know any roles other than their own, so their behavior is frank,
they say only what they see. The Mafia knows each other and all Civilians except the Sheriff, who
sometimes forces them to deliberately lie about the role of another player, as their task is to pretend
civilians, being no more than six. It is this feature that makes people playing for the Mafia behave
unnaturally, they are forced to hide additional information they have for the benefit of the game.
The same goes for the Sheriff, who opens one role every game night, but he is already forced to
hide from the Mafia. This behavior is similar to the behavior of students who use academically
dishonest methods to perform tasks in the exam.
To train the network, it is proposed to choose several sets of Mafia games, that are broadcasted
on the Internet on open platforms, such as Twitch and YouTube. To increase the accuracy of the
experiment, only tournament games are taken for analysis, where all players feel increased tension
and responsibility, which simulates the conditions of passing an important test or exam. Such
games are sufficiently available in the following sources: https://www.youtube.com/channel/
UCZGeFNcc4oVpDPBDlojAplg;https://www.twitch.tv/playmafia;https://www.twitch.tv/mafpro
fi.
This collection is constantly updated, and since the neural network uses individual frames, the
information obtained will be more than enough to create data sets. The set of network training
games will reveal all the data about the roles of players (see Fig. 4) and the system will be to
establish for itself the typical patterns of behavior of players of different teams. In another, training,
set, the system itself will determine the roles of each player (see Fig. 5). For less load on the
system, a classification will be used that divides the image into 2 types: those that show natural
and unnatural manifestations.
Figure 4: Example of a game window with open roles
Figure 5: Example of a game window with hidden roles
5. Experiment and Results
To carry out the experiment, a software product was developed in an integrated Python environ-
ment using standard Keras library methods. To achieve the desired accuracy in a neural network
learning for video recognition of students' normal behavior and that showing signs of academic
dishonesty in the exams, two models were created that differed in the input data.
Characteristics of the parameters of the first model can be seen below (see Fig. 6).
Figure 6: Characteristics of the model parameters
The model created from the very beginning had a very large number of parameters and did not
provide a high performance. The evaluation of the model results for the fulfillment of the condi-
tions given in the task can be checked by the model prediction function (see Fig. 7):
Figure 7: Performance of the model
To obtain a high result, it is proposed to change not only the number of parameters but also to
change certain parameters of the layers to minimize the risk of neural network overlearning. In the
process of learning, the weights of the neural network were adjusted so that the network converted
the inputs to the desired outputs, respectively, between the dependencies found in the learning
data.
The model prediction function, which uses test data for verification, has shown that the model
is able to establish that the image belongs to the appropriate category. The model can distinguish
them, use them in forecasting, and therefore completely performs the task set.
To achieve the best results, a model was created where the input data is taken only from
tournament games. When plotting graphs, it can be seen how changing the characteristics of the
created models affected the change in the quality of the prediction.
The model created from the beginning had the following form (see Fig. 8).
Figure 8: Characteristics of the first model
It is seen that the determination of a large number of parameters does not guarantee high
performance (see Fig. 9).
The second model that was used had more than 5 times fewer parameters. This reduced the risk
of overlearning but certainly did not help eliminate it.
It was suggested to change certain parameters of the layers to correct the results. To do this, an
additional Dropout layer was added, and the L2-regularization method was used [7-9].
Moreover, it was proposed to use the method of data expansion to create additional images
from existing ones and to expand the working sample. Theoretically, this should have helped the
model learn better.
The second model had the following form (see Fig. 10).
Figure 9: Accuracy and loss graphs for the first model
Figure 10: Characteristics of the second model
The developed software product is guided by a certain logic in its operation. First, the user specifies
the folders where the training and test data sets are located. This is necessary if it is required to
retrain the model again to see all the learning outcomes. Fig. 11 shows the learning process.
Figure 11: Learning process of the model
The methods used helped to get better results. As in the case of the first model, the result of the
classification accuracy on the training set is about 93%, but the result of checking test data without
significant shifts is close to 86.5%, which is an excellent performance indicator.
Figure 12: Accuracy and loss graphs for the second model
The program shows the loss and accuracy in each learning epoch. Herewith, the accuracy for
training and test data is differentiated. This helps to understand whether the model is moving in
the right direction when learning.
The results of training data are presented in Fig. 13.
As a result of the experiment, images were selected with chosen individuals who were cheating
during the exam. It is important to note that, although the image in terms of logic may belong to
several classes at the same time, test and training sets contain only one image ratio - class.
The accuracy of the model is quite high, but errors are possible. However, this is a fairly
accurate classification that will minimize the risk of positive evaluation of academically dishonest
students.
Figure 13: Classification of images
6. Conclusions
The research of neural networks is one of the most promising areas today since in the future they
will be used almost everywhere, including various fields of science and technology, as they can
greatly facilitate work and sometimes secure people.
The paper considers the problems of academic dishonesty in the educational sphere. In our
country, the issue of combatting academic dishonesty is urgent. This work aims to use modern
information technology to minimize the risk of positive assessment of academically dishonest
students. As it is known, academic cheating is the behavior of students including the use of
unauthorized materials, information, or other aids for fraudulent personal purposes in the course
of performing educational tasks. An intelligent information system was developed to counteract
large-scale academic dishonesty. A convolutional neural network was created. As a database for
neural network learning, it was proposed to use sets of Mafia games that are broadcasted on the
Internet on open platforms, such as Twitch and YouTube.
The performance of the neural network is estimated to be satisfactory. According to the results
of the work processes, graphs showing the software performance evolution were presented. In the
learning process, the weights of the neural network were adjusted so that the network converted
the inputs to the desired outputs, respectively, between the dependencies found in the training data.
To improve the performance for the second model, the layer parameters were changed, an
additional Dropout layer was added and the L2-regularization method was applied. The data
extension method was used to create additional images from existing ones and the working sample
was expanded. To increase the accuracy of the experiment, only tournament games were taken
into account. The methods used helped improve the results. Overlearning was almost negligible.
Detection of cheating in groups of students, who participated in the experiment, showed an
excellent result - the accuracy of the model accounted for almost 87 percent.
Upon completing the experiment, we counted the number of cases where the network detected
academic dishonesty accompanied by positive assessment. The proportion of such cases present
the risk of positive assessment of academically dishonest students. Experiments on academic
integrity were performed independently of the assessment. The obtained results did not affect
the actual assessment of the students.
7. References
[1] S.Ioffe, C.Szegedy, Batch normalization: Accelerating deep network training by reducing
internal covariate shift, arXiv preprint arXiv:1502.03167, 2015.
https://arxiv.org/abs/1502.03167
[2] T.Xiao, J.Zhang: et al., Error-driven incremental learning in deep convolutional neural
network for large-scale image classification, in International Conference on Multimedia, No.
22, 2014, pp. 177–186
[3] А.Tymchuk, Viola-Jones facial recognition method, 2015.
https://oxozle.com/2015/04/11/metod-raspoznavaniya-lic-violy-dzhonsa-viola-jones/
[4] C.Baskin, L.Natan, M.Avi, E.Zheltonozhskii, Streaming Architecture for Large-Scale
Quantized Neural Networks on an FPGA-Based Dataflow Platform, 2017
[5] Max pooling/pooling., 2018. https://computersciencewiki.org/index.php/Max-
pooling_/_Pooling
[6] Md.Z.Alom, T.Taha., C.Yakopcic, S.Westberg, P.Sidike, M.Nasrin, M.Hasan, B.Essen,
A.Awwal, V.Asari, A State-of-the-Art Survey on Deep Learning Theory and Architectures,
Electronics. 8. 292, 2019. https://doi.org/10.3390/electronics8030292
[7] C.C.Aggarwal, Neural Networks and Deep Learning, A Textbook. Springer International
Publishing AG, 2018. DOI:10.1007/978-3-319-94463-0 ISBN 978-3-319-94462-3
[8] S.А.Shumskiy, Machine intelligence. Essays on Machine Learning and Artificial Intelligence
Theory. М., RIOR, 2019. DOI:10.29039/02011-1
[9] A.Gibson, J.Patterson, Deep Learning. O’Reilly Media, Inc., 2017.
https://www.safaribooksonline.com/library/view/deep-learning/9781491924570/
[10] T. Fishman, 2012. The Fundamental Values of Academic Integrity (2nd edition).
International Center for Academic Integrity, Clemson University.
http://www.academicintegrity.org/icai/assets/AUD_Integrity_Quotes.pdf
[11] M.Zharikova and V.Sherstiuk, Situation diagnosis based on the spatially-distributed dynamic
disaster risk assessment, 2019 IEEE 14th Intern. Conf. CSIT, 2019, pp. 205-209
[12] Z.Říhová, L.Dostálek, Information Management - the Basis for Fulfillment of People's
Information Needs, 2021, 11th International Conference on Advanced Computer Information
Technologies (ACIT), 2021, pp. 469-472. doi: 10.1109/ACIT52158.2021.9548386