=Paper=
{{Paper
|id=Vol-3057/paper14.pdf
|storemode=property
|title=ProFaMI Platform for the Safety of Certain Aspects of Distance Learning
|pdfUrl=https://ceur-ws.org/Vol-3057/paper14.pdf
|volume=Vol-3057
|authors=Aleksandr M. Kadan,Evgeniy O. Protasov
}}
==ProFaMI Platform for the Safety of Certain Aspects of Distance Learning==
https://ceur-ws.org/Vol-3057/paper14.pdf
ProFaMI Platform for the Safety of Certain Aspects of Distance
Learning
Aleksandr M. Kadan 1 and Evgeniy O. Protasov 1
1
Yanka Kupala State University of Grodno, 22 Ozheshko str, Grodno, 230023, Belarus
Abstract
The widespread introduction of distance learning systems has increased interest in such
concepts as "academic honesty", "safety of the educational process" and required the use of
somen The widespread ew technologies, among which "proctoring" stands out - the procedure
for monitoring an online exam, where the administrator - proctor observes the course. Experts
show the greatest interest in carrying out such control in an automatic mode, when the proctor
is a software system that independently verifies the personality of the subject, monitors his
behavior, and records incidents related to violation of the exam requirements on the exam
video. Artificial intelligence algorithms should play a significant role in the operation of the
automatic proctoring system. The article discusses the requirements for the organization and
implementation of a semi-automatic proctoring system using elements of synchronous and
asynchronous work to monitor the progress of a remote control training event. The system is
implemented as an application that uses technologies for remote control of the subject's
computing device and scene recognition technologies to control his workspace using intelligent
video surveillance. Particular attention is paid to the confidentiality of the data of the online
exam, the security of the personal data of the test subject, and his use of non-verbal
communication means.
Keywords 1
Proctoring, automatic proctoring, online exam, remote work, distance learning, intelligent
video surveillance
1. Introduction
An effective response to the problems of the COVID-19 pandemic, not only of educational
institutions but also of many organizations of various kinds of activity, has become a sharp increase in
interest in remote work technologies. At the same time, the procedure for remote control and assessment
of students' knowledge and competencies of employees has become very relevant, the main technology
of which is the technology of online proctoring [1, 8, 9, 21]. Online proctoring is seen as a way to
confirm compliance with the rules of the remote exam. It includes observation of subjects during control
activities and is actively used in various fields, including basic, secondary, higher, and continuing
education.
E-learning has been studied in detail in the literature from different points of view: its features and
achievements [2, 3], learning management systems [4], as well as online courses and “electronic” events
[5]. However, the conduct of “electronic assessment” [6] of student performance using electronic
monitoring tools is still very limited [7, 6, 8].
The purpose of this article is to present the capabilities of a platform for semi-automatic proctoring,
using elements of synchronous and asynchronous work to monitor the progress of certification
activities. The system is implemented as an application using the remote control of the subject's
Proceedings of VI International Scientific and Practical Conference Distance Learning Technologies (DLT–2021), September 20-22,
2021, Yalta, Crimea
EMAIL: kadan@mf.grsu.by (A. 1); protasov_eo_17@mf.grsu.by (A. 2)
ORCID: 0000-0003-3701-8100 (A. 1)
©️ 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
140
computer and scene recognition technology to control his workspace employing intelligent video
surveillance. Particular attention is paid to the confidentiality of the data of the online exam, the security
of the personal data of the test subject, and his use of non-verbal communication means.
2. Online Proctoring Technologies in Higher Education
2.1. Online Proctoring Support Tools
Traditionally, online monitoring events are divided into high and low-rate events. If events with a
low rate are actively carried out online, then information about the mass holding of events with a high
rate (exams in disciplines, the defense of diplomas, and dissertations) among foreign universities have
not yet been announced [18, 19].
Selected universities in the USA and Europe practice pre-testing or a survey based on the use of
Google forms or the Survey Service, written exams on the Gradescope web platform for submitting and
grading assignments, simple exams with proctoring on the Zoom platform, large-scale, final exams with
proctoring in Examity services, ProctorU, SmarterProctoring. Attestation of undergraduates and
postgraduates is carried out remotely with the consent to register. The most popular grading system is
pass/fail.
In the academic environment of the far abroad, the most popular proctoring platforms are Examity
[23] (allows online identification, auto-proctoring, live proctoring, integration with MOODLE),
Proctorio [24] (ensures the security of digital data, integration with other platforms, a secure browser
for testing ), Verificient Proctortrack [25] (implements a hybrid real-time model combining remote
human proctors with advanced automatic AI intervention in cases of suspicious behavior, fraud, or
student assistance), Respondus [26] (online testing modes, auto-proctoring, recognition support
persons, movements and lighting, captures actions on the keyboard, mouse movements, equipment
changes, all violations during the exam will be marked for the proctor). In the CIS space Examus [27]
(analyzes the behavior of users of any online services using face recognition and emotion detection),
ProctorEdu [28] (auto-proctoring and live proctoring, automatic assessment of trust in test results and
biometric identity verification, support for mobile devices, works in a browser and does not require the
installation of extensions, plug-ins, and third-party software), PROCTORU [29] (confirms the identity
of the tested person, monitors through a webcam, the subject is connected with a real proctor who leads
him through the process).
It should be noted that the authors are not aware of open source solutions in the field of proctoring.
All of the above platforms are paid commercial products and do not provide personal data protection
requirements.
2.2. Problems of Using Online Proctoring
At the present stage, two main problems are put forward when using online proctoring tools. First,
this is the discrepancy between the regulatory framework of the educational process and online
proctoring technologies and the lack of guarantees for a reliable assessment of knowledge when using
it [18, 19, 20]. Developing a legal framework for the application of high stakes online exams appears
to be an important challenge. The introduction of such procedures in a short time will inevitably be
accompanied by a large number of errors, both random and systematic [21]. In particular, legal
regulation of the use of proctoring data should be developed, the legal status of attestation results should
be determined for various violations of the attestation procedure, regulations for administrators' actions
in the event of technical failures (for example, due to instability of the Internet connection) should be
developed. The second is the threat of violation by the subjects of the requirements of "academic
honesty", which has always been a problem in higher education [15, 16, 17]. While it is possible to turn
a blind eye to this in examinations with a low rate and regulate the situation with available means, then
in exams with a high rate this can entail unacceptable risks [21]. Undoubtedly, the use of the Internet
and the development of user-friendly technological devices have generated concern on the part of
educators, the research interest of the academic community, and created new risks of inappropriate and
unethical behavior on the part of students [12, 13, 18, 19].
141
3. ProFaMI Platform for Secure Certification in Remote Learning
Environments
Given the steady trend towards the transition to distance learning and the unpredictability of the
situation with the end of the COVID-19 pandemic, at the Faculty of Mathematics and Informatics of
the Yanka Kupala State University of Grodno, it was decided to develop the ProFaMI (Proctoring for
Faculty of Mathematics & Informatics) platform for the safe conduct of certification activities in the
form of oral or written examinations and preliminary or control tests.
When designing and implementing the platform, special attention was paid to ensuring equal
conditions for all subjects, guarantees of independence for them to obtain results, protecting their data
and data on the progress of the certification tests.
3.1. Organizational Requirements
The most important role is assigned to the preparation of the test subject's working space. It is
required to provide a separate room, well lit, without background noise, without outside access during
the certification. The workspace should be as clutter-free as possible. Otherwise, the examiner has the
right to close the session and cancel the result. Prohibited items include food or drinks, headphones,
gadgets such as mobile phones or tablets, notes, and books, as agreed with the examiner.
Before commencing certification, the examiner must inspect the prepared room. If the requirements
are not met, the examiner has the right to both refuse to conduct certification and ask to put the
workspace in order. For added safety, the examiner is encouraged to be present at his workplace
throughout the certification process.
During the certification process, both live proctoring from the examiner's side and auto-procoting
from the platform's side are possible. The subject is prohibited from breaking the silence, getting up,
minimizing the active window on the monitor (where the attestation task is displayed), or running
extraneous programs on the computer. The examiner, in case of violation of the certification
requirements, has the right to complete the process of its conduct ahead of schedule.
3.2. Software and Hardware Requirements
It is assumed that a test subject passing certification remotely must have a computer, a webcam, and
access to the Internet. The webcam should be installed so that the table at which the subject sits, his
entire workspace, and the environment in the room are visible. The door to the room must be locked
and must be within the webcam's field of view. The dealer's hands must always be in the frame.
Throughout the entire course of certification, an application must be running on the subject's
computer that will track any activity that occurs on the computer: minimizing the application window,
opening a new tab in the browser, launching an external program. These actions are prohibited to the
test subject, and the platform will respond to them. The application can be run on platforms such as
Windows, Mac OS, Linux. At startup, the system will be analyzed for the presence of other active
programs. If such programs are found, the application will try to close them.
After the initial analysis of the system, the application will launch a browser with the only valid tab,
from where a request will be made to the resource on which the data for attestation is located.
During certification, one-way video and two-way audio communication between the subject and the
examiner should be carried out. This will allow the examiner to both observe the progress of the
certification and discuss the questions that have arisen with the examinee. The video stream of the
certification progress is recorded on the platform server and is available for further viewing and
analysis.
The subject does not see who his proctor is, and does not know if the proctor is currently seeing him.
This will provide additional reliability to the certification results. In addition, a pre-trained neural
network based on the coco-SSD model is used to assist the examiner. The task of the neural network is
to analyze the video footage received from the subject's webcams to search for security incidents of
prohibited items and actions. For example, if a phone or another person appears in the frame, the
142
platform will inform the proctor about this, at the same time adding a record of the time and nature of
the incident to the attestation progress report.
3.3. Platform Architecture and Security
The architecture of the platform and the Platform includes five components through which the
interaction of three working nodes is carried out: the server on which the attestation resources are
located and the computers of the proctor (examiner) and client (test subject). This is:
1. Application on the side of the examiner (proctor), providing live proctoring functions.
2. An application on the side of the subject, carrying out a continuous analysis of the state of the
system of his computer, the actions of the subject, and providing access to resources for passing
certification.
3. Microservice verification-API for organizing work with security tokens.
4. Microservice action-API, coordinating the requests of the subject and the proctor.
5. Microservice exam-API, to create a report on the progress of certification, which allows you to
conduct intelligent analysis of video stream frames and generate messages about security incidents.
To provide access to the resource with attestation materials only to authorized users (if there is a
running application on the subject's side), a unique bunch of three tokens is used: a program token, a
client token, and a cookie token. This helps ensure that the application is used uniquely during
attestation and prevents unauthorized people from connecting to the attestation session.
The server to which requests are sent from the subject's computer, in addition to the response to the
request itself, sends a cookie token a small piece of data (a randomly generated identifier) that is stored
on the subject's computer and is updated with each new request. Moreover, each new cookie token is
written to the database concerning the subject's identifier. The application reads the sent cookie token
and sends it back to the server to confirm its status as a test subject.
The server constantly monitors the statuses of the cookie-tokens of all subjects and if any of the
tokens is not confirmed within ten seconds, then the subject's status will be transferred to “frozen”. To
prevent a user who does not pass the certification from connecting to the platform, a unique client token
is associated with the test subject's application.
When the application is launched, a third token is generated - a program token. It is necessary to
confirm the uniqueness of the use of the application launched on the subject's side. The software token
and the client token are encrypted using the public key of the RSA algorithm and sent to the server,
which binds the software token and the client token for the duration of the validation session.
The test-taker-side application continuously interacts with the verification-API microservice to
verify the tokens received by the test-takers browser from the server-side on which the attestation takes
place.
The verification-API microservice, when requested to verify the token by the user ID, verifies the
token that came from the subject and the token generated by the server after the tester made a request
to any resource on the server (see Fig. 1a).
The application on the subject's side, in addition to sending tokens for verification, conducts a
preliminary analysis of the video stream of attestation using a pre-trained neural network. The purpose
of the analysis is to find security incidents related to the work environment and behavior of the subject.
Recorded incidents, along with the recognized incident type, time, and frame of the video stream, are
sent to the platform database for recording. When requesting an attestation report, a request is made to
the exam-API microservice, which retrieves from the database all incidents related to the attestation of
a test subject in the form of a single object (see Fig.1b).
143
Figure 1: Requests to microservices for a) working with tokens, b) creating and receiving reports
All requests from the test taker to the resource associated with the attestation will be processed by
the action-API microservice, which acts as a router that controls the work of the test taker and the
proctor.
3.4. Features of Using the Platform
The examiner has two modes of attestation: in live proctoring mode and in the mode of viewing
reports during/after completion of attestation (see Fig. 2a). At the moment the test subject joins the
platform, his webcam, microphone, and desktop become available to the examiner (see Fig. 2b). The
examiner must be present during the preparation of the test subject's workplace.
If you intend to work with several candidates for certification, it is convenient for the examiner to
select the option of viewing reports. Any detected activity will be displayed in the reports: both an
intermediate frame confirming that attestation is under control and no violations were found, and frames
with signatures that indicate when and what type of incident was recorded (see Fig. 3).
Figure 2: List of subjects and the choice of the operating mode, b) screen view of the subject
144
Figure 3: Examples of report frames with different types of incidents: a) using a gadget, b) another
application window is open
4. Conclusions
The development of the ProFaMI platform has proven to be an economically viable project since it
allows the active use of remote control over certification activities without the high-cost involvement
of commercial third-party products. Its use allows solving the problem of confidentiality of data on the
course of certification and the personal data of the subjects. The use of the platform made it possible to
reduce the threat of the test subjects violating the requirements of "academic honesty", which has always
been a serious problem in higher education. Currently, the main efforts of developers are aimed at
implementing a wider use of algorithms for intelligent observation of the workspace of the subject.
5. References
[1] K. Hylton, Y. Levy, & L. P. Dringus, Utilizing webcam-based proctoring to deter misconduct in
online exams. Computers in Education, 2016. 92–93, 53–63. DOI:
10.1016/j.compedu.2015.10.002.
[2] A. Kumi-Yeboah, J. Dogbey, G. Yuan, Online collaborative learning activities: The perspectives
of minority graduate students. Online Learning, 2017. 21. DOI: 10.24059/olj.v21i4.1277
[3] Y.-C. Yeh, O.-M. Kwok, H.-Y. Chien, et al., How college students’ achievement goal orientations
predict their expected online learning outcome: The mediation roles of self-regulated learning
strategies and supportive online learning behaviors. Online Learning, 2019. 23, 23–41. DOI:
10.24059/olj.v23i4.2076.
[4] A. Janson, M. Söllner, & J. M. Leimeister, Individual appropriation of learning management
systems-Antecedents and consequences. AIS Transactions on Human-Computer Interaction, 9,
2017. PP. 173–201.
[5] K. Bovermann, & T. J. Bastiaens, Towards a motivational design? Connecting gamification user
types and online learning activities. Research and Practice in Technology Enhanced Learning,
2020. 15, 1. doi: 10.1186/s41039-019-0121-4.
[6] B. Boitshwarelo, A. K. Reedy, & T. Billany, Envisioning the use of online tests in assessing
twenty-first-century learning: A literature review. Research and Practice in Technology Enhanced
Learning, 2017. 12, 16. DOI: 10.1186/s41039-017-0055-7.
[7] A. S. Milone, A. M. Cortese, R. L. Balestrieri, & A. L. Pittenger, The impact of proctored online
exams on the educational experience. Currents in Pharmacy Teaching & Learning, 9, 2017. PP.
108–114. DOI: 10.1016/j.cptl.2016.08.037.
[8] C. S. González-González, A. Infante-Moro, J. C. & Infante-Moro, Implementation of E-proctoring
in online teaching: A study about motivational factors. Sustainability, 12, 2020. 3488. doi:
10.3390/su12083488.
145
[9] K. K. Hollister, M. L. & Berenson, Proctored versus Unproctored online exams: Studying the
impact of exam environment on student performance. Decision Sciences Journal of Innovative
Education, 7, 2009. PP. 271–294. DOI: 10.1111/j.1540-4609.2008.00220.x.
[10] Joseph A. Rios & Ou Lydia Liu, Online Proctored Versus Unproctored Low-Stakes Internet Test
Administration: Is There Differential Test-Taking Behavior and Performance? American Journal
of Distance Education, 31:4, 2017. PP. 226-241, DOI: 10.1080/08923647.2017.1258628
[11] Sandra Gudiño Paredes, Felipe de Jesús Jasso Peña & Juana María de La Fuente Alcazar, Remote
proctored exams: Integrity assurance in online education?, Distance Education, 42:2, 2021. PP.
200-218, doi: 10.1080/01587919.2021.1910495
[12] Oskar R. Harmon & James Lambrinos, Are Online Exams an Invitation to Cheat? The Journal of
Economic Education, 39:2, 2008. 116-125, DOI: 10.3200/JECE.39.2.116-125
[13] Donald L. McCabe, Linda Klebe Trevino & Kenneth D. Butterfield Cheating in Academic
Institutions: A Decade of Research, Ethics & Behavior, 11:3, 2001. 219-232, DOI:
10.1207/S15327019EB1103_2
[14] G. Ravasco Technology-aided cheating in open and distance e-learning. Asian journal of distance
education. 2012. Vol.10, № 2. PP. 71-77.
[15] V.F. Yakovlev, Countering Students' Academic Dishonesty in Distance Learning. Open and
distance education. 2016.Vol. 61, No. 1. P. 14-19.
[16] V.F. Yakovlev Effective measures to counteract the use of ghostwriter services by students in
distance learning. Open and distance education. 2017. No. 4 (68). DOI: 10.17223/16095944/68/8
[17] McCabe, D. L. (2005). Cheating among college and university students: A North American
perspective. International Journal for Educational Integrity, 1. DOI: 10.21913/IJEI.v1i1.14
[18] Mohammed Juned Hussein, Javed Yusuf, Arpana Sandhya Deb, Letila Fong & Som Naidu. An
Evaluation of Online Proctoring Tools. Open Praxis, vol. 12 issue 4, October–December 2020, pp.
509–525 (ISSN 2304-070X)
[19] International experience in conducting intermediate and final certification of university students
during the COVID-19 coronavirus pandemic. URL https://enic-
kazakhstan.kz/files/1586779981/sistema-proktoringa-mezhdunarodnyy-opyt-cbpiam.pdf
[20] N. V. Tuneva, A. A. Resenchuk, Organization and implementation of intermediate certification in
conditions of distance learning. Professional education in Russia and abroad, 4 (40) 2020. PP. 223-
228
[21] D. Foster, & H. Layman, Online proctoring systems compared. 2013. URL:
https://caveon.com/wp-content/uploads/2013/03/Online-Proctoring-Systems-Compared-Mar-13-
2013.pdf
[22] Assessment of the quality of education in the context of distance learning: the experience of the
pandemic with the school systems of the post-Soviet countries. Compiled and scientific editors
V.А. Bolotov, T.A. Mertsalova. M.: Alex (IP Polikanin A.A.), 2021. 390 p. URL:
https://eaoko.org/upload/library/21_Distance_WEB.pdf
[23] Online proctoring on your terms. – Examity, 2021. URL: https://examity.com.
[24] Proctorio: Securing the integrity of your online assessments. 2013-21. URL: https://proctorio.com.
[25] Verificient Technologies. Welcome, 2013-21. URL: https://testing.verificient.com.
[26] Solutions for Higher Ed – Respondus, 2021. URL: https://web.respondus.com/he/
[27] Examus analysis of user behavior, 2013-21. URL: https://ru.examus.net/
[28] ProctorEdu | Контроль онлайн тестов, 2016-21. URL: https://proctoredu.ru/proctoring
[29] ProctorU - The Leading Proctoring Solution for Online Exams, 2008-21. URL:
https://www.proctoru.com/
146