=Paper=
{{Paper
|id=Vol-2732/20200734
|storemode=property
|title=Students Academic Achievement Assessment in Higher Education
Institutions
|pdfUrl=https://ceur-ws.org/Vol-2732/20200734.pdf
|volume=Vol-2732
|authors=Svitlana Proskura,Svitlana Lytvynova,Olga Kronda
|dblpUrl=https://dblp.org/rec/conf/icteri/ProskuraLK20
}}
==Students Academic Achievement Assessment in Higher Education
Institutions==
https://ceur-ws.org/Vol-2732/20200734.pdf
Students Academic Achievement Assessment in Higher
Education Institutions
Svitlana L. Proskura 1[0000-0002-9536-176X] and Svitlana G. Lytvynova 2[0000-0002-5450-6635]
Olga P. Kronda 3[0000-0003-1780-9167]
1National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Prosp.
Peremohy 37, 03056 Kyiv, Ukraine,
slproskura@gmail.com
2Institute of information technologies and learning tools, M. Berlynskogo St 9, 04060 Kyiv,
Ukraine
s.h.lytvynova@gmail.com
3National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Prosp.
Peremohy 37, 03056 Kyiv, Ukraine,
my.krona@gmail.com
Abstract. Students academic achievement assessment in higher education
institutions is one of the most urgent tasks that teachers face in the role of
organizers of the educational process. The problem of forming a correct and
objective rating point in assessing the results of studying a discipline is the key
point in the final stage in studying thematic and module parts of the material.
The rating score is formed by assessing individual student's educational
activities. Depending on the type of assessment and control the students' vision
of assessment objectivity is different.
This study analyzes the works of Ukrainian and foreign authors who have
devoted their scientific research to studying the question of students' learning
outcomes control and evaluation; highlighting the principles of formative
assessment and the peculiarities of distance learning assessment tasks
formation.
Particular attention is paid to the development of the author's methodology
for educational achievements assessment for future bachelors of computer
science. The contents of Bloom's taxonomy levels are discussed, as well as the
explanation of the types of activities for the assessment system, for example, the
design of practical tasks of the discipline "Algorithmization and Programming"
(C ++ programming language) of the specialty "Computer Sciences", which is
being studied in the first year by future bachelors of computer sciences.
Keywords: students academic achievement assessment, Bloom’s taxonomy,
assessment rating system
1 Introduction
Each year in Ukraine, higher education institutions with IT-specification training
prepare a sufficient number of IT branch bachelors, in particular, future bachelors of
computer science who are in extreme demand in the domestic IT job market.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
But there is enough evidence that only 25% of IT-specification graduates can find
employment in an IT company, while other graduates do not meet the requirements
which are put forward by the employers while headhunting.
Which remains important is the question of improving the quality of professional
training for future bachelors of computer science, of their level of professional com-
petence, and in particular, their IC competence, so that it could meet the IT market
current requirements, global needs as well as employers' requirements [1, р.11].
Nowadays in our country there is still no single educational toolkit that allows
measuring and assessing the outcomes of the learning process as well as the level of
competency acquired, although there are examples of such tools on international level
(for example, TIMSS, PISA, CIVIC Education Project) which Ukrainian students can
use as well, if they are willing to [2, p.154].
The improvement of forms and methods aimed at creating a coherent system of
continuous education allows to develop a unified system for assessing the dynamics
of development motion for bachelors of computer sciences in their educational
activities
2 Research Results
The following methods were used in the research process: theoretical - studying and
analysis of pedagogical, methodical and scientific literature; modern scientists studies
results analysis to find out the state of the problem; observational - educational pro-
cess observance; praximetric - study and analysis of curricula, educational documen-
tation, programs and results of student activities as well as research methods such as
surveys, questionnaires, teacher interviews, methods of comparative analysis and
statistical data processing.
Theoretical and practical issues of the assessment system organization for academ-
ic students achievements in higher education institutions are covered in the works of
V.M. Bocharnikova [3], V.M. Kukharenko [8], S.G. Lytvynova [9], M.I. Tomilova,
Ye.Yu. Vasilyeva, O.A. Kharkova [4], L.M. Ognevchuk [2], B.Ye. Starychenko [7]
and others.
However, special attention needs to be paid to the issues of procedural, technologi-
cal, motivational support for the assessment of the academic achievements of students
in higher education institutions as a coherent pedagogical system that takes into ac-
count individual interests, abilities and inclinations.
3 Research Results
New technologies introduction into the educational process in higher education
institutions leads to the emergence of new unconventional forms and methods of
assessing the educational attainment of students in higher education institutions. In the
framework of our study, we consider the research of scholars in relation to students
educational achievements assessment, in particular future bachelors of computer
sciences.
On one hand we can see, according to V.M. Bocharnyk the assessment, which is
carried out in the course of students' educational achievements control, stimulates
them to active educational and cognitive activity, and therefore, acts as a learning
factor, which determines positive results of the educational process [3, p.23].
On the other hand there is a substantial subjective part to it - M.I. Tomilova,
Ye.Yu. Vasilyeva, O.A. Kharkova investigated the issues of students’ perception of
knowledge assessment done while realizing studying activities in higher education
institutions. They came to the conclusion that students differently evaluate the
objectivity and fairness of their knowledge assessment. Most students highly rate the
objectivity of their knowledge assessment in an oral questionnaire or interview, but
one in ten felt that the results did not correlate with the real level during tests. Most
students praised the objectiveness and fairness of the point-rating scale. More than
half of the students (59%) rated the degree of external factors (mood, attitude to the
teacher) influence on the process [4, p.30].
Students’ evaluations of the assessment they get are not the only thing that diverg-
es, but also the system of assessment making itself as well.
Foreign authors S.A. Borodich, A.N. Teplyakovskaya, comparing the problems and
perspectives of the point-rating system (PRS) of student knowledge assessment at
universities in Russia and Belarus, conclude that in Belarus PRS is practiced only as a
system of ten-point rating basis and only in face-to-face classroom education. Russian
universities adopted a PRS of maximum points in a discipline of 100 points. These are
scored for: intermediate attestation (credit or examination) - 40 points and other 60
points are achieved by the student during current and boundary control of knowledge
build up during the semester educational process [5, p.139]. It is worth while noting
that in the first year at university students have different starting points due to
different conditions of studying in schools and the divergences of goals of educational
process subjects.
Ukrainian higher education institutions have also adopted the system of 100 points
rating for the students’ achievements in one discipline. An example of this is the
provision on the rating system for evaluating the learning outcomes of NTUU " I.
Sikorsky Kyiv Polytechnic Institute” [6].
B.Ye. Starichenko, a Ukainian expert, proves that obtaining a disciplinary rating
score is possible and that its basis is the evaluation of all separate student's
educational activities, provided by the discipline study plan. After calculating the
average share of performance across all activities, it is reduced to a 100-point scale,
which is a disciplinary rating [7, p.205].
A system for evaluating the educational achievements of higher education institu-
tion students based on a competency-based approach was proposed by L.M. Ogni-
vchuk in her works. She also observed the fact that in other countries experts tradi-
tionally identify three main approaches to defining and putting into practice a compe-
tence interpretation to the quality of learning outcomes: a behavioral approach (USA),
a functional approach (UK) and a multidimensional and holistic approach (France and
Germany). These approaches appeared independently from each other first in the
United States, then in the United Kingdom, and most recently in France and Germany
[2, p.155].
A many year research into the issues of distance learning technologies introduction
brought V.M. Kukharenko to the conclusion that at this stage of development of
scientific approaches to students' academic achievement assessment, special attention
has to be paid to the formative assessment which was adopted [8, p.53].
Another Ukrainian researcher S.G. Litvinova agrees with V.M. Kukharenko. In her
work she notes that it is possible to increase the efficiency of control and assessment
of students' knowledge with the help of formative assessment. In the present study we
agree with her view that formative assessment is used by teachers to obtain data on
the current state of students knowledge level in a particular topic as well as to identify
next steps that should be taken to improve them [9, p.112].
The authors of this research clearly believe that benchmarking is needed for suc-
cessful assessment organization as well as points that will allow evaluating the as-
sessment system itself working properly.
Periodicity, educational and cognitive activity motivation, as well as
individualization and differentiation can be seen as indicators of the didactic
effectiveness of educational achievement formative assessment.
The peculiarities of formative assessment lie in the fact that it is student's strive in
realization of his/her educational goals, which is assessed but not his / her personality.
A clear algorithm for determining the grade is offered, which is mase understandable
for the student; the focus is on the student's personal progress rather than the
assessment. The following forms of students progress formative assessment of are
offered: reflexive techniques (hand signals, card signals) to clarify and identify
complex issues; clarifying questions; analytical questions; mini-tests; checking
creative work for error detection etc. [9, p.112]. Among the features are: training,
stimulating, controlling.
Considering the positive experience of the teachers of the National Technical
University "Kharkiv Polytechnic Institute", in particular V.M. Kukharenko, in
conducting an experiment on the use of Bloom's taxonomy to evaluate students'
academic achievement, we will apply this approach to develop authors’ methodology
for evaluating academic achievement of future bachelors of computer science.
The authors of this article would like to point out that this approach is the basis for
the control and assessment unit of the model of professional competence formation
for future bachelors of computer science. Programming students knowledge level
assessment is carried out using Bloom's taxonomy, which contains 6 levels of difficul-
ty. Each practical task will correspond to its level with a certain number of points [10,
p.110].
What should be taken into consideration when forming this model’s control unit is
that the employers put high requirements to employees, and the level of knowledge
and skills of the graduates does not meet these requirements, as their education has
mostly theoretical nature. This, in turn, requires constant correction of curricula and
subjects taught in higher education institutions, as well as regular personnel retraining
[11, p.85].
While assessing the knowledge level, substantial attention is paid to monitoring the
work results of future bachelors of computer science, as an integral part of educational
process, which is aimed at providing "student-teacher" feedback and to identify the
basis of its correct organization [12, p.10].
Tasks formation of is carried out in accordance with the spheres of cognitive
(Cognitive Domain), emotional (Affective Domain) and motor (Psychomotor Do-
main) goals. Cognitive goals cover everything related to knowledge acquisition and
mental skills development. Emotional goals include all tasks related to values for-
mation, relationships, students' emotional self-control development. Motor goals em-
brace motor skills development, as well as physical endurance [13, p.89], [14], [15].
While drawing up practical tasks, particular attention is paid to the field of
cognitive goals, which is divided into the following six levels:
1. Remembering (Knowledge Level) – lower level.
2. Understanding (Comprehension Level).
3. Implementation (Application Level).
4. Analyzing (Analysis Level).
5. Assessment (Synthesis Level).
6. Creation (Evaluation Level) – higher level.
This is a classification of thinking, which is organized according to complexity levels
and gives teachers and students the opportunity to learn and act in informational and
educational space, provides a simple structure for many types of questions [13, p.98].
For example, we consider the content of Bloom's taxonomy levels with the
definition of activity (verbal form) for designing practical tasks of the discipline
"Algorithmization and Programming" (C++ programming language) specialty
"Computer Sciences", which is studied in the first year by future bachelors of
computer Sciences (Table 1).
Table 1. The content of student’s activity (verbal form) during practical tasks realization ac-
cording to Bloom’s taxonomy
Definition of student
Bloom’s taxon-
Level Contents activity
omy levels
(verbal form)
All entry-level goals are formu- Show, characterize,
lated in practical reproduction adhere to a code design
Remembering examples. It is enough to acquaint standard (coding standard,
Level students with theory and relevant programming style),
(Level 1) practical examples so that they program code lines
can repeat it in their programs explanation by means of a
comment
In order to demonstrate the
practical achievements of the un-
derstanding level (comprehen-
sion), programmer students have
to: Compile, implement,
─ draw up a block diagram of a describe, explain,
simple task algorithm; anticipate, define,
Understanding
evaluate, adhere to the
Level
─ implement the algorithm in the standard of code design
(Level 2)
form of a program code (com- (coding standard,
pilation without errors); programming style).
Explain code lines
─ apply the acquired knowledge using the comment
in the program code into specif-
ic simple tasks (information is
remembered and processed in-
dividually).
In the Implementation level a
student-programmer must fully
demonstrate practical achievement
of the Comprehension Level Apply, demonstrate,
Implementation
(Level 2). In addition, solve, exe- count, execute, illustrate,
Level
cute and display a program code show, solve, test
(Level 3)
with advanced levels of complexi-
ty and functionality of previous-
level practical tasks
The goals of the Analyzing lev-
el (analysis) assume that pro-
gramming students are able to:
─ analyze the task;
─ perform decomposition of the
task (partitioning into separate Analyze, decompose
Analyzing
simple tasks); Execute complex prac-
Level
tical tasks algorithm
(Level 4). ─ draw up a general flow chart of (write a program code)
the algorithm and sub-task flow
charts based on decomposition;
─ implement the algorithm of
complex practical tasks in the
form of a program code using
functions
At the level of synthesis (as-
sessment), students-programmers Combine, integrate,
must fully demonstrate practical modify, reposition, re-
Assessment achievements of the Analyzing place, plan, create, design,
Level level (level 4). In doing so, solve, invent, anticipate (what
(Level 5) execute and display program code if?), assemble, formulate,
with increased levels of complexi- prepare, generalize, re-
ty and functionality of previous write
level practical tasks
At the sixth level, programming
students demonstrate:
Evaluate, decide, clas-
─ application of the studied mate- sify, sort, control, meas-
rial as a tool in solving com- ure, recommend, persuade
Creation plex problems of different di- (assure), select (select),
Level rections (logical, mathematical, judge (evaluate), explain,
(Level 6) physical, metric, etc.) distinguish (recognize),
support, conclude (finish),
─ performing the decomposition compare (compel), sum-
of the task (partitioning into marize.
separate simple tasks),
─ drawing up a general flow chart
of the algorithm and a flow
chart of the subtasks,
─ application of the first para-
digm of object-oriented ap-
proach - encapsulation (presen-
tation of the program code in
the form of 2 separate files: in-
terface file (.h-file) and imple-
mentation file (. spp-file))
─ algorithm implementation with
the use of functions
The authors offer an educational achievements assessment rating system using
Bloom’s taxonomy levels which is represented in Table 2.
Table 2. Student educational achievements rating system using Bloom’s taxonomy levels
Levels by Credits
Levels Rating Credits ECTS
Bloom’s Taxonomy ECTS
6 Creation 95-100 А perfect
5 Assessment 85-95 В very good
4 Analysis 75-85 С good
3 Implementation 70-75 D very satisfactory
2 Understanding 65-70 Dх satisfactory
1 Remembering 60-65 Е sufficient
In addition, each level of Bloom's taxonomy is subdivided into the following sub-
levels: practical assignment accomplishment and defence, practical assignment ac-
complishment quality, term of delay (deadline non-meeting) for a practical assign-
ment accomplishment.
The score of each sub-level may be reduced / increased. In order to determine the
percentage reduction / increase in the resultant evaluation of each sub-item, a survey
was conducted with 76 teachers from 23 regions of Ukraine and the city of Kyiv
participating.
The results of the survey, namely the percentage reduction in the resultant
evaluation of each sub-level, are presented in the form of diagrams and are shown as
follows: practical assignment accomplishment - Fig.1, practical assignment
accomplishment defence - Fig.2, the term of delay (deadline non-meeting) for a
practical assignment accomplishment - Fig.3.
Fig. 1. Practical assignment accomplishment
Fig. 2. Practical tasks defence
Fig. 3. Practical assignment accomplishment quality
For convenience, all these results are summarized in Tables 3, 4.
Table 3. Rating reduction percentages for practical assignments accomplishment
1. Practical assignment accomplishment
№ Error type % reduction
1 Syntax errors (no application compilation is running) 100
Logical errors leading to exceptional situations (com-
2 50
puter “freezes”)
3 Incorrect task realization algorithm 25
4 The result does not meet the condition of the task 15
5 Uninformative output and input of data 10
Total 100
2. Practical tasks defence
№ Error type % reduction
The student cannot explain the practical assignment
1 50
program code
The student does not answer test questions on the
2 25
topic of the practical assignment
The student cannot make changes to the program to
3 15
correct logical errors
The student does not know theoretical material on the
4 10
practical assignment topic
Total 100
3. The term of deadline non-meeting
№ Delay period % reduction
1 Two weeks 10
2 Two weeks to four weeks (month) 20
3 Four weeks to six weeks 35
4 Six weeks to eight weeks (two months) 50
5 Eight weeks to 12 weeks 70
6 More than 12 weeks 80
Table 4. Rating reduction percentages for practical assignments accomplishment
Practical assignment accomplishment quality
№ Non-standard approach type % reduction
1 Unconventional problem solving algorithm 40
Using software constructs that optimize code based
2 on performance or RAM volume 30
Using a user-friendly interface. (menu, dialog
3 20
boxes)
Additional extra curriculum literature studying
4 (heuristic combinatorial algorithms, Windows 10
programming, etc.).
Total 100
Let us study an example of practical assignments accomplishment assessment making
for a practical assignment accomplished by a student. The percentages of each
decrease for practical assignments accomplishment component are summarized in
Table 5. All percentages of decrease for practical assignments accomplishment
obtained are summed up.
Table 5. All percentages of decrease for practical assignments accomplishment obtained are
summed up.
Rating % % Rating
planned reduction increase total
1. Practical task accomplishment 25
2. Practical task accomplishment
0
defence
3. Practical task accomplishment
10
report
4. Practical task accomplishment 0
quality
5. Practical task accomplishment
0
delay
Total : 35%
for level 6 100 65,00
for level 5 95 61,75
for level 4 85 55,25
for level 3 75 48,75
for level 2 70 45,50
for level 1 65 42,25
In other words, for example, a student chooses a practical assignment that corresponds
to the sixth level, which maximum score equals 100. Supposing the student receives a
25% reduction for task implementation incorrect algorithm in the section " Practical
assignment accomplishment" and - 10% for the inability to answer test questions on
the topic of the practical task in the section " Practical assignment defence ". The
reduction percentages are added up. The final rating points reduction depends on the
total summed up reduction percentages (see Table 5). There is a possibility that the
students can improve their results as well. For example, if in the “Practical task ac-
complishment quality” section the student received a percentage increase - the final
score will be increased by the amount of that percentage.
Thus, this technique can be used to evaluate students' academic achievement using
Bloom's taxonomy levels in any discipline
4 Conclusions
The present research analyzes works of Ukrainian and foreign authors on the issues of
control and educational activity results assessment while working with students in
higher education institutions. The principles of formative assessment and the levels of
cognitive, emotional and motor goals are highlighted in the article.
Authors’ technique for educational activity results assessment for future bachelors
of computer sciences, which combines Bloom's taxonomy and the classical scoring
system for students' knowledge assessment and makes it possible to implement differ-
entiated and individual approaches and build an individual trajectory of each student’s
development.
The authors substantiate the content of Bloom's taxonomy levels, explain the types
of activities for the assessment system on the example of designing practical tasks of
the discipline "Algorithmization and programming" gives the teacher a new
methodology and a new tool for assessing student achievement.
This technique, thanks to the use of Bloom's taxonomy, will enable the teachers to
determine the level of competence of the student at intermediate stages of learning.
Solving problems of a certain grade correctly, partially or not solving at all - the
student realizes his level of knowledge.
The disadvantages include the fact that for this technique the teacher needs a large
amount of time to create a set of methodological support for specialties.
References
1. Proskura, S. L., Lytvynova, S. H.: Future bachelors of computer sciences professional
competency formation. Physical & Mathematical Education 2(20), Sumy, pp. 137-146.
(2019).
2. Ognivchuk, L. M.: Students’ learning achievements assessment in higher educational insti-
tutions. Educational discourse № 3 (7), pp. 154-166. (2014).
3. Bocharnikova, V. M.: Stimulation function of knowledge and skills control of high school
students: synopsis. dis. ... ped. sci. cand: 13.00.01, Kyiv, 23. (1999).
4. Tomilova, M.I., Vasylieva, Ye.Yu., Kharkova, О.А.: Students’ evaluation of knowledge
assessment in higher education institutions. Human ecology, vol.9, pp. 29–33. (2013).
5. Borodych, S. А., Tieplyakovskaya, А. N.: Point-rating system of students knowledge as-
sessment in higher education institutions: problems and perspectives. Innovative pedagogi-
cal technologies : mat-ls IV Internat. sci. conference. Kazan : Buk, pp. 139–141. (2016).
6. The provision on the rating system for learning outcomes assessment of NTUU "I.
Sikorsky Kyiv Polytechnic Institute”. https://kpi.ua/regulations-RSO, last accessed
2020/05/21.
7. Starychenko, B. Ye. Point-rating system of students’ knowledge assessment: questions of
modeling. Pedagogical education in Russia, vol. 6, pp.205-215. (2017).
8. Kukharenko, V. M.:System approach to the blended learning:Information Technologies in
Education Issue 24. Kherson, pp. 53-67. (2015).
9. Lytvynova, S.H. The model of computer modeling system use for students’ competencies
formation in natural-mathematical subjects students. Physical-mathematical education: sci-
ence journal. Sumy : [A. S. Makarenko SumSPU]. Vol. 1(19), pp. 108-115. (2018).
10. Proskura, S. L., Lytvynova, S. H.: Information technologies specialists training in higher
education institutions of Ukraine: general state, problems and perspectives: Information
Technologies in Education 35, Kherson, pp. 72-88. http://ite.kspu.edu/issue_35/p-72-88.
(2018).
11. Proskura, S. L., Lytvynova, S. H.: Organization of independent studying of future
bachelors in computer science within higher education institutions of Ukraine. 14th
International Conference on ICT in Education, Research, and Industrial Applications (3L-
Person 2018), pp. 348-358. (2018).
12. Kukharenko, V. M., Berezenska, S. M., Bugaychuk, K. L, Oliynyk, T. O., Rybalko, O. V.,
Theory and practice of blended learning, monograph Kharkiv: City Press, NTU KHPI,
P. 284. (2016).
13. Proskura, S. L.: The model of professional competency formation for future bachelors of
computer sciences. Physical & Mathematical Education 3(21), Sumy, pp. 104-112,
https://fmo-journal.fizmatsspu.sumy.ua/publ/4-1-0-522. (2019).
14. Spirin O., Burov O. Models and applied tools for prediction of student ability to effective
learning. 14th International Conference on ICT in Education, Research and Industrial Ap-
plications. Integration, Harmonization and Knowledge Transfer. CEUR-WS, vol. 2104,
pp. 404-411. (2018).
15. Mintii, I. S., Shokaliuk, S.V., Vakaliuk, T. A., Mintii, M.M., Soloviev, V.N. Import test
questions into Moodle LMS. Cloud Technologies in Education : CEUR Workshop Pro-
ceedings, vol. 2433, pp. 529-540. http://ceur-ws.org/Vol-2433/paper36.pdf. (2018).