In the article use innovative possibilities of assessing and predicting the academic success of students in higher education institutions are investigated. The proposed multinomial logistic regression model allows to assess and predict the success of students based on the obtained estimates and taking into account various factors. The multinomial logistic regression model is one of the best tools for determining the quality of training for students, proceeding from the quantitative (intellectual) and qualitative (behavioral) factors. The following factors were used in the study: an assessment of the student's presence in the classroom; student's grade in mathematics; student's grade in computer science; assessment of the student's living conditions. The use of regression analysis methods allows identifying implicit relationships between elements of the educational process and solve many problems related to the analysis and prediction of academic success of students. A multinomial logistic regression model confirms that assessment of the student's presence in the classroom; student's grade in mathematics; student's grade in computer science; assessment of the student's living conditions are important indicators that reflect the academic success of students in higher education institutions. It is possible to predict individual learning outcomes, evaluate student actions, and assess factors related to success, create an individual learning plan through the constructed model. This knowledge is significant for university administration and teachers who want to improve the quality of teaching and ensure better learning outcomes.
Educational data mining plays an important role in the development of the learning environment. Modern higher education institutions operate in a highly competitive and complex environment. The main problems faced by higher education institutions are: analysis of learning outcomes, quality education, evaluation of student performance, strategic decision-making, and formation of human capital at all. Universities need to implement student intervention plans to address the challenges that students face during their studies.
Different methods of machine learning are used to understand and overcome the main problems. Various innovative methods and methods of machine learning are used to understand and overcome the main issues. Modern development of society requires the use of new innovative methods of assessing students' educational levels in higher education institutions, evaluating of the knowledge gained inside the higher education, training of students in various subjects, in particular information systems in management. These methods will allow future professionals to be more competitive in the labor market. Innovative methods in higher education are characterized as technologies based on innovations: organizational (related to the optimization of educational conditions), methodological (updating the
2022 Copyright for this paper by its authors. content of education and improving its quality); technical (related to the use of new tools of artificial intelligence in the field of informatics).
Machine learning methods play an important role in predicting the success of students at risk and dropout, thereby improving learning outcomes. Machine learning and statistical methods for educational data are needed to identify significant patterns that improve students' knowledge and the learning process in higher education institutions. Predicting student achievement at the grassroots level and beyond helps universities develop intervention plans in which the executive heads and university professors are beneficiaries of student performance prediction plans. Predicting academic success of students provides excellent benefits for improving student retention rates, targeted marketing, and overall student performance, as well as for effective enrollment management, graduate management.
The hypothesis is that in order to achieve student's academic success and qualifications in management information systems, it is necessary that internal elements be considered as an integral part of intellectual and behavioral factors. The proposed multinomial logistic regression model allows university administration and teachers to assess and predict the success of students based on the obtained estimates and taking into account various factors. The following factors were used in the study: an assessment of the student's presence in the classroom; student's grade in mathematics; student's grade in computer science; assessment of the student's living conditions. Assessing the academic success of students in the discipline of information systems in management is the ultimate task.
The model also allows predicting the individual learning outcomes, to evaluate student actions and differences between actions, to assess factors related to success, to create an individual learning plan through the constructed model. The theoretical and methodological basis of the paper is the scientific and research works of domestic and foreign academicians, which reveal the issues of educational management, modeling student performance.
The survey rests on the following general and specific methods: a systematic approach to the study of machine learning methods to improve the quality of the educational process in higher education institutions; a deductive method to solve the problem of systematization of the factors that affects the academic success of students; multinomial logistic regression analysis as a machine learning method to solve the problem of forming the model of the influence of factors on the assessing the academic success of students and get an information about the level and quality of the learning in higher education institutions; analytical method for assessing the state of academic success of students in higher education institutions.
Analysis of the academic achievement of students at all levels of training is a prerequisite for the effective functioning of the management system of the educational process. The process of managing human capital preparation involves a wide range of interdependent and complementary components. An important component is the analysis of the organization of the educational process and, foremost, the process of forming the academic success of students.
The relevance of methods of regression data analysis in educational systems is confirmed by a number of studies of learning processes.
Kovacic [1] analyzed the results of predicting student performance through machine learning methods. Demographics (education, employment, gender, status, disability, etc.) and course characteristics (course syllabus, course unit, etc.) were taken into account. The data were collected at the Open University of New Zealand. The main features influencing student success were identified through machine learning algorithms. Three main factors influencing student performance (course syllabus, course unit, and ethnicity) were identified. It is important to choose subjects that determine the success of students in the course program.
Lakkaraju [2] suggested identifying failed students or students who risk not graduating on time. To do this, he used the structure of machine learning. Using this structure, student data was collected from two schools in two districts. The study took into account five machine learning algorithms: Support Vector Machine (SVM), Random-Forest (RF), Logistic Regression, Adaboost and Decision Tree. These algorithms are evaluated using accuracy, response, and AUC for binary classification. Each student is evaluated based on a risk assessment evaluated according to the proposed classification. The author also paid important attention to the impact of student presence and involvement in the learning process. Teachers need to look for opportunities to strengthen presence and involvement of student with regard to learning process in higher education institutions. The results showed that Random Forest demonstrates the best performance.
Oyedeji and other scholars [3] have used machine learning techniques to analyze student performance as an element of the formation of human capital. The obtained results allow educational institutions to eliminate methods that cannot improve student performance. Their study took into account individual attributes, including the age of the student, their demographic distribution, individual attitude to the survey and marital status. The authors noted the need to include in the study of the academic success of students the student's living conditions. Various machine learning algorithms were used in the study. For a comparative analysis of productivity, three models have been identified: learning with a teacher using linear regression and in-depth learning.
Alhusban [4] used machine learning methods to measure and increase the dropout undergraduate students. They collected data from al-Bayt University students. The following factors were taken into account: gender, type of enrollment, enrollment marks, city of birth, marital status, nationality and subjects studied at stage k-12. The researchers used Hadoop and an open source platform based on machine learning. It has been found that the marks for entrance examinations significantly affect specialization. The researchers have been stated that students' social status has a significant impact on academic performance. This is of great importance for the development of higher education institutions and the maintaining human capital.
Our study took into account the ideas of scientists who emphasize the importance of the influence of factors (course programs, student's living conditions, student's presence in the classroom) on the academic success of students. The course program of the specialty "Management information systems" include various subjects. In our study, important subjects such as computer science and mathematics were selected. These subjects are the basis of the specialty "Management information systems".
To assess the academic success of students, a relationship should be established between factors and academic success. We will take the student’s academic success indicator according to three values (good, satisfactory and unsatisfactory). Therefore, it is necessary to predict the success of students through a model of multinomial logistic regression. Logistic regression is used to predict the probability of an event by the value of a set of features.
Now, consider modeling assessments of students' academic success in order to predict their future condition. This means estimating a qualitative variable (intellectual factors) by several quantitative factors (behavioral factors). Discriminant analysis tools can be used to select the most informative quantitative variables. Logit regression allows determining the group of students' academic success. In addition, logistic regression makes it possible to consider the probability that a student will be classified as a certain group success.
The probability function is the basis of the method and expresses the probability density of the simultaneous appearance of the sample results Y1, Y2,…, Yn [5]:
L(Y1,Y2,…,Yk;Θ)=p(Y1;Θ)⋅…⋅p(Yn;Θ)
According to the method of maximum likelihood, the value of Θ=Θ(Y1,…,Yn) [6], which maximizes the function L is taken to estimate the unknown parameter.
The calculation process is simplified by maximizing not the function L, but the natural logarithm ln(L). This is due to the fact that the maximum of both functions is achieved at the same values Θ [7]:
All multiple choice options are numbered in random order 0, 1, 2, ..., J [8]. The probability of occurrence of an option is described by a polynomial logit model
L*(Y;Θ)=ln(L(Y;Θ))→max
x1 – assessment of the students’ presence in the classroom (behavioral factor);
(
b = (b0, b1,…,bm)T,
xi = (1, xi1,…,xim),
xib = b0 + b1xi1 + b2xi1 +…+ bmxim
To identify the model (
The coefficients of the model are estimated by numerically solving the plausibility equations. To write the plausibility equation itself (its logarithmic form) it is convenient to enter the variable dij, which becomes 1, if the i-th observation was chosen j-th alternative among (J + 1), and 0 – otherwise. Then for each i only one dij will be equal to 1.
The solution of this system, given that bj = 0, is carried out numerically using the Newton-Rafson method [13]. The computer implementation is arranged as follows: the values of the model that corresponds to the last of these alternatives become zero. In other words, if we want b0 = 0, instead of bj, then the data corresponding to the alternative number j = 0, must be entered last.
+1 = −
ln ( )[∂2ln ( )] ∂ ∂ ′ −1
To implement the Newton-Rafson method requires a matrix of partial derivatives of the second order [14]: 2ln
=– ∑ ( ′ = ′ ∑=1 ( −
The main data for the statistical analysis of the educational process are data on the success of undergraduate students at the economic faculty of Ivan Franko National University of Lviv (Table 1), x1 – assessment of the students’ presence in the classroom; x2 – students’ grade in mathematics; x3 – students’ grade in computer science; x4 – assessment of the students’ living conditions.
computer science
Students’
living conditions 10 8 8 8 7 7 8 7 7 8 8 8 10 10 = ∙ 1 =
( ∗ ∗) | − | = 0 ∑ = =1 7 6 7 7 8 8 6 8 6 7 7 8 6 5 7 6 8 6 7 6 7 8 6 6 7 7 8 8
Analysis of the data showed the presence of multicollinearity. Therefore, it is advisable to use the method of principal components. The main idea of the method is to replace highly correlated variables with a set of new variables between which there is no correlation. The new variables are linear combinations of the original variables
∗ =
− ̅
, = 1,2, … , ; = 1,2, … ,
Then we need to find the characteristic numbers of the matrix r from the equation
at the same time
2
2
3
1
2
3
3
2
3
3
3
2
2
2
(
The eigenvalues k ( = 1,2, … , ) are ordered by the absolute level of the contribution of each main component to the total variance.
It is necessary to calculate the eigenvectors ak by solving the system of equations ( − ) = 0 (16)
= ∗ ∙ (17)
0,77 −0,01 −0,18 −0,62 (18) −0,43 0,71 0,05 −0,56 = (−0,42 −0,71 0,16 −0,55 )
0,24 0,07 0,97 0,01 = (0,53 0,72 1,03 1,72)
We find the connection of the dependent variable Y with the main principal components z3 and z4, defining all principal components and rejecting those that correspond to small values of characteristic roots. To do this, we build a model of multinomial logistic regression. The parameters of the obtained multinomial logit model in the STATISTICA environment are as follows:
It can be concluded that the obtained estimates of coefficients are statistically significant (all standard errors are less than the obtained estimates, the values of Wald's statistics exceed the critical level, and all error probabilities are less than 0,2).
Let's write down the analytical expression for the constructed multinomial logit model:
−1+(
3,76+(
The resulting expression can be used to assess academic success of students at different values of the factors.
In particular, figures 1 and 2 show the dependence of the studied indicator when changing the factor x3 (students’ grade in computer science) for certain values of the factor x1 (assessment of the students’ presence in the classroom) and the fixed value of the factors x2 and x4 (x2 = 6, x4 = 3).
Figures 1 and 2 show an increase in the probability of receiving a better grade and a decrease in the probability of obtaining a bad grade in the subject of information systems in management, given the
Scale 0 increasing in the students’ grade in computer science. Different values of the students’ presence in the classroom have a great influence on the dynamics of increasing the probability of getting a better grade.
We can say that the key to measuring student achievement is assessing student progress by measuring positive changes in the assessment of the students’ living conditions and students’ presence in the classroom, as well as gradual changes in grade in computer science. n i tsem r)"y
o ssy tac no ifs 1 ti t am (s"a0.8 Students’ grade in computer science x1=5 x1=6
We can say that the key to measuring student achievement is assessing student progress by measuring positive changes in the assessment of the students’ presence in the classroom and students’ living conditions, as well as gradual changes in grade in mathematics. n i Students’ presence in the classroom x2=6 x2=7 Students’ grade in computer science x2=8 x2=9
We can say that the key to measuring student achievement is assessing student progress by measuring gradual changes in grade in computer science and grade in mathematics, as well as positive changes in assessment of the students’ presence in the classroom and students’ living conditions tsem r)"y
o ssy tac tino itfs 1 am (s"a0.8 ro t in enm0.6 f fo eg0.4 Students' grade in computer science x2=8 x2=9
The goal-oriented approach has a solid foundation for improving the effectiveness of student learning. We can rely on modeling results to develop pedagogical frameworks that help students develop an approach to learning.
Now let's evaluate academic success of students through the developed regression model of an academic success of students. Information for assessing academic success of students is presented in table 3.
In the first case, we are likely to receive the grade of "satisfactory". In the second case, we are expecting to receive the grade of "good". It is only in the latter case that a getting the grade of "unsatisfactory" would be most likely.
It is possible to predict individual learning outcomes, to evaluate student actions and differences between actions, to assess factors related to success, to create an individual learning plan through the constructed model. The presented approach has a solid foundation for improving the effectiveness of student learning. We can rely on modeling results to develop pedagogical frameworks that help students develop an approach to learning.
The multinomial logistic regression model confirms the possibility of combining quantitative (intellectual) and qualitative (behavioral) factors. Our study confirms that student's presence in the classroom, student's grade in mathematics, student's grade in computer science, and student's living conditions affect academic success of students. The study highlighted that academic success of students is driven by variant factors of students’ activities. It has been found that the marks for examinations (student's grade in mathematics; student's grade in computer science) significantly affect success. Important quantitative (intellectual) factors that determine the academic success of students are student's grade in mathematics, student's grade in computer science. Important qualitative (behavioral) factors that determine the academic success of students are student's presence in the classroom and student's living conditions.
The current study’s conclusion accords with considered research regarding academic success of students. The current study’s conclusion accords with considered research regarding academic success of students. Student's presence in the classroom are much likelier determines the main indicator we are researching (Lakkaraju). Student's living conditions are usually strongly affects the productivity of the organization (Oyedeji and other scholars).
The findings indicate that selected factors can significantly affect academic success of students, and creating an educational environment for students, and developing ways to motivate students. This study proved that the directions of the students’ activity can be translated into particular approaches to maintain of academic success. 5. Conclusions
The implementation of the regression method to study educational processes in higher education institutions is considered an innovative method. The use of the regression analysis methods allows identifying implicit relationships between elements of the educational process and solve many problems related to the analysis and prediction of academic success of students. It also allows you to build individual educational trajectories. The utilization of these obtained regression relations has а great importance to predict the academic success of students on the basis of statistical information on the control measures of the curriculum. They make it possible to establish a quantifying impacts of disciplines on special disciplines.
This innovative approach will have a greater effect in creating incremental learning behaviors and focusing on the influence of behavioral (non-intellectual) factors. Behavioral factors are difficult to operationalize and suffer from poor prognostic validity. The calculation strategy shows that the mutual influence of intellectual and behavioral factors in students' success can have an impact.
The research methodology can be used to determine the dependence of student learning outcomes on various factors. This is necessary to improve the quality of education in universities. Creative application of regression equations allows carrying out the quality control for each training course, based on individual work with students agreed within teachers of the departments. This research showed to be instrumental in demonstrating practices based on a probit regression model at the level of development of academic success of students for support for the successful completion of their tasks by students, better acquisition of knowledge, acquisition of competencies and profession. The proposed method allows educational institutions to improve the learning process, make learning complete, increase student achievement strategies.