In present work we study, with use of multivariate ANOVA model, the influence of independent factors such as year, faculty, gender, on the indicators of students' general intelligence (IQ) with a sample collected in 1991-2013 at the Novosibirsk State Technical University. The peculiarity of models of this type is that the response is a quantitative variable, and the input features must be qualitative. Therefore, first, the problem of converting quantitative features into categorical ones (discretization) arises, second, with a large number of levels of input qualitative features their grouping is required. If the variables are strongly correlated, then both tasks should be solved simultaneously. In this case, the optimal quality of the model should be ensured in accordance with a certain criterion.
Existing
The intelligence quotient (IQ) is associated with the quality of people’s life and its duration. Thus,
a study carried out in Scotland, and presented in [
There are other reasons for the influence of IQ on the quality of life, and, as a consequence, on its
duration. Thus, in the article [
On the other hand, recent studies show that high intelligence is associated with increased anxiety
and stress, and can also cause chronic depression [
2020 Copyright for this paper by its authors.
Razumnikova) Razumnikova)
All of the above indicates the relevance of conducting research based on the accumulation and
multivariate statistical analysis of intelligence indicators and its relationship with various time and
demographic factors. A special place in these studies is played by the phenomenon of a gradual
increase of IQ in the 20th century, known as the "Flynn effect". The effect was observed in different
countries and for different categories of test subjects [
At the Novosibirsk State Technical University for 23 years from 1991 to 2013, the intelligence of 1st year students was tested according to the Amthauer method. The sample consisted of 3,677 students of both sexes from various departments of the university in the natural science, technical, and humanitarian fields of knowledge. As a result of the analysis of these data, it becomes possible to establish the influence of factors such as gender and faculty on the IQ of students, as well as to study the temporal dynamics of changes in the IQ of students studying at a Russian university.
For the research, a multivariate analysis of variance was chosen, in which the response (dependent variable) is the final IQ of students, measured on a scale of relationships. Categorical independent features, measured on a nominal scale, are student gender, faculty, and year of study. The aim of the study is to identify the influence of independent factors on the dependent variable - a quantitative indicator of IQ. It is important to assess not only the impact of factors separately, but also their interactions.
When conducting long-term studies of intelligence, it is not always possible to develop an experimental design that makes it possible to obtain optimal estimates of the effects in the ANOVA model, since it is difficult to ensure such conditions under which a similar sample population of individuals would be surveyed every year. In this regard, the analyzed sample is characterized by an uneven distribution of students across faculties and survey years, i.e. in one year, students from one subset of faculties were surveyed, and in the next year, students from another subset. To construct an acceptable analysis of variance model under these conditions, in present paper the method of agglomerative discretization and grouping of input features was developed, investigated and applied.
The article has the following structure. Section 2 provides an overview of the existing discretization methods, substantiates the development of a new method. Section 3 presents the quality criteria investigated in the article for constructing optimal discretization. In Section 4, we describe the ANOVA model used. Section 5 describes the developed discretization algorithm. Section 6 contains the results of the studies of the proposed approach, and section 7 contains their interpretation for solving the multifactor task of studying the IQ of students. In section 8 we provide a conclusion on the work.
A good overview of the current state of research on discretization methods is presented in [
For example, an efficient recursive partitioning algorithm MDLP [13] evaluates the quality based on information gain based on entropy, and the stopping criterion is derived from the principle of minimum length description. The chi-square statistic is popular in the agglomerative merging problem. Algorithms such as ChiMerge [14], Chi2 [15] were built on its basis. Both approaches are designed for classification tasks, that is, they assume that the response is categorical. Therefore, their application to transform a set of input variables in the construction of ANOVA models requires discretizing the response, which can lead to the loss of significant information.
Another group of discretization methods, the so-called wrapping methods, focuses on the quality of the estimated model. Thus, these methods simultaneously solve the learning problem. The existing algorithms are built for classifiers, for example, such simple ones as a majority class voting classifier [16], or more general classifiers such as Naive Bayes [17].
Compared to the problem of discretization, the grouping problem has not been studied so deeply in the literature. A fairly complete overview of grouping methods is presented in [18]. Many commercial data mining packages suggest excluding variables that have too many categories. This approach, however, cannot be considered acceptable in cases where the research interest is to assess the effects of just such variables. Effective grouping methods allow for fewer, more informative categories. This can be done by Sequential Forward Selection method [19]. It is a greedy that initializes a group with the best category and then iteratively adds new categories to this first group. Decision tree algorithms often solve the grouping problem with a greedy heuristic based on bottom-up categorization. The CHAID algorithm [20] uses this greedy approach with a criterion close to the ChiMerge criterion [14]. In [18], a new method of grouping MODL based on the Bayesian approach was proposed, as well as the discretization method MODL [21]. It searches for the most likely grouping model for the given dataset. Optimization is done using a greedy bottom-up algorithm.
Thus, most of the existing supervised discretization algorithms are designed to solve classification problems, that is, for categorical response. They are mainly aimed at improving the quality of predicting the response (quality of classification) [[22],[23]]. Moreover, they are usually univariate. In this regard, it seems relevant to develop an algorithm for the optimal categorization of input features, taking into account their interrelationships, to build a model of analysis of variance. Here categorization includes two tasks: discretization of quantitative variables and grouping of nominal features. Due to the specifics of the practical task, the construction of response predictions is secondary, therefore, the use of criteria such as cross-validation in order to assess the quality of the model and avoid overfitting is limited. The main task was to obtain and interpret estimates of the effects of influencing factors. As a result, we had to resort to goodness-of-fit criteria.
Most often, the quality of a regression model is judged by the coefficient of determination, calculated as
R2 1
ESS
, TSS where ESS is residual sum of squares of the model, TSS is total sum of squares of the model. However, this indicator has an obvious drawback. With increasing complexity of the model (including new variables), it is possible to better describe the response, thereby decreasing ESS and increasing R2 . However, the number of the degrees of freedom decrease, which is in no way taken into account when calculating the coefficient of determination.
To check the significance of the model, the F-statistic is used, calculated as
R2 N m 1 R2 m 1 where N is the number of observations, m is the number of estimated parameters. It takes degrees of freedom into account, so the increase in model complexity must be offset by a sufficient decrease in the residual sum of squares.
Akaike information criterion is often used in the problem of feature selection, for example, in the stepwise regression procedure. It provides a trade-off between goodness of fit and complexity of the model (number of parameters). The Akaike criterion is calculated as follows.
AIC 2m N log ESS .
F ,
It should be borne in mind that with a very large number of categories, building good groupings is difficult because of the risk of overfitting the model. In the extreme case, to avoid overfitting, efficient grouping methods can combine all values into one group, thereby excluding the variable from consideration. In order to prevent such a situation, the stopping criterion must include a condition for the minimum number of categories (for example, two).
For research, the following model of analysis of variance was formulated: yktji k t j kt kj tj ktj ktji , (1) where yktji – i -th observed value corresponding to the IQ level for a student of the k -th sex of the j th faculty in the year t , k is the effect of the k -th sex ( k 1 for male, k 2 for female), t is the effect of the year t , t 1991,..., 2013 , j is the effect of the j -th faculty, j 1,...,11 , kt is the interaction effect of the k -th sex and t -th year, kj is the effect of the interaction of the k -th year and the j -th faculty, tj is the effect of the interaction of the j -th faculty and the t -th year,
ktj is the effect of the interaction of the k -th sex, t -th year and j -th faculty, ktji is a random error.
It is impossible to estimate all the effects in model (1). Usually they resort to reduction. This estimates paired comparisons with some baseline, for example, 2 1 is the influence of female versus male. The first levels of factors are taken as the baseline levels.
The distribution of the studied students is uneven over the years (see table 1). There is a close relationship between the variables Faculty and Year. The chi-square statistic is 8092.9, which indicates a significant correlation at 0.1% significance level. Nevertheless, it should be borne in mind that the original contingency table has a very large dimension (220 degrees of freedom), and, as a consequence, cells with a small number of observations, which negatively affects the correctness of the chi-square test. For confirmation, the correlation ratio was calculated, showing the influence of the faculty for the year. It is 0.192 (F-statistic is equal to 86.9), which also speaks of a significant connection at 0.1% significance level.
The algorithm is developed for the case when it is required to discretize one quantitative variable and group one categorical variable, and the variables are highly correlated. It can be extended to the case when there are more than two variables, but with a large number of variables and levels the curse of dimension arises.
The pseudocode of the algorithm for the optimal categorization of input features, taking into account their interrelationships for constructing an analysis of variance model, is shown in Figure 1.
Q quality x1, x2 repeat for k 0 to T0 1 do if k 0 &T0 2 then x1 merge x1, k, k 1 else x1 x1 if k 0 then Q1 0,0, k quality x1, x2 for i 1 to K0 1 do for j i 1 to K0 do if K0 2 then x2 merge x2 ,i, j else x2 x2
Q1 i, j, k quality x1, x2 end for end for end for Q opt Q1
i, j,k if Q p Q | Q Q then break Q : Q i*, j*, k* arg opt Q1
i, j,k if k* 0 then x1 : merge x1, k*, k* 1 , T0 : T0 1 if i* 0 then x2 : merge x2 ,i*, j* , K0 : K0 1 end repeat return x1, x2
Input: raw data including response values, quantitative factor x1 with T0 levels, and qualitative factor x2 with K0 levels.
The thresholds were selected simultaneously for two variables by the agglomerative merging method. The initial model was built taking into account all available levels of factors. Further, one boundary between the levels was successively removed. For a categorical variable, all possible pairs of factor levels were considered, for a quantitative variable, only adjacent values. In addition, such an option was considered when the levels were not combined. It was assigned an index 0 according to the variable for which the levels were not combined. This is done in case the optimal solution is to combine levels in only one of the variables. If the best value of the quality index corresponding to the optimal solution was achieved, the levels were combined. Then the procedure was repeated until an improvement was obtained.
The function quality x1, x2 returns an indicator of the quality of fitting an ANOVA model of the form (1) (determination coefficient, F-statistic, AIC) depending on the input data.
The function merge x,i, j combines the levels i, j of a variable x so that the number of levels is reduced by one. If the input variable included K levels, then the function returns the transformed variable with K 1 levels numbered from 1 to K 1 .
Since the optimization of the goodness-of-fit criteria can go in different directions (for the determination coefficient and F-statistic it is maximization, for AIC it is minimization), we denoted the optimal value as opt . Wherein Q p Q | Q Q means that Q is no better than Q .
The choice of the determination coefficient as an evaluation criterion did not give any results, since the original partition provided the minimum residual sum of squares. As expected, any merging of intervals led to a decrease in the determination coefficient.
The use of the F-statistic, on the contrary, led to the fact that at each step there was an improvement in the values of the evaluation function. Thus, the work of the algorithm ended only when the intervals could no longer be combined, that is, when there were two categories left for each feature. The faculty of AMCSF stood out in a separate group, as well as 1991. The results of evaluating such a model indicate one significant effect - on the AMCSF, compared to the rest of the faculty, IQ is 6.3 points higher (significant at the 1% level).
The use of the Akaike information criterion made it possible to obtain more interesting results. When applying the algorithm, three groups of faculties were distinguished. From table 1 it is clearly seen that there are years in which some faculties were not covered by the study. This problem was partially solved by discretizing the variable Year. Table 2 shows the proportions of students of faculties of three groups studied in a given range of years. For example, for the first group, there were no periods left when the faculties of this group were not covered by the study. Nevertheless, there is a gap for the second group of faculties in 2009, and for the third - in 2008 and 2010-2013. Therefore, it was not possible to estimate the corresponding effects.
Survey years 1991-1996 1997 1998-1999 2000 2001 2002 2003-2005 2006-2007 2008 2009 2010-2013 0.697 0.556 0.661 0.203 0.296 0.579 0.306 0.394 0.600 0.759 0.556
After discretizing the variables, a model was estimated describing the dependence of IQ on gender, faculty, and year and on their interactions. It turned out that gender has an insignificant effect on the level of intelligence. Therefore, the gender factor was eliminated and the model was re-estimated.
Table 3 provides a summary table with the values of the F-statistic and p-value for 1% significance level. Almost all the effects of the variable Year turned out to be significant at the 5% or 10% level. For the base year, the effect of the faculties of the second group compared to the first was –5.3 and is significant at the 1% level. The effect of faculties of the third group compared to the first for the base year is estimated as 1.2 and is significant at the 10% level. Most of the interactions between the year and the faculty were significant. The general average is estimated at 112.3.
1991-1996 1997 1998-1999 2000 2001 2002
From the point of view of specialization, the distinguished groups of faculties can be divided as follows. The first group is technical and economic faculties, the second is humanitarian and applied faculties, and the third is physics and mathematics. The latter group, on average, is characterized by the highest level of intelligence. Although since 2006 the IQ has dropped and has become comparable to the level of intelligence of students in other faculties. But during this period, a group of students with a physical and mathematical specialization was observed very little (see Table 1): only PEF 2006 (22 students) and 2009 (14 students). Therefore, the decline in IQ may be due to the nonrepresentativeness of the sample.
In the 2000s, there was instability of IQ indicators among students of technical and economic specialization. Growth period 2006-2007 can be explained by the fact that in 2007 only the NSF was observed from this group, which was characterized by higher IQ indices.
For students of humanitarian and applied specialties from 2000 to 2005 in general, there was an increase in intelligence indicators, and then a sharp decline began in 2006-2008. In 2009, the faculties p-value <210-16 <210-16 <210-16 1 2 3 of this group were not studied, so the interaction effect could not be estimated, and the IQ forecast is based only on the main effects. This explains the sharp increase in the IQ forecast in 2009, which cannot be considered reasonable.
Thus, in this work, an analysis of variance model was constructed to study the influence of input factors on the IQ of students. To build a qualitative model, taking into account the specifics of the collected data, a new agglomerative method for discretizing and grouping input features was developed and tested. The interpretation of the obtained estimation results is carried out. In practice, the obtained results of interpretation can be used in the construction of individual educational trajectories, which is one of the key problems of the modern digital educational environment [24].
Future work involves the improvement of the developed algorithm in terms of finding the optimal solution, as well as the development of alternative models for the study of students' IQ with subsequent comparison of the results.
The research is supported by Ministry of Science and Higher Education of Russian Federation (project No. FSUN-2020-0009). 10.References [13] U. Fayyad, K. Irani, Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning, in: Proceedings of the 13th Int’l Joint Conf. Artificial Intelligence (IJCAI), 1993, pp. 1022–1029. [14] R. Kerber, ChiMerge: Discretization of Numeric Attributes, in: Proceedings of the Nat’l Conf.
Artifical Intelligence Am. Assoc. for Artificial Intelligence (AAAI), 1992, pp. 123–128. [15] H. Liu, R. Setiono, Feature Selection via Discretization, IEEE Trans. Knowledge and Data Eng..
9 (1997) 642–645. doi: 10.1109/69.617056 [16] D. Ventura, T. R. Martinez, BRACE: A Paradigm for the Discretization of Continuously Valued Data, in: Proceedings of the Seventh Ann. Florida AI Research Symp. (FLAIRS), 1994, pp. 117– 121. [17] M.J. Pazzani, An Iterative Improvement Approach for the Discretization of Numeric Attributes in Bayesian Classifiers, in: Proceedings of the First Int’l Conf. Knowledge Discovery and Data Mining (KDD), 1995, pp. 228–233. [18] M. A Boullé, Grouping method for categorical attributes having very large number of values, in: International Workshop on Machine Learning and Data Mining in Pattern Recognition, Springer, Berlin, Heidelberg, 2005. pp. 228–242. [19] G. Cestnik, I. Konenenko, I. Bratko, A knowledge-elicitation tool for sophisticated users, in:
Progress in Machine Learning, SigmaPress, Wihnslow, England, 1987. [20] G. V. Kass, An exploratory technique for investigating large quantities of categorical data, Journal of the Royal Statistical Society: Series C (Applied Statistics). 29 (1980) 119–127. doi: 10.2307/2986296 [21] M. Boullé, MODL: a Bayes optimal discretization method for continuous attributes, Machine learning. 65 (2006) 131–165. doi: 10.1007/s10994-006-8364-x [22] W. Huang, Y. Pan, J. Wu, Supervised discretization for optimal prediction, Procedia Computer
Science. 30 (2014) С. 75–80. doi: 10.1016/j.procs.2014.05.383 [23] J. L. Lustgarten, V. Gopalakrishnan, H. Grover, S. Visweswaran, Improving classification performance with discretization on biomedical datasets, in: AMIA annual symposium proceedings, American Medical Informatics Association, 2008. pp. 445–449. [24] D. Parfenov, V. Zaporozhko, M. Lapina, D. Sora, Development and Research of Algorithms for the Formation the Individual Educational Trajectories of Students in the Digital Educational Platform, CEUR Workshop Proceedings. 2494 (2019).