Modelling the relationship between learner autonomy and cognitive abilities – worth the effort? Lenka Firtova University of Economics, Prague nám. W. Churchilla 1938/4 130 67 Prague 3 www.scio.cz s.r.o. Pobržení 34 186 00 Prague 8 +420 739 840 803 lfirtova@scio.cz ABSTRACT highest possible level of learner autonomy a person can acquire. More and more schools recognize that as nobody knows what the So far, little research has been dedicated to answering the question world will look like in the future and what skills we will need, it is of what factors may be underlying people’s predispositions to necessary to teach children how they can learn effectively and on achieve a high level of learner autonomy. As this is a very their own throughout life, i.e. to develop their learner autonomy. complex question, this paper only focuses on the relationship However, despite its importance, little research has been dedicated between learner autonomy and cognitive skills, trying to answer to exploring the relationship between learner autonomy and other the question of how the level of learner autonomy depends on domains, such as cognitive skills. Therefore, this paper models the results obtained in the tests of the Czech language and Math. relationship between the level of learner autonomy and percentiles First, learner autonomy is going to be defined in more detail. obtained in standardized tests in the Czech language and Math in Second, data acquired thanks to the learning analytics tools a group of third-graders and fifth-graders in an attempt to find out developed within the LEA’s Box project are going to be presented whether and how the learner autonomy is related to other skills with the aim to explore the relationship between learner autonomy taught at schools. and cognitive skills. Finally, suggestions for further research questions are going to be presented. CCS Concepts • Social and professional Topics ➝ Computing education➝ Computing education programs ➝ Student assessment 2. LEARNER AUTONOMY Learner autonomy is a very complex concept whose definition Keywords differs from domain to domain, from scientist to scientist and learner autonomy; cognitive skills, Czech language, Math; from teacher to teacher. We may assume that children with a high primary schools, learning analytics; student assessment; level of learner autonomy display the ability and willingness to take responsibility for their own process of learning, but such a 1. INTRODUCTION definition is rather vague. Learner autonomy has recently become a buzz word among Each researcher focuses on slightly different aspects of learner teachers as many of them have realized that it is not sufficient to autonomy. For example, Benson (2001) emphasizes the fact that teach children specific knowledge and to make them memorize children’s ability to increase their level of learner autonomy facts, because these may soon become outdated. The world keeps depends, to a great extent, on teachers’ ability to guide them changing fast – what is true today may be proven wrong through the process of becoming autonomous, while Tarone and tomorrow, skills that are needed today may not be needed in 5 Yule (1989) stress out that one of the most critical aspects of years’ time. Therefore, a priority should be given to teaching learner autonomy is the ability to deal with mistakes that children how to learn new skills themselves, i.e. to develop their invariably occur in each process of learning. learner autonomy. The ability to learn is one of the few skills which are certainly going to be needed in the future no matter The concept of learner autonomy has been explored mostly in the what the world will look like and having a high level of learner context of language learning, probably because learning autonomy may help children acquire new knowledge and skills a language requires in general a higher amount of self-study time faster and more easily. Furthermore, learner autonomy is likely to compared to other skills. For example, Omaggio (1978) believes be one of the few skills that are easily transferable from one there are seven main attributes of learner autonomy when learning domain to another – a person with a high level of learner a new language: having insights into learning styles and strategies, autonomy will probably find this ability useful no matter if he or taking an active approach to the task at hand, willingness to take she is a teacher, a scientist, a translator or a painter. risks, being a good guesser, placing importance on both form and content, willingness to revise and rejects rules that do not apply However, although teachers may help every child increase his/her and having a tolerant and outgoing approach to the target level of learner autonomy, there might be a difference in the language. Another important question is whether learner autonomy is a learner autonomy is limited or, at least, that it may cost them more purely Western construct. Children in tribal communities or less effort to achieve a desired level or learner autonomy compared to developed societies live in conditions which are more or less children who are naturally gifted in this aspect. stable throughout their life and they acquire most of what they need for survival very naturally, because it is easy for them to see 3.2 Data the importance of such skills. By comparison, children in Western The data comes from a pilot study conducted in June 2016 as societies often face a situation when they need to learn something a part of the LEA’s Box project. This project is focused on they do not immediately need or something that takes years to aggregating and visualizing data from different sources in order to acquire, such as a foreign language, mathematics or physics. A allow teachers to make more competent decisions about their higher level of reflection over the process of learning and more students’ learning process. long-term thinking is therefore required, both of which are One of the main benefits of LEA’s Box for teachers is a tool undoubtedly related to the concept of learner autonomy. which allows for a comparison of self-assessment, teacher’s Due to the lack of universal definition of learner autonomy, a new assessment and standardized assessment. In the pilot study, definition has been made for the purpose of this paper and further self-assessment was omitted, but it is going to form part of the research. Learner autonomy has been defined as a concept next study which is to be conducted in September. Therefore, data consisting of seven domains. collected in June 2016 only come from two sources: standardized tests and teacher’s assessment. The standardized tests measured 1. The ability to set meaningful and achievable goals: this children’s knowledge and abilities in the Czech language and means a child is able set long-term and short-term goals and Math and the results are in the form of percentiles. As for evaluate whether they are attainable and realistic. teacher’s assessment, this focused on learner autonomy and was 2. The ability to find ways to reach a goal: this means a child is done using a table containing a 4-point scale (0 to 3) for each of able to determine several possible strategies to reach a goal the seven domains described in the previous section. For each and pick up the most appropriate one, can make a realistic domain, there was a description of what behaviour it is possible to estimate of the time and effort needed to reach a goal and has observe at a child at different levels. The teacher’s task was to a good sense of how tasks and activities depend on each assign each child with a number from 0 to 3, where 0 means the other. child performs poorly in that particular domain while 3 means the child performs exceptionally. For the final “autonomy score” the 3. The ability to identify a source of failure: this means that in seven numbers were simply summed, therefore it was possible to the case of failure, a child is able to explain what went wrong, obtain a score ranging from 0 to 21. and can distinguish between what was caused by him/her and what was caused by external factors. Altogether, the data contains values for 62 children: 29 attending the third grade, which, in the Czech educational system, 4. The ability to learn from one’s mistakes: this means a child corresponds to ages of 8 to 9, and 33 attending the fifth grade, avoids past mistakes and is able to anticipate possible future which corresponds to ages 10 to 11. mistakes and take appropriate measures. All of them were evaluated by the same learner autonomy 5. The ability to perceive mistakes as an opportunity for questionnaire. As for the standardized tests, 28 third-graders and improvement: this means a child does not become frustrated 33 fifth-graders took a test in the Czech language while 26 by mistakes but rather perceives them as a source of new third-graders and 18 fifth-graders took a test in Math, these two knowledge and experience and tries to improve. tests being different for each grade and testing age-appropriate 6. The ability to work independently towards reaching a goal: skills (see Table 1 for a summary of the number of study this means that a child can independently work towards subjects). reaching a goal, can handle distractions and has an age-appropriate attention span. Table 1. Number of children evaluated in different subjects 7. The ability to evaluate if a goal has been reached: this means Grade autonomy Czech Math that a child can determine if a task has been successfully completed and alternatively come up with ideas how to third 29 28 26 complete it. fifth 33 33 18 3. CASE STUDY Table 2 shows average values or learner autonomy score in the 3.1 Aim of the study third and fifth grade for each of the seven domains described above. It can be seen that fifth-graders achieved a higher score at The aim of the study presented in this paper is to determine every single domain compared to third-graders. The smallest whether there is any relationship between the level of learner difference can be seen in domain 2, the ability to find ways to autonomy, as described above, and cognitive abilities as measured reach a goal, while the largest difference can be seen at domain 6, by standardized tests. the ability to work independently towards reaching a goal, Most teachers acknowledge that cognitive abilities vary among suggesting that younger children may still rely more on being children and that they are, to a certain extent, hereditary, while at “pushed” while working or studying. the same time it is often taken for granted that the ability to achieve a high level of learner autonomy is universal. However, should these two be related, teachers might need to accept the fact that for some children, the ability to achieve a high level of Table 2. Average score in each of the seven domains of learner learner autonomy evaluation (0 to 21) and the percentiles obtained autonomy in the standardized tests. The relationships appear to be linear, therefore, we ran a simple linear regression with percentiles being Grade 1 2 3 4 5 6 7 Total the dependent variable and autonomy scores being the explanatory Third 1,38 1,17 1,28 0,86 1,28 1,10 1,38 8,45 variable. Then, the resulting regression lines have been added to Fifth 1,52 1,24 1,58 1,15 1,33 1,45 1,64 9,91 the graphs, with the following coefficients: ̂ 3𝑟𝑑 𝑔𝑟𝑎𝑑𝑒, 𝐶𝑧𝑒𝑐ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 32.55 + 2.34 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠 3.3 Results Let us now explore the relationship between the level of learner ̂ 5𝑡ℎ 𝑔𝑟𝑎𝑑𝑒, 𝐶𝑧𝑒𝑐ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 26.76 + 3.03 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠 autonomy and percentiles in the Czech language and Math ̂ 3𝑟𝑑 𝑔𝑟𝑎𝑑𝑒, 𝑀𝑎𝑡ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 35.09 + 2.69 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠 standardized tests. ̂ 5𝑡ℎ 𝑔𝑟𝑎𝑑𝑒, 𝑀𝑎𝑡ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 22.94 + 2.56 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠 Table 3 shows correlations between the percentiles obtained in the Czech and Math tests (the group – third or fifth grade – being denoted by the number in brackets) with each of the seven All slope coefficients are statistically significant (p-value < 0.05). domains of learner autonomy as well as with the total “autonomy The regression coefficients suggest that, ceteris paribus, for each score”. increase in the learner autonomy score by one point, there is, on average, a two to three percentile increase in the results obtained Table 3: Correlation of tests results and learner autonomy in the cognitive tests, depending on grade and subject. points Correlation 1 2 3 4 5 6 7 Total Czech (3) 0.34 0.41 0.52 0.42 0.34 0.26 0.42 0.45 Figure 1: relationships between the total number of autonomy points and percentiles for different grades and subjects Czech (5) 0.59 0.52 0.66 0.64 0.51 0.74 0.67 0.69 Math (3) 0.39 0.37 0.55 0.53 0.33 0.44 0.42 0.50 Math (5) 0.28 0.29 0.63 0.42 0.61 0.46 0.52 0.53 The correlation coefficients vary for grades, subjects and domains, but there is undeniably a strong positive relationship between what is measured by standardized tests in the Czech language and Math and what is evaluated in the learner autonomy questionnaire. The highest correlations can be seen in domain 3 (the ability to identify a source of failure), 4 (the ability to learn from one’s mistakes) and 7 (the ability to evaluate if a goal has been reached), although there is also a particularly high correlation between the fifth graders’ results in the Czech language test and their results in domain 6 (the ability to work independently towards reaching a goal). A possible explanation may be that these domains require a higher level of cognitive and metacognitive skills which are also necessary in cognitive-based tests. In comparison, domains 1 (the ability to set meaningful and achievable goals) and 2 (the ability to find ways to reach a goal), where the correlations seem to be lower, may be more related to a child’s personality and motivation (note that it may be difficult to differentiate between a child’s ability to set goals and ways to reach them and his/her willingness to do so). Domain 5 (the ability to perceive mistakes as an opportunity for improvement) may be more related to personality as well, even though the correlations here are quite high for five-graders, suggesting that those who see mistakes as a source of improvement may make more progress in cognitive-based domains over the course of time and thus subsequently outperform those who score lower in this domain. A similar assumption may hold true for domain 6 (the ability to work independently towards reaching a goal), which is, once again, somewhat related to personality and one’s level of self-control: those who score high in this domain may make more significant progress over the course of time compared to those with a low score, which may explain why the correlations here are higher in the older group. Let us now explore graphs depicting the relationships under scrutiny. Figure 1 shows, for each grade and subject, the relationship between the total number of points obtained in the For the domains which have been previously said to be possibly dependent on personality rather than on one’s level of cognitive skills, the slope coefficients have in some cases not proven to be statistically significant. As an example, figure 2 shows a graphical representation of the relationship between the results in the Czech language test and the number of points obtained in domain 3 (the ability to identify a source of failure). Figure 2: relationship between the number of autonomy points obtained in domain 3 and percentile obtained in the Czech language test Table 4 shows regression coefficients for each of the seven domains of learner autonomy separately, i.e. the relationship between the number of points obtained in a particular domain (explanatory variable) and the percentile obtained in the standardized test (dependent variable), with bold values representing statistically significant parameters (p-value < 0.05). This time, the values of explanatory variables only range from 0 to 3, so the slope coefficients associated with a 1-point increase are higher. The highest slope coefficient is 23.13 (fifth grade, Czech language, domain 4), suggesting that in the older group, a 1-point increase in this domain is, ceteris paribus, associated with, on average, a 23.13 percentile increase in the percentile obtained in the Czech language test. Table 4: Coefficients of regressions lines with individual learner autonomy domains as explanatory variables dep.variable domain (explanatory variable) 4. DISCUSSION Czech (3) 1 2 3 4 5 6 7 4.1 Summary intercept 38.69 37.27 32.46 38.89 36.60 43.44 33.93 This study explored the relationship between the level of learner slope 9.78 12.77 15.45 15.05 12.57 8.03 13.55 autonomy as defined in the second section of this paper and the Czech (5) 1 2 3 4 5 6 7 results obtained in standardized tests in two different subjects, the Czech language and Math. The children evaluated for the purpose intercept 35.17 38.66 28.58 30.19 36.23 28.45 30.12 of this study comprised third graders and fifth graders. slope 14.29 14.62 17.92 23.13 15.43 19.50 16.32 It has been shown there is a significant relationship between these Math (3) 1 2 3 4 5 6 7 two variables regardless of grade and subject. When split into intercept 42.07 44.71 36.56 41.70 42.53 43.55 39.25 individual domains, it was shown that the relationship between learner autonomy and the results in cognitive tests is stronger for slope 11.46 11.46 16.53 19.34 12.45 13.22 13.74 some domains than for others, possibly because some domains Math (5) 1 2 3 4 5 6 7 require a higher level of cognitive and metacognitive skills (e.g. intercept 35.36 35.90 22.30 28.81 22.75 27.71 24.51 the ability to analyze a source of error or the ability to evaluate if slope 7.65 9.15 16.75 16.75 19.55 12.85 14.57 a goal has been reached), while others may be more related to motivation or certain aspects of personality (resilience, self-control etc.). However, one should interpret the results presented in this paper 4.3 Suggestions for further research with care. It is not possible to say that a high level of learner As mentioned above, this study was limited to a handful of autonomy leads to a high level of cognitive skills or vice versa – subjects and age groups. It would be especially beneficial to the causal relationship between these two variables is unclear and explore the relationship of learner autonomy with the results of both of them may be strongly influenced by an entirely different scholastic aptitude tests (such as SAT) or IQ tests, and with the variable. results of personality tests. It would also be useful to extend the In any case, when trying to increase a child’s level of learner research to more age groups and to include self-assessment. autonomy, which is a trend in many schools, it is necessary to take into account the child’s level of cognitive development and to 5. ACKNOWLEDGMENTS realize that these two may be related. The paper was written on the basis of research conducted as part of the LEA’s Box Project, which is supported by the European 4.2 Limitations Commission under the 7th framework programme; grant no. As this was a pilot study, not too much data was available. Also, 619762. the results of this study are limited to only two age groups and two subjects (the Czech language and Math). Another important limitation is the definition of learner autonomy 6. REFERENCES itself. The definition suggested in this paper is not universal – each researcher or psychologist may define it in a different way, [1] Benson, P., 2001. Teaching and researching learner depending on what they consider to be a priority and what their autonomy. Harlow: Pearson Education. goal is. When defining the concept of learner autonomy, it is crucial to choose such domains/aspects that are easily describable [2] Omaggio, A.C., 1978. Successful language learners: What do and observable and simple to evaluate, yet that cover all parts of we know about them. ERIC/CLL News Bulletin, 5, pp.2-3. learner autonomy deemed important, which is not an easy task. [3] Tarone, E., Yule, G. and Yule, G., 1989. Focus on the Furthermore, evaluating the level of learner autonomy may also language learner: Approaches to identifying and meeting the be challenging for teachers as it is always somewhat subjective. needs of second language learners (pp. 34-39). Oxford: Oxford University Press.