=Paper=
{{Paper
|id=Vol-1738/IWTA_2016_paper6
|storemode=property
|title=Modelling the Relationship Between Learner Autonomy and Cognitive Abilities - Worth the Effort?
|pdfUrl=https://ceur-ws.org/Vol-1738/IWTA_2016_paper6.pdf
|volume=Vol-1738
|authors=Lenka Firtova
|dblpUrl=https://dblp.org/rec/conf/ectel/Firtova16
}}
==Modelling the Relationship Between Learner Autonomy and Cognitive Abilities - Worth the Effort?==
https://ceur-ws.org/Vol-1738/IWTA_2016_paper6.pdf
Modelling the relationship between learner autonomy and
cognitive abilities – worth the effort?
Lenka Firtova
University of Economics, Prague
nám. W. Churchilla 1938/4
130 67 Prague 3
www.scio.cz s.r.o.
Pobržení 34
186 00 Prague 8
+420 739 840 803
lfirtova@scio.cz
ABSTRACT highest possible level of learner autonomy a person can acquire.
More and more schools recognize that as nobody knows what the So far, little research has been dedicated to answering the question
world will look like in the future and what skills we will need, it is of what factors may be underlying people’s predispositions to
necessary to teach children how they can learn effectively and on achieve a high level of learner autonomy. As this is a very
their own throughout life, i.e. to develop their learner autonomy. complex question, this paper only focuses on the relationship
However, despite its importance, little research has been dedicated between learner autonomy and cognitive skills, trying to answer
to exploring the relationship between learner autonomy and other the question of how the level of learner autonomy depends on
domains, such as cognitive skills. Therefore, this paper models the results obtained in the tests of the Czech language and Math.
relationship between the level of learner autonomy and percentiles First, learner autonomy is going to be defined in more detail.
obtained in standardized tests in the Czech language and Math in Second, data acquired thanks to the learning analytics tools
a group of third-graders and fifth-graders in an attempt to find out developed within the LEA’s Box project are going to be presented
whether and how the learner autonomy is related to other skills with the aim to explore the relationship between learner autonomy
taught at schools. and cognitive skills. Finally, suggestions for further research
questions are going to be presented.
CCS Concepts
• Social and professional Topics ➝ Computing
education➝ Computing education programs ➝ Student
assessment 2. LEARNER AUTONOMY
Learner autonomy is a very complex concept whose definition
Keywords differs from domain to domain, from scientist to scientist and
learner autonomy; cognitive skills, Czech language, Math; from teacher to teacher. We may assume that children with a high
primary schools, learning analytics; student assessment; level of learner autonomy display the ability and willingness to
take responsibility for their own process of learning, but such a
1. INTRODUCTION definition is rather vague.
Learner autonomy has recently become a buzz word among
Each researcher focuses on slightly different aspects of learner
teachers as many of them have realized that it is not sufficient to
autonomy. For example, Benson (2001) emphasizes the fact that
teach children specific knowledge and to make them memorize
children’s ability to increase their level of learner autonomy
facts, because these may soon become outdated. The world keeps
depends, to a great extent, on teachers’ ability to guide them
changing fast – what is true today may be proven wrong
through the process of becoming autonomous, while Tarone and
tomorrow, skills that are needed today may not be needed in 5
Yule (1989) stress out that one of the most critical aspects of
years’ time. Therefore, a priority should be given to teaching
learner autonomy is the ability to deal with mistakes that
children how to learn new skills themselves, i.e. to develop their
invariably occur in each process of learning.
learner autonomy. The ability to learn is one of the few skills
which are certainly going to be needed in the future no matter The concept of learner autonomy has been explored mostly in the
what the world will look like and having a high level of learner context of language learning, probably because learning
autonomy may help children acquire new knowledge and skills a language requires in general a higher amount of self-study time
faster and more easily. Furthermore, learner autonomy is likely to compared to other skills. For example, Omaggio (1978) believes
be one of the few skills that are easily transferable from one there are seven main attributes of learner autonomy when learning
domain to another – a person with a high level of learner a new language: having insights into learning styles and strategies,
autonomy will probably find this ability useful no matter if he or taking an active approach to the task at hand, willingness to take
she is a teacher, a scientist, a translator or a painter. risks, being a good guesser, placing importance on both form and
content, willingness to revise and rejects rules that do not apply
However, although teachers may help every child increase his/her
and having a tolerant and outgoing approach to the target
level of learner autonomy, there might be a difference in the
language.
Another important question is whether learner autonomy is a learner autonomy is limited or, at least, that it may cost them more
purely Western construct. Children in tribal communities or less effort to achieve a desired level or learner autonomy compared to
developed societies live in conditions which are more or less children who are naturally gifted in this aspect.
stable throughout their life and they acquire most of what they
need for survival very naturally, because it is easy for them to see 3.2 Data
the importance of such skills. By comparison, children in Western The data comes from a pilot study conducted in June 2016 as
societies often face a situation when they need to learn something a part of the LEA’s Box project. This project is focused on
they do not immediately need or something that takes years to aggregating and visualizing data from different sources in order to
acquire, such as a foreign language, mathematics or physics. A allow teachers to make more competent decisions about their
higher level of reflection over the process of learning and more students’ learning process.
long-term thinking is therefore required, both of which are
One of the main benefits of LEA’s Box for teachers is a tool
undoubtedly related to the concept of learner autonomy.
which allows for a comparison of self-assessment, teacher’s
Due to the lack of universal definition of learner autonomy, a new assessment and standardized assessment. In the pilot study,
definition has been made for the purpose of this paper and further self-assessment was omitted, but it is going to form part of the
research. Learner autonomy has been defined as a concept next study which is to be conducted in September. Therefore, data
consisting of seven domains. collected in June 2016 only come from two sources: standardized
tests and teacher’s assessment. The standardized tests measured
1. The ability to set meaningful and achievable goals: this children’s knowledge and abilities in the Czech language and
means a child is able set long-term and short-term goals and Math and the results are in the form of percentiles. As for
evaluate whether they are attainable and realistic. teacher’s assessment, this focused on learner autonomy and was
2. The ability to find ways to reach a goal: this means a child is done using a table containing a 4-point scale (0 to 3) for each of
able to determine several possible strategies to reach a goal the seven domains described in the previous section. For each
and pick up the most appropriate one, can make a realistic domain, there was a description of what behaviour it is possible to
estimate of the time and effort needed to reach a goal and has observe at a child at different levels. The teacher’s task was to
a good sense of how tasks and activities depend on each assign each child with a number from 0 to 3, where 0 means the
other. child performs poorly in that particular domain while 3 means the
child performs exceptionally. For the final “autonomy score” the
3. The ability to identify a source of failure: this means that in seven numbers were simply summed, therefore it was possible to
the case of failure, a child is able to explain what went wrong, obtain a score ranging from 0 to 21.
and can distinguish between what was caused by him/her and
what was caused by external factors. Altogether, the data contains values for 62 children: 29 attending
the third grade, which, in the Czech educational system,
4. The ability to learn from one’s mistakes: this means a child corresponds to ages of 8 to 9, and 33 attending the fifth grade,
avoids past mistakes and is able to anticipate possible future which corresponds to ages 10 to 11.
mistakes and take appropriate measures.
All of them were evaluated by the same learner autonomy
5. The ability to perceive mistakes as an opportunity for questionnaire. As for the standardized tests, 28 third-graders and
improvement: this means a child does not become frustrated 33 fifth-graders took a test in the Czech language while 26
by mistakes but rather perceives them as a source of new third-graders and 18 fifth-graders took a test in Math, these two
knowledge and experience and tries to improve. tests being different for each grade and testing age-appropriate
6. The ability to work independently towards reaching a goal: skills (see Table 1 for a summary of the number of study
this means that a child can independently work towards subjects).
reaching a goal, can handle distractions and has
an age-appropriate attention span. Table 1. Number of children evaluated in different subjects
7. The ability to evaluate if a goal has been reached: this means Grade autonomy Czech Math
that a child can determine if a task has been successfully
completed and alternatively come up with ideas how to third 29 28 26
complete it. fifth 33 33 18
3. CASE STUDY Table 2 shows average values or learner autonomy score in the
3.1 Aim of the study third and fifth grade for each of the seven domains described
above. It can be seen that fifth-graders achieved a higher score at
The aim of the study presented in this paper is to determine
every single domain compared to third-graders. The smallest
whether there is any relationship between the level of learner
difference can be seen in domain 2, the ability to find ways to
autonomy, as described above, and cognitive abilities as measured
reach a goal, while the largest difference can be seen at domain 6,
by standardized tests.
the ability to work independently towards reaching a goal,
Most teachers acknowledge that cognitive abilities vary among suggesting that younger children may still rely more on being
children and that they are, to a certain extent, hereditary, while at “pushed” while working or studying.
the same time it is often taken for granted that the ability to
achieve a high level of learner autonomy is universal. However,
should these two be related, teachers might need to accept the fact
that for some children, the ability to achieve a high level of
Table 2. Average score in each of the seven domains of learner learner autonomy evaluation (0 to 21) and the percentiles obtained
autonomy in the standardized tests. The relationships appear to be linear,
therefore, we ran a simple linear regression with percentiles being
Grade 1 2 3 4 5 6 7 Total the dependent variable and autonomy scores being the explanatory
Third 1,38 1,17 1,28 0,86 1,28 1,10 1,38 8,45 variable. Then, the resulting regression lines have been added to
Fifth 1,52 1,24 1,58 1,15 1,33 1,45 1,64 9,91
the graphs, with the following coefficients:
̂
3𝑟𝑑 𝑔𝑟𝑎𝑑𝑒, 𝐶𝑧𝑒𝑐ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 32.55 + 2.34 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠
3.3 Results
Let us now explore the relationship between the level of learner ̂
5𝑡ℎ 𝑔𝑟𝑎𝑑𝑒, 𝐶𝑧𝑒𝑐ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 26.76 + 3.03 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠
autonomy and percentiles in the Czech language and Math ̂
3𝑟𝑑 𝑔𝑟𝑎𝑑𝑒, 𝑀𝑎𝑡ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 35.09 + 2.69 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠
standardized tests.
̂
5𝑡ℎ 𝑔𝑟𝑎𝑑𝑒, 𝑀𝑎𝑡ℎ: 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 = 22.94 + 2.56 ∙ 𝑎𝑢𝑡𝑜𝑛𝑜𝑚𝑦_𝑝𝑜𝑖𝑛𝑡𝑠
Table 3 shows correlations between the percentiles obtained in the
Czech and Math tests (the group – third or fifth grade – being
denoted by the number in brackets) with each of the seven All slope coefficients are statistically significant (p-value < 0.05).
domains of learner autonomy as well as with the total “autonomy The regression coefficients suggest that, ceteris paribus, for each
score”. increase in the learner autonomy score by one point, there is, on
average, a two to three percentile increase in the results obtained
Table 3: Correlation of tests results and learner autonomy
in the cognitive tests, depending on grade and subject.
points
Correlation 1 2 3 4 5 6 7 Total
Czech (3) 0.34 0.41 0.52 0.42 0.34 0.26 0.42 0.45 Figure 1: relationships between the total number of autonomy
points and percentiles for different grades and subjects
Czech (5) 0.59 0.52 0.66 0.64 0.51 0.74 0.67 0.69
Math (3) 0.39 0.37 0.55 0.53 0.33 0.44 0.42 0.50
Math (5) 0.28 0.29 0.63 0.42 0.61 0.46 0.52 0.53
The correlation coefficients vary for grades, subjects and domains,
but there is undeniably a strong positive relationship between
what is measured by standardized tests in the Czech language and
Math and what is evaluated in the learner autonomy questionnaire.
The highest correlations can be seen in domain 3 (the ability to
identify a source of failure), 4 (the ability to learn from one’s
mistakes) and 7 (the ability to evaluate if a goal has been
reached), although there is also a particularly high correlation
between the fifth graders’ results in the Czech language test and
their results in domain 6 (the ability to work independently
towards reaching a goal). A possible explanation may be that
these domains require a higher level of cognitive and
metacognitive skills which are also necessary in cognitive-based
tests. In comparison, domains 1 (the ability to set meaningful and
achievable goals) and 2 (the ability to find ways to reach a goal),
where the correlations seem to be lower, may be more related to
a child’s personality and motivation (note that it may be difficult
to differentiate between a child’s ability to set goals and ways to
reach them and his/her willingness to do so). Domain 5 (the
ability to perceive mistakes as an opportunity for improvement)
may be more related to personality as well, even though the
correlations here are quite high for five-graders, suggesting that
those who see mistakes as a source of improvement may make
more progress in cognitive-based domains over the course of time
and thus subsequently outperform those who score lower in this
domain. A similar assumption may hold true for domain 6 (the
ability to work independently towards reaching a goal), which is,
once again, somewhat related to personality and one’s level of
self-control: those who score high in this domain may make more
significant progress over the course of time compared to those
with a low score, which may explain why the correlations here are
higher in the older group.
Let us now explore graphs depicting the relationships under
scrutiny. Figure 1 shows, for each grade and subject, the
relationship between the total number of points obtained in the
For the domains which have been previously said to be possibly
dependent on personality rather than on one’s level of cognitive
skills, the slope coefficients have in some cases not proven to be
statistically significant.
As an example, figure 2 shows a graphical representation of the
relationship between the results in the Czech language test and the
number of points obtained in domain 3 (the ability to identify a
source of failure).
Figure 2: relationship between the number of autonomy
points obtained in domain 3 and percentile obtained in the
Czech language test
Table 4 shows regression coefficients for each of the seven
domains of learner autonomy separately, i.e. the relationship
between the number of points obtained in a particular domain
(explanatory variable) and the percentile obtained in the
standardized test (dependent variable), with bold values
representing statistically significant parameters (p-value < 0.05).
This time, the values of explanatory variables only range from 0
to 3, so the slope coefficients associated with a 1-point increase
are higher. The highest slope coefficient is 23.13 (fifth grade,
Czech language, domain 4), suggesting that in the older group,
a 1-point increase in this domain is, ceteris paribus, associated
with, on average, a 23.13 percentile increase in the percentile
obtained in the Czech language test.
Table 4: Coefficients of regressions lines with individual
learner autonomy domains as explanatory variables
dep.variable domain (explanatory variable)
4. DISCUSSION
Czech (3) 1 2 3 4 5 6 7 4.1 Summary
intercept 38.69 37.27 32.46 38.89 36.60 43.44 33.93 This study explored the relationship between the level of learner
slope 9.78 12.77 15.45 15.05 12.57 8.03 13.55 autonomy as defined in the second section of this paper and the
Czech (5) 1 2 3 4 5 6 7
results obtained in standardized tests in two different subjects, the
Czech language and Math. The children evaluated for the purpose
intercept 35.17 38.66 28.58 30.19 36.23 28.45 30.12 of this study comprised third graders and fifth graders.
slope 14.29 14.62 17.92 23.13 15.43 19.50 16.32
It has been shown there is a significant relationship between these
Math (3) 1 2 3 4 5 6 7 two variables regardless of grade and subject. When split into
intercept 42.07 44.71 36.56 41.70 42.53 43.55 39.25 individual domains, it was shown that the relationship between
learner autonomy and the results in cognitive tests is stronger for
slope 11.46 11.46 16.53 19.34 12.45 13.22 13.74
some domains than for others, possibly because some domains
Math (5) 1 2 3 4 5 6 7 require a higher level of cognitive and metacognitive skills (e.g.
intercept 35.36 35.90 22.30 28.81 22.75 27.71 24.51 the ability to analyze a source of error or the ability to evaluate if
slope 7.65 9.15 16.75 16.75 19.55 12.85 14.57
a goal has been reached), while others may be more related to
motivation or certain aspects of personality (resilience,
self-control etc.).
However, one should interpret the results presented in this paper 4.3 Suggestions for further research
with care. It is not possible to say that a high level of learner As mentioned above, this study was limited to a handful of
autonomy leads to a high level of cognitive skills or vice versa – subjects and age groups. It would be especially beneficial to
the causal relationship between these two variables is unclear and explore the relationship of learner autonomy with the results of
both of them may be strongly influenced by an entirely different scholastic aptitude tests (such as SAT) or IQ tests, and with the
variable. results of personality tests. It would also be useful to extend the
In any case, when trying to increase a child’s level of learner research to more age groups and to include self-assessment.
autonomy, which is a trend in many schools, it is necessary to take
into account the child’s level of cognitive development and to 5. ACKNOWLEDGMENTS
realize that these two may be related. The paper was written on the basis of research conducted as part
of the LEA’s Box Project, which is supported by the European
4.2 Limitations Commission under the 7th framework programme; grant no.
As this was a pilot study, not too much data was available. Also, 619762.
the results of this study are limited to only two age groups and
two subjects (the Czech language and Math).
Another important limitation is the definition of learner autonomy 6. REFERENCES
itself. The definition suggested in this paper is not universal –
each researcher or psychologist may define it in a different way,
[1] Benson, P., 2001. Teaching and researching learner
depending on what they consider to be a priority and what their autonomy. Harlow: Pearson Education.
goal is. When defining the concept of learner autonomy, it is
crucial to choose such domains/aspects that are easily describable [2] Omaggio, A.C., 1978. Successful language learners: What do
and observable and simple to evaluate, yet that cover all parts of we know about them. ERIC/CLL News Bulletin, 5, pp.2-3.
learner autonomy deemed important, which is not an easy task. [3] Tarone, E., Yule, G. and Yule, G., 1989. Focus on the
Furthermore, evaluating the level of learner autonomy may also language learner: Approaches to identifying and meeting the
be challenging for teachers as it is always somewhat subjective. needs of second language learners (pp. 34-39). Oxford:
Oxford University Press.