=Paper=
{{Paper
|id=Vol-2497/paper3
|storemode=property
|title=Elaborative Feedback in L2 Reading Games
|pdfUrl=https://ceur-ws.org/Vol-2497/paper3.pdf
|volume=Vol-2497
|authors=Matthew Pattemore,Roger Gilabert,Judit Serra
|dblpUrl=https://dblp.org/rec/conf/gamilearn/PattemoreGS19
}}
==Elaborative Feedback in L2 Reading Games==
https://ceur-ws.org/Vol-2497/paper3.pdf
Elaborative Feedback in L2 Reading Games
Matthew Pattemore Roger Gilabert Judit Serra
University of Barcelona University of Barcelona University of Barcelona
Barcelona, Spain Barcelona, Spain Barcelona, Spain
matthew.pattemore@ub.edu rogergilabert@ub.edu juditserra@ub.edu
ABSTRACT (SLA) feedback has frequently been investigated in terms
In combination with instruction, feedback can assist of written CF or oral CF (see Li, 2010 for a general meta-
students in comprehension, engagement, and strategy analysis; see Lyster & Saito, 2010 for a meta-analysis of
development in second language acquisition. Outcome oral CF; Kang & Han, 2015 for a meta-analysis of written
feedback allows learners to understand whether they are CF). This study focuses instead on the relatively newer
correct or incorrect. Elaborative feedback can be used to subject of digital or audiovisual CF.
remind learners of the underlying processes behind the
activities or encourage them to use learning strategies. This This research is being conducted within the iRead project,
study is part of a European project called iRead which aims which aims to develop a suite of educational software for
to create and test personalised learning technologies to android tablet computers, aimed at helping children to
assist primary school children in reading development. We develop their reading skills. It is currently localised into
investigate to what extent elaborative feedback, rather than four languages – English, Spanish, German, and Greek. The
solely outcome feedback, is taken up by EFL learners. software covers over 250 aspects of phonology,
Data analytics from the games will be collected, including morphology, morphosyntax, and syntax, and consists of
response correctness, feedback type received, and the learning games, a graded e-reader, and an analytics
impact of the feedback on subsequent responses. Results application. The software is presently only available for
will be discussed in light of both serious games and SLA researchers, but aims to have a public release after trials
theories interested in the effects of feedback on second have been conducted. This study will be focusing on the use
language reading development. of the learning games application by 10- and 11-year-old
English as a foreign language (EFL) learners in Spain, and
Author Keywords how the feedback in those games is designed and what
Serious games; feedback; second language acquisition; impact it may have on their game behavior and reading
reading skills. development.
ACM Classification Keywords These types of games are known as serious games as they
J.5. Linguistics; K.3.1. Computer-assisted instruction are designed with a prevalent gaming element, but include
(CAI). the intent to improve learning (Ratan & Ritterfield, 2009),
INTRODUCTION as opposed to games simply for entertainment purposes.
Feedback is information given to learners to allow them to Ritterfield, Cody, and Vorderer (2009) define serious
evaluate their output. Corrective feedback (CF) is games as intrinsically motivating because they are fun to
information given to help learners fix errors in their output play, but containing content that is complex enough to
and/or understanding. Based on cognitive interactionist provide learning opportunities. This differs from
theories of language acquisition (e.g. Long, 1996), there is a ‘gamification’ as, in serious games, learning is expected to
belief that CF assists learners to notice and attend to happen within the game, while in gamification the gamified
language errors in a manner that is beneficial for their long- elements are intended to encourage learning external to the
term language development. Many studies have found that, game through greater engagement and motivation (Landers
in combination with instruction, feedback can assist 2015). The iRead project, then, can be said to be gamifying
students in comprehension, engagement, and strategy the reading development process by introducing (among
development. In terms of Second Language Acquisition other things) a serious literacy game. Serious games are a
multimodal environment where users are engaged in
Copyright © 2019 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
multiple cognitive tasks concurrently (Johnson et. al. 2017).
In: J. Arnedo-Moreno, C.S. González, A. Mora (eds.): Proceedings of the CORRECTIVE FEEDBACK
3rd International Symposium on Gamification and Games for Learning
(GamiLearn’19), Barcelona, Spain, 22-10-2019, published at http://ceur-
Rod Ellis (2010) laid out a framework for investigating CF,
ws.org which can be easily adapted to include digital feedback. He
listed six areas of study: type of feedback; individual
difference factors; contextual factors; engagement with
feedback, and learning outcomes.
Form of feedback synthetic voices have been found to increase processing
Regarding form of feedback, researchers have proposed demands (Sinatra et. al. 2013). This all adds up to a
many categories of feedback (see e.g. Hattie & Timperley, cognitively demanding environment in which working
2007). A central distinction is between outcome feedback memory may play a role in facilitating feedback use.
and elaborative feedback (Johnson et. al., 2017). Outcome
Engagement with feedback
feedback is simply when the learner is informed if their
Engagement with feedback is another dimension that will be
answer is correct or incorrect. In addition to mere
investigated. Affective engagement will be studied through
knowledge of result, outcome feedback can provide direct
smaller scale studies involving think aloud protocols, as
or indirect forms of feedback (also called input-providing or
seen in Hookham et. al. (2016), in order to find out what the
output-pushing feedback). Direct/Input-providing feedback
learners perceive from the feedback and how they feel
identifies what the right answer should have been (e.g.
about it, and to what extent it assists or detracts from the
recasts), while indirect/output-pushing feedback highlight
learning experience. Finally, behavioural engagement will
exactly where the error was made, and encourages the
be observed through the data provided by the software, to
learner to self-correct. There is evidence (Yang & Lyster
show whether, and in what way, learners take up the
2010, Lyster & Saito 2010) that indirect/output-pushing
feedback, as demonstrated by Smith et. al. (2016).
feedback better assists long-term development if learners
have previously been introduced to the forms. Learning outcomes
Finally, an analysis of feedback would be sorely lacking
Elaborative feedback (also known as process feedback) without a measure of Ellis’ final dimension, learning
gives learners more details, such as explaining why an outcomes. Research from the Language and Reading
answer is correct or incorrect, providing information about Research Consortium (2015) has shown that around 90% of
the specific concepts or processes involved in the task, or the variance in reading comprehension can be explained by
guiding learners to particular strategies that will help them word recognition and listening comprehension skills, both
with the task. These types of feedback are not mutually modulated by vocabulary. Using pre-, post-, and delayed
exclusive, and indeed it is difficult to give elaborative post-tests, in conjunction with the data provided by the
feedback without also giving outcome feedback. There is software, we will have a measure of how effective the
little evidence of the effectiveness of different types of feedback is in creating reading development gains.
feedback in serious games for SLA, and Benton et. al.
(2018) found that many commercially available early There is a lack of studies on the effects of different types of
literacy games contained limited elaborative feedback. feedback in the context of serious games for SLA. This is
However, there is some evidence from the fields of botany true both for within game learning (how well does the
(Moreno, 2004) and electrical circuits (Mayer & Johnson, feedback help learners progress through the game) and
2010) that providing outcome and elaborative feedback external learning (how well has the feedback assisted the
together in a game helps to develop learning more than just learner in their overall reading development). There is
outcome feedback. limited understanding of how learners engage with
feedback in the context of serious games for SLA, and how
Complexity of feedback
that might be moderated by individual differences between
One additional aspect that might be profitably explored is
the users, such as working memory. These are the gaps that
complexity of feedback. In terms of feedback complexity
this research project aims to help fill.
generally, Shute (2008) points out that there is no evidence
for more complex feedback providing greater results, and AIMS AND METHODOLOGY
some evidence that it worsens results. It seems possible that The aim of this study is to gain a better understanding of the
elaborative, metalinguistic feedback may be challenging to effects of feedback in serious literacy games. This will
use for some lower level learners, or that these more involve gathering data to attempt to answer a number of the
explicit forms may be beneficial to older learners. As Li questions posed above, such as:
(2010) notes though, there is surprisingly little research on
1) In the context of serious literacy games, does outcome
metalinguistic complexity of feedback.
plus elaborative feedback lead to better in-game results than
Individual differences and feedback outcome feedback alone?
Ellis also suggests individual differences as an area to study
2) Are different types of elaborative feedback taken up
for CF. While there are many possible individual
differently by learners playing serious literacy games?
differences that could be explored, including age, language
aptitude, gender, prior knowledge, spatial ability, and 3) Do individual differences in working memory affect
motivation, this research will target working memory. uptake of elaborative feedback by learners playing serious
Working memory is of particular relevance due to the literacy games?
multimodal setting, which increases processing demands, in
addition to the audio only feedback delivered (mostly) by a 4) How do 10- and 11-year-old users of serious literacy
synthetic voice. Audio feedback is considered preferable to games engage with the feedback provided to them on a
reduce processing demands (Johnson et. al. 2017) but cognitive, affective, and behavioural level?
The primary data collection for the iRead program will for additional study, including the use of think-aloud
consist of one academic year’s worth of use of the system, protocols, or using the application under feedback/no
for at least one hour a week, by several classes of 10- and feedback conditions for a short period of time.
11-year-old EFL learners in Spain.
Additionally, one or two classes of students at the same
grade level but who will not be using the system (but rather
taking part in their regular reading development activities)
will also undergo the pre- and post-tests in order to serve as
a control group.
Figure 2. Study design
Figure 1. iRead literacy games with both outcome and
elaborative feedback Outcome feedback in the game is presented in numerous
ways, including visual indicators (e.g. a bridge breaking)
Several variables will be involved in the research. Type of
and as oral speech (e.g. “Great job!”). Elaborative feedback
feedback will be an independent variable. Working memory
is all presented orally to the learner when they make an
and engagement will be moderating variables.
error. By undertaking (primarily) classroom research with
The dependent variables will fall into two categories: data a commercial quality application, the study will have strong
internal to the system, and data external to the system. ecological validity. In addition, the large numbers of
participants involved and constant digital data collection
Measures
will help to strengthen any findings.
Internal measures will include data on which features and
games learners are using, errors and error recovery, and
timing of responses. These will allow us to judge the CONCLUSION
effectiveness of the feedback, and whether the different
types are taken up in different ways. External measures will
This is an ongoing study and as such does not yet have final
be assessed with a pre/post-test design, which will allow us
results. However data collection has already begun and
to measure reading development and motivation. At the
there are some emerging trends from the pilot testing and
start of the academic year, all students taking part in the
early data. In general, the elaborative feedback is not
project will be given pre-tests, including tests of:
proving to be more effective than simple outcome feedback.
1) Reading skills, including measures of word and non- The 10- and 11-year-old learners in the study are largely
word recognition, listening comprehension, and oral seen to ignore longer feedback explanations and make
reading fluency in L1 and L2. second attempts before the elaborative feedback has
finished playing. In early think-aloud sessions, participants
2) Vocabulary size, as measured by a children’s productive
regularly refer to the outcome feedback (e.g. tiles turning
vocabulary test developed by Anthony and Nation (2017).
red or green) as helping them understand their progression
3) Working memory, as measured by a backwards digit through the topic, but rarely mention using the longer audio
span. hints. This finding is in contrast to those of Moreno (2004)
and Mayer & Johnson (2010). A possible explanation for
4) A questionnaire gauging individual motivation for this is to do with the age of the participants – our learners
learning English, reading, and using learning games. are primary school children rather than university students.
Twice during the year, learners will complete interim Another preliminary finding relates to the primacy of
engagement questionnaires with additional questions mechanics over language. Many participants struggle
specifically relating to the application they have been using. initially with some of the game mechanics and make
At the end of the academic year, students will undergo post- numerous non-language errors. On the other hand, by their
tests covering all the topics mentioned above in order to third game trial, learners seem to take actions by finding
assess development over time. mechanical similarities with previous trials, rather than due
To investigate cognitive and affective engagement, smaller to understanding and utilizing the target language points.
samples of students will be taken out from their classrooms
Further data and analysis will be available for discussion at 12. Laura Benton, Asmina Vasalou, Kay Berkling,
Gamilearn’19. Wolmet Barendregt, and Manolis Mavrikis. 2018,
April. A Critical Examination of Feedback in Early
ACKNOWLEDGMENTS
Reading Games. In Proceedings of the 2018 CHI
This project has received funding from the European Conference on Human Factors in Computing Systems
Union’s Horizon 2020 research and innovation programme (p. 373). ACM.
under grant agreement No 731724. 13. Roxana Moreno. 2004. Decreasing cognitive load for
novice students: Effects of explanatory versus
corrective feedback in discovery-based multimedia.
REFERENCES
1. Michael Long. 1996. The role of the linguistic Instructional science 32, 1-2: 99-113.
environment in second language acquisition. In 14. Richard E. Mayer and Cheryl I. Johnson. 2010. Adding
Handbook of second language acquisition. Wiley. instructional features that promote learning in a game-
like environment. Journal of Educational Computing
2. Shaofeng Li. 2010. The effectiveness of corrective
feedback in SLA: A meta‐analysis. Language Learning Research 42, 3: 241-265.
60, 2: 309-365. 15. Valerie J. Shute. 2008. Focus on formative feedback.
3. Roy Lyster and Kazuya Saito. 2010. Oral feedback in Review of educational research 78, 1: 153-189.
classroom SLA: A meta-analysis. Studies in second 16. Anne M. Sinatra, Valerie K. Sims, Shannon K.T.
language acquisition 32, 2: 265-302. Bailey, and Maxine B. Najle. 2013, September.
Differences in the performance of older and younger
4. Eunyoung Kang and Zhaohong Han. 2015. The
adults in a natural vs. synthetic speech dichotic
efficacy of written corrective feedback in improving L2
listening task. Proceedings of the Human Factors and
written accuracy: A meta‐analysis. The Modern
Ergonomics Society Annual Meeting 57, 1: 1565-
Language Journal 99, 1: 1-18.
1569).
5. Rabindra Ratan and Ute Ritterfeld. 2009. Classifying
17. Geoffrey Hookham, Bridgettte Bewick, Frances Kay-
serious games. In Serious games: Mechanics and
Lambkin, and Keith Nesbitt. 2016, September. A
effects. Eds. Ute Ritterfeld, Michael Cody, Peter
concurrent think aloud study of engagement and
Vorderer. Routledge. 32-46.
usability in a serious game. In Joint International
6. Ute Ritterfeld, Michael Cody, and Peter Vorderer. Conference on Serious Games. Eds. Tim Marsh,
2009. Serious games: Explication of an oxymoron- Minhua Ma, Manuel Fradinho Oliveira, Jannicke
introduction. In Serious games: Mechanics and effects. Baalsrud Hauge, and Stefan Göbel. Springer. 241-219.
Eds. Ute Ritterfeld, Michael Cody, Peter Vorderer.
18. Shamus P. Smith, Daniel Hickmott, Erica Southgate,
Routledge. 3-9.
Ross Bille, and Liz Stephens. 2016, September.
7. Richard N. Landers. 2014. Developing a theory of Exploring playlearners’ analytics in a serious game for
gamified learning: Linking serious games and literacy improvement. . In Joint International
gamification of learning. In Simulation & gaming 45, Conference on Serious Games. Eds. Tim Marsh,
6: 752-768. Minhua Ma, Manuel Fradinho Oliveira, Jannicke
8. Cheryl I. Johnson, Shannon K.T. Bailey, and Wendi L. Baalsrud Hauge, and Stefan Göbel. Springer. 13-24.
Van Buskirk. 2017. Designing effective feedback 19. Language and Reading Research Consortium. 2015.
messages in serious games and simulations: a research Learning to read: Should we keep things simple?.
review. In Instructional Techniques to Facilitate Reading Research Quarterly 50, 2: 151-169.
Learning and Motivation of Serious Games. Eds. Pieter
20. Laurence Anthony and Paul Nation. 2017. Picture
Wouters and Herre van Oostendorp. Springer. 119-140.
Vocabulary Size Test (Version 1.2.0) [Computer
9. John Hattie and Helen Timperley. 2007. The power of Software]. Tokyo, Japan: Waseda University.
feedback. Review of educational research, 77, 1: 81- Available from
112. http://www.laurenceanthony.net/software/pvst
10. Rod Ellis. 2010. A framework for investigating oral
and written corrective feedback. Studies in Second
Language Acquisition 32, 2: 335-349.
11. Yingli Yang and Roy Lyster. 2010. Effects of form-
focused practice and feedback on Chinese ESL
learners’ acquisition of regular and irregular past tense
forms. Studies in second language acquisition 32, 2:
235-263.