Supporting Language Learning Strategies for Erasmus Students with a Mobile Tool Using Tangible User Interfaces and Interactive Storytelling: the EULALIA Approach

Abstract

In the framework of the EULALIA project (Enhancing University Language courses with an App powered by game-based Learning and tangible user Interfaces Digital Creativity Enhanced in Teacher education) the authors develop a methodology for the language teaching/learning for Erasmus students by using scenario based and multisensorial methodologies applying Tangible User Interfaces and Mobile Learning approaches. The paper presents the structure of the application and the scenario based learning (SBL) and digital storytelling approach that supports language learning strategies (LLS) employing mobile learning and tangible user interfaces (TUI) systems. The approach embraces both the virtual and real dimensions of leaning and is characterised by the embodiment of interaction in physical objects. Tangible objects mediate the interaction between the learner and digital interface, innovatively engaging all senses during learning content fruition, and making learning experiential. The paper also presents the app deployment plan in the real education context as a tool to support second language learning strategies (LLS) for Erasmus students.

Keywords

• Tangible User Interfaces
• Scenario Based Learning
• Digital Game Based Learning
• multisensory learning

-

"Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0)."

Introduction

Tangible User Interfaces embodying leaning interaction within scenarios

Language Learning Strategies (LLS) [ 1 ] refer to the behaviours and thoughts that a learner engages in during learning. They also include the processes and actions that are consciously deployed to learn or use a language more effectively [ 2 ]. Language learning strategy is an important contribution to positive achievements in second language learning. Language use is directly connected with social norms and practices, cultural codes and real-life contexts where individuals are expressing themselves and communicating with others. While research shows that knowledge is best acquired and more fully understood when situated within its context [ 3 ], the vast majority of language teaching and training continues to apply traditional fronted methods characterized by repetition and memorization, with a limited focus on promoting reflection and transfer of knowledge.

The internalization of the learning experience, fundamental in the learning process, is successful when concepts and knowledge are anchored to real-life problems and scenarios, as “Knowledge is a storehouse of representations, which can be called upon for use in reasoning and which can be translated into language” [ 4 ].

In language learning, scenarios facilitate language learners to practice in real life simulated situations. Scenario-based learning (SBL) provides meaningful learning experiences by engaging students in authentic environments to support reflective practices and active learning in a real-world problem and in a subsequent solution finding process [ 5 ]. As scenarios are sequences of communicative situations, they offer a means of incorporating Common European Framework of Reference for Languages (CEFR) descriptors into language learning path. A CEFR-based scenario provides a set of realworld variables, including a domain, context, tasks, language activities and, in which “Can-Do” descriptors can be integrated as learning objectives, together with aspects of strategic, pragmatic and linguistic competence as enabling objectives, and quality criteria for evaluation purposes [ 6 ].

In addition, the scenarios adhere to the storytelling approach. The storytelling (and the newborn digital storytelling) [ 7 ] is a well-known framework for enhancing achievement and learning motivation for higher education students [ 8 ] and with good results for second language learning [ 9, 10, 11 ].

Furthermore, knowledge and the world are both construed and interpreted through action and mediated through symbol use [ 12 ]. From this prospective Tangible User Interfaces (TUIs) [ 13 ] represents a system embodying the interaction with physical objects of the scenarios that support knowledge acquisition through experience. From a cognitivist learning perspective, TUIs can support the storing of new knowledge related to the physical world. This is done either through expressive or explorative activity [ 14, 15 ], as the physical interaction which allows the learner to construct knowledge of the world through experiencing it [ 16 ] - or to learn by doing (manipulating things in the physical world) [ 17 ]. In this view, the learner acts and intervenes on the scenario with physical interaction on the system, promoting action in line with the interactive storytelling approach [ 18 ].

Prototype description

The EULALIA App

The EULALIA ((Enhancing University Language courses with an App powered by game-based Learning and tangible user Interfaces Digital Creativity Enhanced in Teacher education)) App has the structure of a serious game, a game which “primary goal is education rather than entertainment" [ 19 ], in particular to enhance the player’s level within CEFR. The application combines utilitarian training aspects with playful means that reinforce engagement. Research has shown that lack of boredom links to learning achievement with a consequent engagement [ 20 ]. Effectiveness of the learning process has been directly correlated with learners’ engagement in learning activities [ 21 ]. Research has shown that engagement increases when activities are tailored to personal needs and emotional states [ 22 ] in line with the Universal Design for Learning framework (UDL) [ 23 ].

Digital Game Based Learning (DGBL) is also considered to address some of the pitfalls faced by educators such as lack of student’s motivation or student’s low level of confidence [ 24 ]. Entertainment is also a reward for achievements in the game and empowers the development of language competences and acquisition of knowledge about the cultural aspect of the hosting city.

The design of the game implemented is oriented by its educational purpose, balancing accuracy and accessibility, graphics and interactivity. The activities are built as tasks of an established narrative. To develop a language learning framework, the scenarios are CEFR-based and consist in a set of real-world variables, including a domain, context and task, that correspond to learning objects. Making choices and solving problems, the player built his knowledge through simulated experience. According to Fröbel’s ideas of DGBL [ 25 ], the educational value of learning results from the contextualization of knowledge in a challenging situation where the players can assess their way of thinking and behaving and develop what is needed to adjust to this situation [ 26 ]. The player deals with multidimensional data through a gamified learning experience in which visual and spatial, auditive and tactile aspects interact through the use of TUI applications. According to Mayer´s model [ 27 ], as summarized by Farías, Obilinovic and Orrego [ 28 ], learning through multimedia potentially leads to deeper learning and understanding than do sessions that are presented solely in one format. Furthermore, sensorimotor experiences are connected with cognitive functions within the language processing and comprehension. In a multisensory learning experience, when the player/learners understand words, the same sensorimotor areas are recruited as for interacting with the physical objects in a tangible environment [ 29 ].

EULALIA proposes a technology enhanced learning tool providing multimodal communication and multisensory applications that transform the approach to language learning, immersing the player/learner in a simulated scenario that situate the knowledge in a context with an interactive storytelling approach. Within EULALIA active learning is also fostered by the possibility to develop scenarios, involving students working their way through a storyline and providing opportunities for self-regulated language learning strategies to promote language learners’ awareness. Recent theoretical discussions on foreign language learning have yielded valuable insights for the field while also reporting various positive impacts on second or foreign language learning [ 30, 31 ].

The software that allows the scenarios development is STELT [ 32 ]. This authoring system is also suitable for users with low programming skills. STELT allows applications using TUIs [ 33 ]. STELT joins: i) communication protocols of the hardware (RFID/NFC readers), ii) logic of the scenarios, iii) learning analytics about the learners, and iv) adaptive tutor’s modules. STELT allows the connection meaning-object, assigned during the tagging task: each object is equipped with a RFID/NFC passive antenna attached to it. Attaching thin RFID/NFC tags inside (or behind) any object, makes it possible to turn it into a Smart Object [ 34 ] and link it to a multisensory learning scenario [ 35 ].

This technological tool in the form of EULALIA APP could be installed on Android smartphones and Windows applications. The users will interact with the digital interfaces, with the scope to perform quests or treasure hunts, to solve quizzes or to interact with stories. This is done through the use of physical and real objects and through employing the senses including smell and taste. In this context, a typical exercise is the quest on a physical map by browsing with the phone equipped with NFC sensor, which represents the sensitive points (well known by the designer of the scenario and programmed in the app using STELT). Each interaction with the map is aimed to improve the foreign language skills of the user, by browsing and searching the next answer on the map.

In this view the students and teachers are able to creatively design an Open Educational Resource (OER), co-creating their Language Learning Strategy (LLS), in the form of a game scenario. The game will be embedded in a mobile APP that embody knowledge acquisition to real life context using tangible object starting from maps, cards, including also smells or tastes. 3

Conclusion and future directions

EULALIA is developed to support the development of second or foreign language learning strategies. The primary target of the EULALIA approach are the Erasmus students. This approach can become an innovative tool for language learning.

This short paper presents the structure of a game application embodying multisensorial learning experiences via TUI application and presents the pedagogical framework underpinning its deployment based on robust literature.

The EULALIA application developed proposes scenarios in three main fields, namely: 1) cultural heritage and traditions, 2) daily life situations, 3) university knowledge under a practical profile. This will be delivered through language domain activities (i.e. Scratch) [ 36, 37 ];and tangible user interfaces applications [ 38, 39, 40, 41 ]; supporting technology enhanced language learning strategies [ 42, 43, 44, 45, 46 ].

The next step brings to the deployment of a piloting phase, during which 1) university teachers and lecturers will be trained on how to use and embed the tool within university language courses; 2) Erasmus students will make use of the tool to develop their language learning strategies, developing their own scenarios, developing their own contents or reusing the courses contents. 4

Acknowledgment

EULALIA (Enhancing University Language courses with an App powered by gamebased Learning and tangible user Interfaces Digital Creativity Enhanced in Teacher education) is co-founded by the Erasmus+ programme of the European Union, in the call Key Activity 2 – Strategic Partnership (Grant Agreement 2019-1-IT02-KA203063228) and runs between October 2019 and September 2021. The European Commission's support for the production of this publication does not constitute an endorsement of the contents, which reflect the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

References

1. Oxford, R.: The role of styles and strategies in second language learning, ERIC Digest, ERIC clearinghouse and linguistic , Washington DC. ( 1989 ).

2. Weinstein , C. , Mayer , R. : The Teaching of Learning Strategies . In: Wittrock, M. , Ed., Handbook of Research on Teaching , pp. 315 - 327 . Macmillan, New York ( 1986 )

3. Kindley , R. W.: Scenario-based e-learning: A step beyond traditional e-learning . ASTD Online Magazine Learning Circuits 3 ( 2002 )

4. Hung , D. : Theories of Learning and Computer-Mediated Instructional Technologies . Educational Media International, 38 ( 4 ), 281 - 287 ( 2001 ).

5. Marocco , D. , Dell'Aquila , E. , Zurlo , M.C. , Vallone , F. , Barajas , M. , Frossard , F. , Di Ferdinando A. , Di Fuccio R. , Lippens M. , Van Praag L. , Protopsaitis A. , Swertz C. , Yüksel Arslan P. , Mazzucato A. ,: Attain Cultural Integration through teachers' COnflict Resolution skills Development: the ACCORD Project . Qwerty - Open and Interdisciplinary Journal of Technology, Culture and Education , 14 ( 2 ) 11 - 30 ( 2019 ).

6. Council of Europe: The Common European Framework of Reference for Languages. Learning, Teaching , Assessment. Cambridge University Press, Cambridge ( 2001 ).

7. Liapis , A. : The Newborn World: Guiding Creativity in a Competitive Storytelling Game . In: 2019 IEEE Conference on Games (CoG) , pp. 1 - 8 . IEEE ( 2019 ).

8. Yang , Y. T. C. , Wu , W. C. I. : Digital storytelling for enhancing student academic achievement, critical thinking, and learning motivation: A year-long experimental study . Computers & education , 59 ( 2 ), 339 - 352 ( 2012 ).

9. Kim , S. : Developing autonomous learning for oral proficiency using digital storytelling . Language Learning & Technology , 18 ( 2 ), 20 - 35 ( 2014 ).

10. Green , L. S. : Language learning through a lens: The case for digital storytelling in the second language classroom . School Libraries Worldwide , 19 ( 2 ), 23 - 36 ( 2013 ).

11. Liu , K. P. , Tai , S. J. D. , & Liu , C. C. : Enhancing language learning through creation: The effect of digital storytelling on student learning motivation and performance in a school English course . Educational Technology Research and Development , 66 ( 4 ), 913 - 935 ( 2018 ).

12. Ackermann , E.: Constructing knowledge and transforming the world . In: Tokoro, M. and Steels , L. ( eds) A Learning Zone of One's Own: Sharing Representations and Flow in Collaborative Learning Environments , pp. 15 - 37 , IOS Press, Washington, DC. ( 2004 ).

13. Ishii , H.: The tangible user interface and its evolution . Communications of the ACM , 51 ( 6 ), 32 - 36 ( 2008 ).

14. Marshall , P. , Price , S. , Rogers , Y. : Conceptualising tangibles to support learning . Proceedings of the Interaction Design and Children (IDC'03) , ACM Press. Preston, UK ( 2003 ).

15. O 'Malley , C. , Stanton , D. : Tangible technologies for collaborative storytelling . In Proceedings of the European Workshop on Mobile and Contextual Learning pp. 3 - 6 . Educational Technology Research Papers, Birmingham ( 2002 ).

16. Le Francois , G. R. : Psychology for teaching . Tenth edition , Wadsworth/Thomson Learning, Belmont, CA ( 2000 ).

17. Markova , M. S. , Wilson, S. , Stumpf , S. : Tangible user interfaces for learning . International Journal of Technology Enhanced Learning , 4 ( 3-4 ), 139 - 155 ( 2012 ).

18. Thue , D. , Bulitko , V. , Spetch , M. , Wasylishen , E.: Interactive Storytelling: A Player Modelling Approach . AIIDE 43- 48 ( 2007 ).

19. Michael , D. , Chen , S. : Serious Games: Games that Educate, Train, and Inform . Thomson Course Technology PTR . Boston, MA ( 2005 ).

20. Standen , P. , Brown , D. J., Taheri , M. , Galvez Trigo , M. J. , Boulton , H. , Burton , et al...: An evaluation of an adaptive learning system based on multimodal affect recognition for learners with intellectual disabilities . British Journal of Educational Technology 0 ( 0 ), 1 - 18 ( 2020 ).

21. Hamari , J. , Sherno , D. J. , Rowe , E. , Coller , B. , Asbell-Clarke , J. , Edwards , T. : Challenging games help students learn: An empirical study on engagement, flow and immersion in gamebased learning . Computers in Human Behavior , 54 , 170 - 179 ( 2016 ).

22. Athanasiadis , C. , Hortal , E. , Koutsoukos , D. , Lens , C. Z. , & Asteriadis , S. ( 2017 ) : Personalized, affect and performance-driven computer-based learning . In proceedings of the 9th International Conference on Computer Supported Education vol. 1 , CSEDU. Porto, Portugal ( 2017 ).

23. Meyer, A., Rose , D. H. , Gordon , D. T.: Universal design for learning: Theory and practice . CAST Professional Publishing , Wakefield, MA ( 2014 ).

24. Egenfeldt-Nielsen , S., Meyer, B. , Holm Sørensen , B. : (eds) Serious games in education: a global perspective . Aarhus University Press, Santa Barbara ( 2011 ).

25. Brougère , G ,: Jeu et éducation . L'Harmattan, Paris ( 2000 ).

26. Sanchez , E. , Monod-Ansaldi , R. Vincent , C. Safadi , S.: A praxeological perspective for the design and implementation of a digital role-play game . Education and Information Technologies 22 ( 6 ) 2805 - 2824 ( 2017 ).

27. Mayer , R.: Multimedia learning . Cambridge University Press, New York ( 2001 ).

28. Farías , M. , Obilinovic , K. , Orrego , R.: Implications of multimodal learning models for foreign language teaching and learning . Colombian Applied Linguistics Journal 19 , 174 - 200 ( 2007 ).

29. Jirak , D. , Menz , M. M. , Buccino , G. , Borghi , A. M. , Binkofski , F. : Grasping language 100 A short story on embodiment , in Consciousness and Cognition 19 ( 3 ), 711 - 720 ( 2010 ).

30. Andrade , M. S. , Bunker , E. L. : A model for self-regulated distance language learning . Distance Education , 30 , 47 - 61 ( 2009 ).

31. Beishuizen , J. , Steffens , K. : A conceptual framework for research on self-regulated learning . In R. Carneiro, P. Lefrere , K. Steffens , & J. Underwood (eds.), Self-regulated learning in technology enhanced learning environments: A European perspective , pp. 3 - 19 . Sense Publishers, Rotterdam, The Netherlands ( 2011 ).

32. Miglino , O. , Di Ferdinando , A. , Schembri , M. , Caretti , M. , Rega , A. , Ricci , C. : STELT (Smart Technologies to Enhance Learning and Teaching): una piattaforma per realizzare ambienti di realtà aumentata per apprendere, insegnare e giocare . Sistemi intelligenti , 25 ( 2 ), 397 - 404 ( 2013 ).

33. Di Fuccio , R. , Mastroberti , S. : Tangible User Interfaces For Multisensory Storytelling At School: A Study Of Acceptability . Qwerty-Open and Interdisciplinary Journal of Technology, Culture and Education , 13 ( 1 ), 62 - 75 ( 2018 ).

34. Kortuem , G. , Kawsar , F. , Sundramoorthy , V. Fitton , D. : Smart objects as building blocks for the Internet of Things . IEEE Internet Computing , 14 ( 1 ), 44 - 51 ( 2010 ).

35. Di Fuccio , R. , Ponticorvo , M. , Ferrara , F. , Miglino , O. : Digital and Multisensory Storytelling: Narration with Smell, Taste and Touch . In: Verbert, K. , Sharples , M. , Klobučar , T. ( eds) Adaptive and Adaptable Learning . EC-TEL 2016. Lecture Notes in Computer Science , vol 9891 . Springer, Cham ( 2016 ).

36. Grizioti , M. , Kynigos , C. : Game modding for computational thinking: an integrated design approach . In Proceedings of the 17th ACM Conference on Interaction Design and Children . ACM. 687 - 692 ( 2018 ).

37. Wilson, A. , Hainey , T. , Connolly , T. M.: Using Scratch with primary school children: an evaluation of games constructed to gauge understanding of programming concepts . International Journal of Game-Based Learning (IJGBL) , 3 ( 1 ), 93 - 109 ( 2013 ).

38. Di Fuccio , R. , Siano , G. , De Marco , A. : The Activity Board 1.0: RFID-NFC WI-FI Multitags Desktop Reader for Education and Rehabilitation Applications . In World Conference on Information Systems and Technologies . Springer, Cham. 677 - 689 ( 2017 ).

39. Lancioni , G. E. , Singh , N. N. , O'Reilly , M. F. , Alberti , G.: Assistive Technology to Support Communication in Individuals with Neurodevelopmental Disorders . Current Developmental Disorders Reports , ( 3 ) 1 - 5 ( 2019 ).

40. Miglino , O. , Di Ferdinando , A. , Di Fuccio , R. , Rega , A. , Ricci , C. : Bridging digital and physical educational games using RFID/NFC technologies . Journal of e-Learning and Knowledge Society , 10 ( 3 ) ( 2014 ).

41. Ponticorvo , M. , Di Fuccio , R. , Ferrara , F. , Rega , A. , & Miglino , O. : Multisensory educational materials: five senses to learn . In International Conference in Methodologies and intelligent Systems for Techhnology Enhanced Learning ( 45 - 52 ). Springer, Cham ( 2018 ).

42. Nomass , B. B. : The impact of using technology in teaching English as a second language . English Language and Literature Studies , 3 ( 1 ), 111 - 116 ( 2013 ).

43. Nomass , B. B. : The impact of using technology in teaching English as a second language . English Language and Literature Studies , 3 ( 1 ), 111 - 116 ( 2013 ).

44. Smith , S. M. , Woody , P. C. : Interactive effect of multimedia instruction and learning styles . Teaching of Psychology , 27 ( 3 ), 220 - 223 ( 2000 ).

45. Salaberry , M. R.: The use of technology for second language learning and teaching: A retrospective . The Modern Language Journal , 85 ( 1 ), 39 - 56 ( 2001 ).

46. Tsou , W. , Wang , W. , Tzeng , Y. : Applying a multimedia storytelling website in foreign language learning . Computers & Education , 47 , 17 - 28 ( 2006 ).