Personalizing VR Educational Tools for English Language Learners Michael J. Lee, Adam Spryszynski, Eric Nersesian New Jersey Institute of Technology Newark, New Jersey, USA {mjlee,as2569,nersesian}@njit.edu ABSTRACT Virtual Reality (VR) provides a unique opportunity for non-native speakers of a language to learn within an immersive platform. This may be particularly useful for English Language Learners (ELLs), who may face many difficulties learning English and acclimating to their new environment and culture. However, many current educa- tional tools use a static, one size-fits-all approach to teach students. We believe that empirical research in VR pedagogy—specifically focused on how to personalize and adapt to, and support second language learners (e.g., ELLs) in these interactive and immersive systems—is an important step in providing educational equity to those that may easily fall behind their peers due to cultural and Figure 1: High school students (English language learners) language barriers. In this paper, we discuss the current state of participating in a chemistry lesson in their class using vir- ELL education, and propose personalized and adaptable VR educa- tual reality headsets and applications. tional tools to help reach a wide range of users with different skills, abilities, and needs. schools have begun to adopt these technologies as ways to engage CCS CONCEPTS their students with course materials [16, 23]. However, although the number of educational VR applications are increasing, they • Human-Centered Computing → Virtual Reality; User Models; typically provide an non-ideal, one-size-fits-all experience for both Interactive Systems and Tools; • Social and Professional Topics content and types of learners. → K-12 Education. One particular group of learner that may particularly benefit from the immersive nature of VR are second language learners (see KEYWORDS Figure 1) [25]. For example, in the USA, K-12 (primary) schools Virtual Reality; Personalization; English Language Learners; Edu- spend an enormous amount of resources (e.g., providing specialized cation classes, customized lessons or educational content, multi-lingual ACM Reference Format: instructors, and translators) in serving English Language Learners Michael J. Lee, Adam Spryszynski, Eric Nersesian. 2019. Personalizing VR (ELLs) [17], helping students learn English and get acclimated to Educational Tools for English Language Learners. In Joint Proceedings of the culture. ELLs are well studied within the education research the ACM IUI 2019 Workshops, Los Angeles, USA, March 20, 2019 , 3 pages. community, but to a lesser extent in educational technology and VR communities. We believe that ELLs are an important user group 1 INTRODUCTION to consider when designing educational VR applications because many will need additional support to succeed academically, educa- Virtual reality (VR) is becoming increasingly accessible to a wider tional policy typically requires fair access to all users, and solutions audience as hardware becomes more affordable and users can utilize created for this group may be applicable and beneficial to a wider their existing devices (e.g., mobile phones) to drive applications [8]. audience of users. Moreover, VR content has significantly improved, showing remark- As these VR resources become more available to the masses, it able promise for collaboration [24], simulation, and particularly has the potential to reach a wide range of users with different skills, in education [2, 18]. VR provides strong content immersion [28], abilities, and needs—especially those in under-served or underrep- allowing learners to interact directly with simulations and focus resented groups—so a static one-size-fits-all approach will not work on the information presented to them, enabling a new educational for everyone. We believe that empirical research in VR pedagogy— medium that can fundamentally change how ideas are shared and specifically focused on how to personalize and adapt to [20], and experienced. With these advancements in content and cost, K-12 support second language learners (such as ELLs) in these inter- active and immersive systems—is an important step in providing IUI Workshops’19, March 20, 2019, Los Angeles, USA educational equity to those that may easily fall behind their peers Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted due to cultural and language barriers. This paper highlights the for private and academic purposes. This volume is published and copyrighted by its editors. importance of thinking about secondary users of a system, and providing interventions and personalization to help them succeed. IUI Workshops’19, March 20, 2019, Los Angeles, USA M.J. Lee et al. 2 RELATED WORK helping students: learn content and language, receive individual in- structional support, and increase engagement [21]. Other research 2.1 School Support for English Language focuses on games and virtual/digital worlds. Chen explored adult Learners ELL’s use of an immersive digital world, "Second Life," for sec- ELLs can face a variety of issues in learning a new language that ond language acquisition and found that the conspicuous features, can benefit from a personalized approach [13, 15]. It is difficult to immersion, and sense of tele- and co-presence within the game enumerate the different types of ELLs as they may be facing is- helped to engaged ELLs with the content [3]. Similarly, Zheng et sues beyond linguistic and cultural integration. For example, some al. investigated the effects of avatar embodiment, collaboration, ELLs may be facing interrupted formal education [4], a lack of lit- and affordances of a virtual world, finding that ELLs with diverse eracy skills in their native language [6], or even active or recent language background preferred virtual environments that used min- refugee status [26]. These types of issues put these ELL students imal text/spoken language [34]. Finally, Freeman explored how a behind their peers in academic readiness and achievement. Educa- digital math application, Help Math, impacted ELL students’ math- tional researchers currently agree that effective teaching for second ematical capabilities by using an interactive visualization to make language acquisition (i.e., English for ELLs) should be based on associations between words and their meanings, concluding that language development instruction combined with opportunities for "digital student directed learning environments, content, and tools second language usage [5, 15]. However, due to ELL students’ di- must be purposefully designed and sensitive to diversity, in or- verse situational needs, the exact balance between direct instruction der to effectively redress academic inequalities and improve ELLs’ and learning through supplemental and complementary sources learning outcomes" [12]. Her study highlights the importance of (such as online tutorials or mobile applications) is unknown [15]. designing educational technologies with an explicit connection Recognizing the differing needs of the diverse population of between the technology (HELP Math), content (math), educational ELLs, there are a variety of programs that K-12 schools in the approach (SI), and context (secondary ELL). These studies demon- United States (US) (and other countries such as the United King- strate the benefits of using virtual spaces and tools for education, dom) use across the country [13]. There is typically an intense and suggest that they might transition well into VR applications introductory program for newcomers, lasting for approximately for ELL education. 3 semesters (1.5 academic years), intended to familiarize students with the cultural and educational routines of the country, region, and local community [13]. Next, schools place these students in a longer-term English Language Development program for the 3 PROPOSAL remainder of their public education. Transitional Bilingual Educa- In this paper, we propose adding personalization in VR educational tion (TBE) programs begin by teaching curriculum in the students’ tools for language learning and cultural acclimation. We believe native language alongside English development, while reducing that this would work best with the Transitional Bilingual Educa- bilingual support as students develop proficiency in English. TBE tion style of programming with Sheltered Instruction. This would is commonly integrated with a Sheltered Instruction (SI) approach provide learners with a fully immersive world about specific school where students learn core curriculum subjects via English instruc- topics, that can be personalized and automatically adapt to their tion that has been adjusted for their language needs [6, 13]. Some changing needs and skill level of their first and second languages. schools also opt to use Developmental Bilingual Education (DBE) Non ELLs can also benefit by learning from a fully immersive and in- programs, which instruct students in both English and their native teractive world, and can conversely explore other languages within language, aiming to integrate students while preserving their cul- the context of the program. ture and language [13]. However, while these method offer some Personalization can occur through information provided by the level of personalization (e.g., there is a teacher or translator helping user that can generate a general user model (e.g., personality traits). students with language instruction), school districts may not be Personality traits have shown to be a suitable general user model able to hire enough instructors to support all the students and/or as it characterizes a person’s thoughts, feelings, social adjustments, the languages they speak. Also, language instructors must have and behaviors, which subsequently influences their expectations, knowledge of the school topics being covered, and/or spend signif- self-perceptions, values, attitudes, and their reactions to others, icant amounts of time with instructors to learn and translate the problems, and stress [19, 32]. Ideally, existing data sources could lessons beforehand. include single-sign on connections to users’ social media accounts. Past work in this area has demonstrated that user-generated content from social networking services (e.g., Facebook [10], Twitter [27], 2.2 Technological Support for English and Instagram [9, 11]) can predict users’ personality and prefer- Language Learners ences. These sites are typically configured in the users’ first or pre- Educational researchers have examined how different technological ferred language, and may include culturally relevant information solutions can help ELLs with their transition into learning English. that can help with the VR educational tool. Posts can also indicate Lopez used interactive white boards for an implementation of a reading/writing level and the users’ command of specific languages. digital learning classroom and raised ELL student achievement to Moreover, Ferwerda et al. have shown that even restricted Facebook performance parity with the rest of the class [22]. Liu et al. found accounts (that severely limit the amount of information provided) that the use of mobile technology by ELLs transitioning from a can be used to relaibly infer personality traits by by examining bilingual to SI approach provided numerous benefits, including whether/which profile sections are disclosed by the user [10]. Personalizing VR Educational Tools for ELLs IUI Workshops’19, March 20, 2019, Los Angeles, USA Personalization can also occur based on users’ performance over and Personality in Personalized Systems 2015. ACM, 7–10. time. For example, spaced repetition is a learning principle that [10] Bruce Ferwerda, Markus Schedl, and Marko Tkalcic. 2016. Personality traits and the relationship with (non-) disclosure behavior on facebook. In Proceedings of depends on variable time intervals between reviews of previously the 25th International Conference Companion on World Wide Web. International learned content for increased learning [31]. This technique is com- World Wide Web Conferences Steering Committee, 565–568. [11] Bruce Ferwerda, Markus Schedl, and Marko Tkalcic. 2016. Using instagram mon in language learning due to being well-suited for accumu- picture features to predict users’ personality. In International Conference on lating large vocabularies but is not limited to it [1]. The Leitner Multimedia Modeling. Springer, 850–861. System [14] and SuperMemo [33] are some common implementa- [12] Barbara Freeman. 2012. Using digital technologies to redress inequities for English language learners in the English speaking mathematics classroom. Computers & tions of spaced repetition. Existing applications use this technique Education 59, 1 (2012), 50–62. to teach Cartesian product and relationships, highlighting the role [13] Fred Genesee. 1999. Program Alternatives for Linguistically Diverse Students. of classification and categorization of content [30]. Additionally, Educational Practice Report 1. (1999). [14] Robert Godwin-Jones. 2010. Emerging technologies from memory palaces to spac- implementation of spaced repetition in games and mobile learn- ing algorithms: approaches to second language vocabulary learning. Language, ing, validate the flexibility that this technique affords in adapting Learning & Technology 14, 2 (2010), 4. [15] Claude Goldenberg. 2008. Teaching English language learners: What the research to different platforms [29]. Modern approaches in this area de- does-and does not-say. (2008). pend on using machine learning and data science for personalized [16] Khe Foon Hew and Wing Sum Cheung. 2010. Use of three-dimensional (3-D) learning [31]. Existing algorithms can be implemented in adaptive immersive virtual worlds in K-12 and higher education settings: A review of the research. British journal of educational technology 41, 1 (2010), 33–55. learning scenarios [7]. The effectiveness of spaced repetition in [17] Sonya Douglass Horsford and Carrie Sampson. 2013. High-ELL-Growth States: language acquisition settings, along with prior research in ELLs’ Expanding Funding Equity and Opportunity for English Language Learners. interactions with immersive systems (e.g., [3, 34]), are good indica- Voices in Urban Education 37 (2013), 47–54. [18] Lasse Jensen and Flemming Konradsen. 2018. A review of the use of virtual reality tors of the potential for using it in personalizable VR applications head-mounted displays in education and training. Education and Information for English language learners. Technologies 23, 4 (2018), 1515–1529. [19] Charles J Krauskopf and David Robertson Saunders. 1994. Personality and ability: As more people turn to new connected and immersive technol- The personality assessment system. Univ Pr of Amer. ogy to learn new skills, there will be an increasing need for systems [20] Michael J Lee and Bruce Ferwerda. 2017. Personalizing online educational tools. to understand and adapt to the needs of their users. We believe In Proceedings of the 2017 ACM Workshop on Theory-Informed User Modeling for Tailoring and Personalizing Interfaces. ACM, 27–30. that personalizing and adapting content in VR instruction can lead [21] Min Liu, Cesar C Navarrete, and Jennifer Wivagg. 2014. Potentials of mobile to extensive benefits for learners. This is especially true for ELLs, technology for K-12 Education: An investigation of iPod touch use for English who might not have complete command of a specific language, language learners in the United States. Journal of Educational Technology & Society 17, 2 (2014). but can use another language and visual/tactile cues within VR to [22] Omar S López. 2010. The digital learning classroom: Improving English lan- transition naturally from their first language to another. VR has guage learners’ academic success in mathematics and reading using interactive whiteboard technology. Computers & Education 54, 4 (2010), 901–915. limitless potential to place learners into an immersive world, and [23] Zahira Merchant, Ernest T Goetz, Lauren Cifuentes, Wendy Keeney-Kennicutt, we hope that further research can help inform the next generation and Trina J Davis. 2014. Effectiveness of virtual reality-based instruction on of engaging and effective VR educational tools. students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education 70 (2014), 29–40. [24] Michael Narayan, Leo Waugh, Xiaoyu Zhang, Pradyut Bafna, and Doug Bow- ACKNOWLEDGMENTS man. 2005. Quantifying the benefits of immersion for collaboration in virtual environments. In Proceedings of the ACM symposium on Virtual reality software This work was supported by Oculus Education. Any opinions, find- and technology. ACM, 78–81. ings, conclusions or recommendations are those of the authors and [25] Eric Nersesian, Adam Spryszynski, Ulysee Thompson, and Michael Lee. 2018. do not necessarily reflect the views of Oculus Education. Encompassing English Language Learners in Virtual Reality. In 2018 IEEE In- ternational Conference on Artificial Intelligence and Virtual Reality (AIVR). IEEE, 200–203. REFERENCES [26] Barbara Nykiel-Herbert. 2010. Iraqi Refugee Students: From a Collection of [1] Meltem Baturay, Soner Yıldırım, and Aysegül Daloğlu. 2009. Effects of Web- Aliens to a Community of Learners–The Role of Cultural Factors in the Acquisi- Based Spaced Repetition on Vocabulary Retention of Foreign Language Learners. tion of Literacy by Iraqi Refugee Students with Interrupted Formal Education. Eurasian Journal of Educational Research (EJER) 34 (2009). Multicultural Education 17, 3 (2010), 2–14. [2] Samantha Adams Becker, Malcolm Brown, Eden Dahlstrom, Annie Davis, Kristi [27] Daniele Quercia, Michal Kosinski, David Stillwell, and Jon Crowcroft. 2011. Our DePaul, Veronica Diaz, and Jeffrey Pomerantz. 2018. NMC Horizon Report: 2018 twitter profiles, our selves: Predicting personality with twitter. In Privacy, Security, Higher Education Edition. EDUCASE. Risk and Trust (PASSAT) and 2011 IEEE Third Inernational Conference on Social [3] Julian Cheng Chiang Chen. 2016. The crossroads of English language learn- Computing (SocialCom), 2011 IEEE Third International Conference on. IEEE, 180– ers, task-based instruction, and 3D multi-user virtual learning in Second Life. 185. Computers & Education 102 (2016), 152–171. [28] Eric D Ragan, Doug A Bowman, and Karl J Huber. 2012. Supporting cognitive pro- [4] Andrea DeCapua and Helaine W Marshall. 2010. Students with limited or in- cessing with spatial information presentations in virtual environments. Virtual terrupted formal education in US classrooms. The Urban Review 42, 2 (2010), Reality 16, 4 (2012), 301–314. 159–173. [29] Florian Schimanke, Robert Mertens, and Oliver Vornberger. 2014. Spaced repeti- [5] Susana Dutro and Carrol Moran. 2003. Rethinking English language instruction: tion learning games on mobile devices: foundations and perspectives. Interactive An architectural approach. English learners: Reaching the highest level of English Technology and Smart Education 11, 3 (2014), 201–222. literacy 227 (2003), 258. [30] Florian Schimanke, Robert Mertens, Oliver Vornberger, and Stephanie Vollmer. [6] Jana Echevarria and Deborah Short. 2010. Programs and practices for effective 2013. Multi category content selection in spaced repetition based mobile learning sheltered content instruction. Improving education for English learners: Research- games. In 2013 IEEE International Symposium on Multimedia. IEEE, 468–473. based approaches (2010), 250–321. [31] Burr Settles and Brendan Meeder. 2016. A trainable spaced repetition model for [7] Darren Edge, Stephen Fitchett, Michael Whitney, and James Landay. 2012. Mem- language learning. In Proceedings of the 54th Annual Meeting of the Association Reflex: adaptive flashcards for mobile microlearning. In Proceedings of the 14th for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1848–1858. international conference on Human-computer interaction with mobile devices and [32] John F Winne and John W Gittinger. 1973. An introduction to the personality services. ACM, 431–440. assessment system. Journal of Community Psychology 1, 2 (1973), 99–163. [8] Adeola Fabola, Alan Miller, and Richard Fawcett. 2015. Exploring the past with [33] Piotr Wozniak. 2007. Supermemo 2004. TESL EJ 10, 4 (2007), 1–12. Google Cardboard. In Digital Heritage, 2015, Vol. 1. IEEE, 277–284. [34] Dongping Zheng, Michael F Young, Manuela Maria Wagner, and Robert A Brewer. [9] Bruce Ferwerda, Markus Schedl, and Marko Tkalcic. 2015. Predicting personality 2009. Negotiation for action: English language learning in game-based virtual traits with instagram pictures. In Proceedings of the 3rd Workshop on Emotions worlds. The Modern Language Journal 93, 4 (2009), 489–511.