Using a Design Science Research Methodology Process to Design a Text Entry Technique for Mobile VR Eduardo Gabriel Queiroz Palmeira1, Regis Kopper2, Edgard Afonso Lamounier Júnior1, Alexandre Cardoso1 1 Faculty of Electrical Engineering, Federal University of Uberlândia (UFU), 38400-902 Uberlândia, MG, Brazil egqpalmeira@gmail.com, {lamounier,alexandre}@ufu.br 2 Department of Computer Science, University of North Carolina at Greensboro, 27402-6170 Greensboro, NC, USA kopper@uncg.edu Abstract. Mobile-based virtual reality (VR) is the more affordable way for the general public to experience immersive VR, and text input is a frequent process in mobile VR. Usually, the raycasting selection technique is used to perform this task. However, using this technique may present some limitations. Hence, this short paper aims to briefly present the work in progress of designing and developing an alternative text entry technique for mobile-based VR using a design science research (DSR) methodology process. Our technique uses a one- handed ambiguous keyboard and focuses on improving user performance and typing experience for short-term text input. Further improvement and the evaluation of the technique will compose the next steps of this research. Keywords: Mobile VR, Text Input, Design Science Research, Human- Computer Interaction 1 Introduction In recent years, besides hardware’s cost reduction for the general public to experience immersive virtual reality (VR) systems based on personal computers (PCs), this simulated experience approachability was promoted further by the possibility of being experienced through smartphones [1-3]. The main advantage of mobile-based VR is its portability—autonomy and wireless setting; furthermore, it is more affordable than the equipment required for an entry-level PC-based VR experience [1]. Nevertheless, its performance and graphical capability are limited, as well as its positioning tracking [3,4]. For example, head and hand tracking in mobile VR—through the headset and VR controller, respectively—is performed only in 3 degrees of freedom (DoF). In other words, for both visualization and control, only rotation movements (directions) are tracked instead of rotation and translation movements (directions and Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). tridimensional positions)—as occur in PC-based VR with the use of head-mounted displays (HMDs) and handheld controllers with 6 DoF. Text input is a frequent process in mobile VR. Usually, the raycasting selection technique is used to perform this task. It works as follows: users must keep the virtual pointing steady, intersecting the desired virtual key (1) until they confirm the selection, for instance, by pressing a button or (2) until the system’s automatic selection is confirmed after a predetermined time (dwell time). In mobile VR, the raycasting selection can be performed using the VR controller or the touchpad located on the side of the active headset 1; for the latter, the direction in which the user looks is considered the virtual pointing and a dot in the middle of the screen represents it. Whereas previous work had focused on presenting in more detail the work in progress of the design and development of our alternative text entry technique for mobile-based VR [5], this short paper focuses on its design science research (DSR) methodology process. Additional information regarding the problem identification and the motivation behind this research will be more explored in the next section. 2 The Artifact Development Process In contexts where it may be useful to design a novel artifact to solve a specific problem or improve existing situations, the design science fits better than traditional science [6-8]. Due to the real-world complexity, the DSR does not seek an optimal solution but a satisfactory one capable of positively impacting people’s life [7,9]. Accordingly, the development process of our artifact was performed using as reference the DSR methodology described by Peffers et al. [10]. These authors built this methodology based on a consensus approach of well-accepted common elements within DSR literature. Their DSR methodology process is divided into six iterative activities in a nominal sequence: problem identification and motivation, objective of the solution, design and development, demonstration, evaluation, and communication—represented by previous work, this short paper itself, and further publications. 2.1 Problem Identification and Motivation In this activity, researchers should define a specific problem and justify the value of a solution for the context. In our case, as our approach is problem-centered, some factors must be made evident. Designing efficient and user-centered solutions for text input in VR is a challenge; hence, many researchers have been developing different devices and techniques for text entry in PC-based VR [11]. However, the literature indicates a lack of alternative techniques for mobile-based VR. Using a narrow search, we found only a couple of studies [12,13] presenting alternative techniques for 1 As this work is related to text entry techniques (an interactive process), we are considering mobile VR as those with the use of active headsets (e.g., Samsung Gear VR). That is because passive headsets, such as Google Cardboard, have some limitations [2], enabling only passive visualization of 360° multimedia (non-interactive). mobile VR. It is worth noting that speech-based text entry techniques have limitations regarding text editing [14] and their use in public places: these techniques can cause privacy issues to users, and noisy environments can impair speech recognition [12- 14]. It is also useful to mention that speech-based techniques may not accurately recognize acronyms, new words, or words in other languages. As mentioned before, the standard selection technique for VR is raycasting. However, using this technique may present issues concerning the difficulty to aim at the desired virtual key and the fatigue generated when using both hands for “shooting” fast [15]; this first issue may be caused by the users’ hand tremors [16]. Due to the lack of physical support for their hands, it is difficult for users to aim and keep the virtual pointing steady, intersecting the desired (small) virtual key until the selection is confirmed [17]. Furthermore, unlike PC-based VR, which uses two controllers, mobile VR uses only one controller or the active headset itself for raycasting selection. This condition may cause more tiredness in the long term, as its text input speed is even slower. Moreover, during the immersive experience using mobile VR, users may receive and send messages through the system’s interface itself. Besides, one hand always remains unused in settings in which the VR hand controller is used. All things considered, these circumstances emerge as opportunities for researchers to design novel text input techniques controlled by the available hand, focused on short-term text entry. In this way, users would not need to switch devices while immersed in the virtual world and “blinded” to the real one. 2.2 Objective of the Solution In this activity, researchers should infer the solution objectives based on the analysis of the problem definition. Depending on the research, such objectives can be: describe how it is expected that the novel artifact can assist problem solutions not addressed so far or predict where the desired solution would be better than existing solutions. In this way, our objective is to develop an alternative text entry technique for mobile VR by using a one-handed ambiguous keyboard. The novel technique is focused on improving user performance and typing experience for short-term text input. The identified related works presenting an alternative text entry technique for mobile VR are hands-free [12,13]. Alternately, our solution objective in terms of interaction setup is that users will use the headset with the smartphone inside as the visualization device, the VR controller as navigation and main interaction device, and the ambiguous keyboard as the text input device. Therefore, both hands will be used during the process, making it faster and efficient: the hand holding the VR controller will be used for navigation within the text for copy editing tasks, meanwhile, the hand using the ambiguous keyboard will be used for character input. 2.3 Design and Development In this activity, researchers should determine the artifact’s functionality first and then build it. The research contribution may be incorporated into the artifact design itself. Our ambiguous keyboard prototype comprises a bent case and a Printed Circuit Board (PCB) for 16 keys (4x4) custom mechanical keyboard, Cherry MX Greens tactile switches, 3D printed round-shaped white keycaps, and a Raspberry Pi 3 with Bluetooth and camera modules. Small blue square-shaped pieces of electrical tape were stuck over the keycaps to serve as markers for the computer vision tracking system (see Fig. 1). Fig. 1. Computer vision for visual feedback. This tracking system was designed to provide visual feedback, detecting which keys are being occluded by the user’s fingers in real-time. For this, we used computer vision through the camera module associated with the Python programming language and the OpenCV library. The strong tactile bump switches themselves were used to provide tactile feedback from the user’s fingers hitting keys. Our 16 keys ambiguous keyboard is appropriate to be used with one hand, and it has sub-layouts that support not only lowercase letters but also uppercase letters, numbers, and symbols (Fig. 2). Users can switch between layouts by pressing the toggle keys. In this direction, besides being efficient, our layout is a familiar layout, similar to old phones’ ambiguous keypad. So, for instance, if users want to input the ‘b’ letter, they can press the key corresponding to the number two twice with all the toggle keys turned off. 2.4 Demonstration In this activity, researchers should demonstrate how the artifact use can solve the identified problem. That can be accomplished through experimentation or simulation, for instance. To achieve this, we developed a mobile VR application using Unity 3D. Its user interface comprises the ambiguous keyboard virtual representation with real- time visual feedback, text phrases to be copied by the user, and an empty text entry field (Fig. 3). Fig. 2. Sub-layouts of the text input technique. Fig. 3. A screenshot of the application. When building an artifact, avoiding unwanted side effects is a challenge [6], but it is essential. The virtual representation of the user’s hands is, in fact, important for a positive user experience. Nevertheless, the uncanny valley can occur if the hands’ tracking is inaccurate [18,19]; this phenomenon can impair user experience and sense of presence. Thus, considering that mobile VR is stationary (3 DoF), user virtual hands’ representation was not included. However, despite users not seeing the representation of their virtual hands, they can see their typing results in real-time to preserve a positive user experience. Lastly, we used a Samsung Galaxy S8 smartphone, a Samsung Gear VR active headset, and a Samsung VR hand controller (3 DoF) to run the application. 2.5 Evaluation In this activity, researchers should observe and measure how well the artifact can solve the problem. Relevant metrics and analysis techniques related to the researched context should be used to evaluate the artifact, and the evaluation should result in adequate empirical evidence. Before starting this activity, we decided to iterate back to activity 3 (design and development) to improve the tracking system. As a limitation, the computer vision tracking system only tracks one finger per keyboard’s column; it also provides imprecise feedback depending on the environment’s lighting and the module camera position in relation to the ambiguous keyboard. Hence, before the evaluation through empirical comparison, we decided to develop an accurate touch-based tracking system (Fig. 4). Instead of using blue square-shaped markers associated with computer vision, we placed metallic square-shaped high impedance sensors over the keycaps to detect the user’s fingers’ touch. This new prototype is under development and will be presented in further detail in future publications. Fig. 4. The under-development touch-based tracking system. Usually, the evaluation of DSR’s artifacts is based on methodologies available in the knowledge base [8]. In this way, after improving our prototype, also in future works, we will evaluate our technique using methodological aspects from the study by Boletsis and Kongsvik [15]. Participants’ task will be to copy pre-defined phrases from MacKenzie and Soukoreff phrase sets created for evaluating text entry techniques [20]. In this way, the empirical experiment will collect data regarding performance—typing speed [21,22] and error rate [22,23]—and user preferences— usability [24] and user experience [25]. In addition, it will compare the following techniques: (1) raycasting head-directed selection technique using the headset; (2) raycasting selection technique using the VR hand controller (3 Dof); (3) the technique presented in this paper (multi-tap approach); and (4) the technique presented in this paper but with a single-tap approach associated with machine learning to predict possible corresponding words. 3 Conclusion This short paper presented the work in progress of the design and development process of a novel text entry technique for mobile VR, focusing on its DSR methodology. Reliance on creativity and trial-and-error are characteristic of this type of research focused on artifact development [8]. Thus, an iteration back to activity 3 (design and development)—seeking the improvement of the artifact effectiveness— and activity 5 (evaluation) will compose the next steps of this work. We will also conduct a broad search through a systematic literature review to identify more related works. Finally, after further improvement and evaluation, we will continue to communicate our research resulting knowledge through academic publications. Acknowledgments. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001. References 1. Tori, R., Hounsell, M. (eds.).: Introdução a Realidade Virtual e Aumentada. 3rd edn. Editora SBC, Porto Alegre (2020) https://doi.org/10.5753/sbc.6654.2 2. Powell, W., Powell, V., Brown, P., Cook, M., Uddin, J.: Getting Around in Google Cardboard – Exploring Navigation Preferences with Low-Cost Mobile VR. In: 2016 IEEE 2nd Workshop on Everyday Virtual Reality (WEVR). IEEE (2016) 5–8 https://doi.org/10.1109/WEVR.2016.7859536 3. Steed, A., Julier, S.: Design and Implementation of an Immersive Virtual Reality System Based on a Smartphone Platform. In: 2013 IEEE Symposium on 3D User Interfaces (3DUI). IEEE (2013) 43–46 https://doi.org/10.1109/3DUI.2013.6550195 4. Sharma, P.: Challenges with Virtual Reality on Mobile Devices. In: ACM SIGGRAPH 2015 Talks (SIGGRAPH ’15). ACM, New York (2015) https://doi.org/10.1145/2775280.2792597 5. Palmeira, E., Moraes, I., Telles, E., Martin, V., Gonçalves, V., Kopper, R., Lamounier Jr., E., Cardoso, A.: One-Handed Text Entry in Mobile-Based Virtual Reality: An Ambiguous Keyboard Technique. In: Ahram, T., Falcão, C. (eds.): Advances in Usability, User Experience, Wearable and Assistive Technology. Lecture Notes in Networks and Systems, Vol. 275. Springer International Publishing, Cham (2021) 310–318 https://doi.org/10.1007/978-3-030-80091-8_36 6. March, S., Smith, G.: Design and Natural Science Research on Information Technology. Decision Support Systems, 15(4) (1995) 251–266 https://doi.org/10.1016/0167- 9236(94)00041-2 7. Dresch, A., Lacerda, D., Antunes Jr., J.: Design Science Research: Método de Pesquisa para Avanço da Ciência e Tecnologia. 1st edn. Bookman, Porto Alegre (2015) 8. Hevner, A., March, S., Park, J., Ram, S.: Design Science in Information Systems Research. Management Information Systems Quarterly, 28(1) (2004) 75–105 https://doi.org/10.2307/25148625 9. Simon, H.: The Sciences of the Artificial. 3rd edn. MIT Press, Cambridge (1996) 10. Peffers, K., Tuunanen, T., Rothenberger, M., Chatterjee, S.: A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, 24(3) (2007) 45–77 https://doi.org/10.2753/MIS0742-1222240302 11. Dube, T., Arif, A.: Text Entry in Virtual Reality: A Comprehensive Review of the Literature. In: Kurosu, M. (ed.): Human-Computer Interaction. Recognition and Interaction Technologies. Lecture Notes in Computer Science, Vol. 11567. Springer International Publishing, Cham (2019) 419–437 https://doi.org/10.1007/978-3-030-22643-5_33 12. Xu, W., Liang, H., Zhao, Y., Zhang, T., Yu, D., Monteiro, D.: RingText: Dwell-Free and Hands-Free Text Entry for Mobile Head-Mounted Displays Using Head Motions. IEEE Transactions on Visualization and Computer Graphics, 25(5) (2019) 1991–2001 https://doi.org/10.1109/TVCG.2019.2898736 13. Lu, X., Yu, D., Liang, H., Feng, X., Xu, W.: DepthText: Leveraging Head Movements Towards the Depth Dimension for Hands-free Text Entry in Mobile Virtual Reality Systems. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE (2019) 1060–1061 https://doi.org/10.1109/VR.2019.8797901 14. Grubert, J., Witzani, L., Ofek, E., Pahud, M., Kranz, M., Kristensson, P.: Text Entry in Immersive Head-Mounted Display-Based Virtual Reality Using Standard Keyboards. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE (2018) 159– 166 https://doi.org/10.1109/VR.2018.8446059 15. Boletsis, C., Kongsvik, S.: Controller-Based Text-Input Techniques for Virtual Reality: An Empirical Comparison. International Journal of Virtual Reality, 19(3) (2019) 2–15 https://doi.org/10.20870/IJVR.2019.19.3.2917 16. Kopper, R., Bowman, D., Silva, M., McMahan, R.: A Human Motor Behavior Model for Distal Pointing Tasks. International Journal of Human-Computer Studies, 68(10) (2010) 603–615 https://doi.org/10.1016/j.ijhcs.2010.05.001 17. Tu, H., Huang, S., Yuan, J., Ren, X., Tian, F.: Crossing-Based Selection with Virtual Reality Head-Mounted Displays. In: 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Paper 618. ACM, New York (2019) 1–14 https://doi.org/10.1145/3290605.3300848 18. Palmeira, E., Martin, V., Moraes, I., Kopper, R., Lamounier Jr., E., Cardoso, A.: O Uncanny Valley das Mãos Virtuais em Aplicações de Realidade Virtual Imersiva: uma Revisão Sistemática da Literatura. RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação, 2020(E31) (2020) 497–512 https://www.proquest.com/docview/2496338859 19. Berger, C., Gonzalez-Franco, M., Ofek, E., Hinckley, K.: The Uncanny Valley of Haptics. Science Robotics, 3(17) (2018) 1–2 https://doi.org/10.1126/scirobotics.aar7010 20. MacKenzie, I., Soukoreff, R.: Phrase Sets for Evaluating Text Entry Techniques. In: CHI ’03 Extended Abstracts on Human Factors in Computing Systems. ACM, New York (2003) 754–755 https://doi.org/10.1145/765891.765971 21. Wobbrock, J.: Measures of Text Entry Performance. In: Mackenzie, S., Tanaka-Ishii, K. (eds.): Text Entry Systems: Mobility, Accessibility, Universality. Morgan Kaufmann Publishers, Burlington (2007) 47–74 https://doi.org/10.1016/B978-012373591-1/50003-6 22. Arif, A., Stuerzlinger, W.: Analysis of Text Entry Performance Metrics. In: 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH). IEEE (2009) 100–105 https://doi.org/10.1109/TIC-STH.2009.5444533 23. Soukoreff, R., MacKenzie, I.: Metrics for Text Entry Research: An Evaluation of MSD and KSPC, and a New Unified Error Metric. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘03). ACM, New York (2003) 113–120 https://doi.org/10.1145/642611.642632 24. Brooke, J.: SUS: A Retrospective. Journal of Usability Studies, 8(2) (2013) 29–40 https://dl.acm.org/doi/10.5555/2817912.2817913 25. IJsselsteijn, W., Kort, Y., Poels, K.: The Game Experience Questionnaire. Technische Universiteit Eindhoven, Eindhoven (2013) https://research.tue.nl/en/publications/the-game- experience-questionnaire