Two New Mobile Touchscreen Text Entry Techniques Ahmed Sabbir Arif* Mauricio H. Lopez† Wolfgang Stuerzlinger‡ Department of Computer Science & Engineering, York University, Toronto, Canada identify those letters including space characters that have less than ABSTRACT .01% probability of appearing after the preceding input. Such This article introduces two new mobile touchscreen text entry unlikely characters are then made to be more difficult to enter. In techniques. One is timeout-based and the other is pressure-based. practice, we use a digram frequency table [5] for letter-pairs of the Also, this work examines the effects of tactile feedback on text English language to calculate the probability ρ of a character Cn’s entry techniques. Empirical comparisons between conventional appearance given the preceding character Cn-1, using Equation (1): and proposed techniques show that the new techniques, as well as Total (Cn −1 , Cn ) tactile feedback, enhance overall text entry performance. ρ (Cn | Cn −1 ) = (1) Total (C ) KEYWORDS: Text entry, error prevention, touchscreens, virtual There, Total (C) is the total number of characters including keyboard, delay, timeout, pressure. space and Total (Cn-1, Cn) is the total number of a specific digram (Cn | Cn-1) in the table. We use digrams mainly for simplicity here. INDEX TERMS: H.5.2 [User Interfaces]: Haptic I/O However, n-grams, a dictionary, or grammar rules could also be used to identify less probable characters more accurately. 1 INTRODUCTION In the timeout-based technique, we force users to press unlikely Recently, touchscreens have become one of the dominant keys longer than 0.5 seconds, to make them harder to input. In interaction modality for handheld devices. Many of these devices other words, users will have to press-hold those keys for longer replace physical keyboards with virtual ones, which permit larger than usual. In the pressure-based technique we use pressure and displays, less weight or size. It also enables adaptation to different users will have to apply more force on keys that are unlikely. layouts and orientations. However, virtual keyboards are more 4 MEASURING PRESSURE WORKAROUND error prone [3], mainly due to smaller key sizes [8] and the absence of tactile feedback [8], . To counteract these issues, we All present handheld touchscreen devices do not provide hardware present two new techniques that are timeout-, respectively, support for measuring pressure. Therefore, we detect pressure by pressure-based. We also examine if providing synthetic tactile measuring the movements of the touch centre over time, which information can improve overall text entry performance. identifies different levels of contact force [7], . 2 RELATED WORK 4.1 Pilot Study MultiTap is the dominant technique for standard 12-key keypads We created an application with the iPhone SDK on an Apple on mobile devices. In MultiTap, keys are pressed repeatedly until iPhone 3G at 320×480 pixel resolution for our pilot study. The users get the intended character. Then, one can proceed to the next application’s virtual Qwerty keyboard was practically identical to character, assuming that it is on a different key. If not, users have the default one, see Figure 1, and provided users with auditory to either wait for a timeout for the system to accept a character on and visual feedback via clicks and highlighting during a press. the same key, or have to press a predetermined kill button. Three participants aged from 22 to 24 participated in the pilot McCallum et al. [6] introduced a pressure-based technique for study. One of them was female, two of them had prior experience 12-key mobile keypad with three pressure levels. Their technique with touchscreens, and all of them were right-hand mouse users. was shown to have higher expert entry speed compared to MultiTap, but at the expense of higher error rates. Likewise, Tang et al. [10] developed a 3-key chorded keyboard with three pressure levels, which again yielded higher error rates. Hoffmann et al. [2] designed a physical keyboard that used pressure to prevent errors. This reduced mistyped characters by 87% and correction attempts by 46%. Brewster and Hughes [1], used pressure-based techniques to switch between upper and lower case. This technique was faster and more accurate than standard Figure 1. (a) Illustration of movement with contact force. (b) Touch touchscreen techniques. centre movement during medium and hard presses. 3 NEW TECHNIQUES During the pilot, participants entered all the characters on the The main idea of this work is to generate a list of potential next keyboard holding the device in the portrait position. Two pressure characters based on the preceding input in real-time. Then we levels, medium and hard, were tested. During the medium condition participants entered all characters using regular force from the top-left to the bottom-right, and then from the top-right * e-mail: asarif@cse.yorku.ca to the bottom-left; using at first their left and then the right thumb. † e-mail: cs241053@cse.yorku.ca During the hard condition, participants repeated the same tasks, ‡ e-mail: wolfgang@cse.yorku.ca but applied more force than usual. We recorded the distances between the initial and the release touch centres. In total we 22 recorded 3 participants × 2 sessions (pressure levels) × 2 blocks From the results it is clear that tactile feedback reduces errors (thumbs) × 27 keys (including space) = 324 presses. for all techniques without reducing the speed in a significant An ANOVA showed that there was significant effect of manner. The results also confirmed that pressure-based techniques different pressure levels on touch centre movements (F1,2 = 21.19, have the potential to offer higher performance. We believe that p < .0001). However, there was no significant effect of different with proper training the advantages will increase even more, as thumbs (F1,2 = 0.36, ns). On average left and right touch centres previous studies [6], , showed that response time increases with moved 3.16 pixels (SE = 0.19) during the medium and 4.39 pixels practice for different pressure levels. (SE = 0.19) during the hard presses. 5 AN EXPERIMENT For our experiment, we used the same apparatus and software as for the pilot study. Based on the pilot results, we used a threshold of 4.4 pixels on touch centre movements to identify hard presses. Twelve participants aged from 19 to 34, average 26 years, took part in the experiment. Five of them were female, four of them were touch-typists, and all of them were right-hand mouse users. Figure 2. Screenshot of the application used during the user study. Figure 3. Average WPM and Total ER for different techniques. 5.1 Procedure and Design We tested 3 conditions, namely the regular, timeout-based, and 6 CONCLUSION pressure-based techniques. Each condition was tested with and Here, we presented and evaluated two new mobile touchscreen without synthetic tactile feedback. For the synthetic tactile text entry techniques: one timeout-based and one pressure-based. feedback we activated the iPhone’s vibration motor for 500 ms. The pressure-based techniques had better overall performance Participants were asked to enter a set of short English phrases compared to the conventional one. Our results also showed that [4], all in lowercase, as shown on the display. They held the synthetic tactile feedback significantly reduces errors. device in a portrait position and were asked to take the time to read and understand the phrases, to enter them as fast and accurate REFERENCES as possible, and to press the Return key after completion of a [1] S. A. Brewster and M. Hughes. Pressure-based text entry for mobile phrase to see the next. Timing started from the entry of the first devices. MobileHCI 2009, ACM (2009), 1-4. character and ended with the last. Participants were informed that [2] A. Hoffmann, D. Spelmezan, and J. Borchers. TypeRight: a they could rest between sessions, or before typing a phrase. They keyboard with tactile error prevention. CHI 2009, ACM (2009), were asked to work normally, that is, to correct their errors as they 2265-2268. noticed them. However, they had to use the Backspace button, [3] J. R. Lewis, K. M. Potosnak, and R. L. Magyar. Keys and keyboards. exclusively, for editing. Based on the 3 × 2 = 6 techniques, we M. Helander, T. K. Landauer, and P. V. Prabhu, eds. Handbook of used a within-subjects, 6 × 6 balanced Latin Square design for our Human-Computer Interaction. Elsevier, 1997, 1285-1311. experiment. In summary the design was: 12 participants × 6 [4] I. S. MacKenzie and R. W. Soukoreff. Phrase sets for evaluating text sessions (techniques) × 20 phrases =1440 phrases. entry techniques. Ext. Abstracts CHI 2003, ACM (2003), 754-755. [5] M. S. Mayzner and M. E. Tresselt. Tables of single-letter and digram 5.2 Results and Discussion frequency counts for various word-length and letter-position combinations. Psychonomic Monograph Sup. 1, 2 (1965), 13-32. An ANOVA for the techniques showed that there was significant [6] D. C. McCallum, E. Mak, P. Irani, and S. Subramanian. effect of entry techniques on WPM (F5,11 = 3.21, p < .01). There PressureText: pressure input for mobile phone text entry. Ext. was, however, no significant effect of tactile feedback Abstracts CHI 2009, ACM (2009), 4519-4524. (F1,11 = 0.12, ns). The ANOVA on the Total ER also showed that [7] G. Ramos, M. Boulos, and R. Balakrishnan. Pressure widgets. CHI there was a significant effect of entry techniques (F5,11 = 2.38, 2004, ACM (2004), 487-494. p < .05) and tactile feedback (F1,11 = 7.57, p < .01). [8] A. Sears, D. Revis, J. Swatski, R. Crittenden, and B. Shneiderman. Deeper analysis showed that the regular with tactile Investigating touchscreen typing: the effect of keyboard size on (16.27 WPM, 9.46 Total ER) and pressure with tactile typing speed. BIT 12, 1 (1993), 17-22. (16.08 WPM, 9.24 Total ER) conditions had better overall [9] A. Sears. Improving touchscreen keyboards: design issues and a performance with higher text entry and lower error rates. Timeout comparison with other devices. IWC 3, 3 (1991), 253-269. with tactile was the slowest of all (14.91 WPM, SE = 0.28). A [10] H. Tang, D. J. Beebe, and A. F. Kramer. A multilevel input system Tukey-Kramer test revealed that it was significantly slower than with force-sensitive elements. IJHCS 54, 4 (2001), 495-507. regular with tactile and pressure with tactile. However, it was, at [11] F. Wang, X. Cao, X. Ren, and P. Irani. Detecting and leveraging the same time, the most accurate (8.0% Total ER, SE = 0.70). finger orientation for interaction with direct- touch surfaces. UIST 2009. ACM (2009), 23-32. 23