=Paper=
{{Paper
|id=None
|storemode=property
|title=Selecting Gestural User Interaction Patterns for Recommender Applications on Smartphones
|pdfUrl=https://ceur-ws.org/Vol-1050/paper3.pdf
|volume=Vol-1050
|dblpUrl=https://dblp.org/rec/conf/recsys/WorndlWL13
}}
==Selecting Gestural User Interaction Patterns for Recommender Applications on Smartphones==
https://ceur-ws.org/Vol-1050/paper3.pdf
Selecting Gestural User Interaction Patterns for
Recommender Applications on Smartphones
Wolfgang Wörndl Jan Weicker Béatrice Lamche
TU München TU München TU München
Boltzmannstr. 3 Boltzmannstr. 3 Boltzmannstr. 3
85748 Garching 85748 Garching 85748 Garching
Germany Germany Germany
woerndl@in.tum.de weicker@in.tum.de lamche@in.tum.de
ABSTRACT - such as initiating a search for recommended items or rating an
Modern smartphones allow for gestural touchscreen and free-form item - to reasonable gesture and motion interaction patterns. We
user interaction such as swiping across the touchscreen or shaking designed a prototype to allow comparing user interface options
the device. However, user acceptance of motion gestures in and conducted a user study to find out which interaction patterns
recommender systems have not been studied much. In this work, users would select when given a choice.
we investigated the usage of gestural interaction patterns for
mobile recommender systems. We designed a prototype that
2. BACKGROUND
implemented at least two input methods for each available 2.1 Gestural User Interaction Patterns
function: standard on-screen buttons or menu options, and also a Saffer [3] distinguishes between two different forms of gestural
gestural interaction pattern. In a user study, we then compared interaction: touchscreen and free-form. Touchscreen gestures
what input method users would choose for a given function. allow users to tap on the screen, either using on-screen buttons or
Results showed that gesture usage depended on the specific task. other interface elements, e.g. sliders. Free-form gestures do not
In general, users preferred simpler gestures and rarely switched require the user to actively touch the screen but to move the
their input method for a function during the test. devices to initiate functions. Current mobile devices offer several
sensors that enable motion detection such as accelerometers and
Categories and Subject Descriptors gyroscopes. The following touchscreen and free-form gestures are
H.5.2 [Information Interfaces and Presentation]: User commonly used in mobile applications (Fig. 1).
Interfaces – Input devices and strategies, Interaction styles
General Terms
Design, Experimentation, Human Factors.
Keywords
user interfaces, mobile applications, recommender systems, user
study, gestural interaction.
1. INTRODUCTION
Recommender systems recommend movies, restaurants or other
items to an active user based on ratings of items or other
information about users and items. Recently, the focus in
recommender systems research has been changing from
investigating algorithms to studying the user experience [1]. This
is especially true in mobile scenarios, for example on
smartphones. Mobile information access suffers from limited Figure 1. A visualization of how the different gestures are
resources regarding input capabilities, displays and other performed. Circles represent touches by fingers, arrows
restrictions of small mobile devices. Therefore, user interfaces for indicate movement. (1) Spread, (2) Pinch, (3) One-Finger-
mobile recommender systems have to be adapted to the specific Hold Pinch, (4) Fling, (5) Flick/Swipe, (6) Rectangular
properties of mobile devices [2]. Pattern, (7) Shake Device, (8) Tilt Device.
Single Tap is a brief one-finger tap on the screen and used in
The aim of this project is to study gestural interaction patterns for
virtually every application to interact with on-screen buttons and
mobile recommender systems on smartphones, such as swiping
similar interface objects. Double Tap means to tap the screen
across the touchscreen, or shaking the device. The specific goal of
twice in rapid succession with one finger. Pinch/Spread is a two-
the work described in this paper is to map recommender functions
finger gesture. The user places two fingers on the screen and
moves them together (Pinch) or away from each other (Spread).
October 12–16, 2013, Hong Kong, China.
Paper presented at the 2013 Decisions@RecSys workshop in This is most commonly used for zooming in (Spread) and out
conjunction with the 7th ACM conference on Recommender Systems. (Pinch). One-Finger-Hold Pinch is a more complex two-finger
Copyright 2013 for the individual papers by the papers' authors. gesture. In this case, one finger rests on the screen, while a second
Copying permitted for private and academic purposes. This volume is
published and copyrighted by its editors.
17
finger moves on the screen to adjust a slider or other numerical • on-screen buttons,
value, for example. • menu options (the user has to select a specific "menu"
option1 to show additional buttons), or
Slide means to move a single finger over the screen in a
continuous motion. Slide is generally used for dragging objects • gestural interface options (cf. Section 2.1).
like sliders and slowly scrolling through views exceeding the The next subsection describes considerations for mapping
screen’s dimensions. Fling is a quick, long movement of one gestures to application-specific functions.
finger in one direction and can also be used for quickly scrolling
through list views. Flick (or Swipe) is a shorter gesture similar to 3.2 Considerations for Mapping Gestures to
the longer Fling and commonly used as Swipe-To-Delete in file Application-Specific Functions
systems: a Flick gesture performed on an item generally deletes
this item from a list. Another usage is moving to the next screen, Single Tap is commonly used for interaction with on-screen
resembling turning pages in a book. Shake Device and Tilt Device interface objects and should not be used for other application-
(along x, y or z axis) are free-form motion gestures with no screen specific purposes. The same applies to Slide and Fling for
interaction required. scrolling screens or dragging objects. Contrariwise, Double Tap is
not bound to any standard features and thus application-specific
Technically, any touch pattern can be drawn on the screen using features can be mapped to it. As Pinch/Spread is generally used
one or more fingers, e.g. a rectangular pattern. However, finding for zooming, mapping it to other application features may be
the balance between gesture detection precise enough to confusing as well. However, the One-Finger-Hold Pinch (OFHP)
distinguish different patterns, and vague enough to allow for user variation of this gesture is applied in our application.
errors when drawing the patterns is difficult. In addition,
explaining complex patterns to the user is challenging and Since no screen interaction is necessary for the free-form gesture
therefore, complex patterns are rarely used in mobile applications. Shake Device, this gesture may be used independently from any
interface restrictions, for example for application-wide functions.
2.2 Related Work An application-wide function can be called at any time, regardless
Previous research on the usage of gestures in mobile scenarios of the current screen of the application, e.g. the "home" button on
focused on the user acceptance of motion gestures in general and most mobile systems. Functions depending on viewing items on-
hardly applied these techniques for the interaction with screen may not be viable for use with Shake Device, since shaking
recommender systems. In own previous work, we designed a the screen makes focusing on displayed objects on the screen
minimalistic user interface for a map-based recommender based harder. The nature of the other motion gesture, Tilt Device,
on gestural interaction, but for the larger screens of tablets [4]. suggests either a use for simple actions like a binary +/- rating
(making use of the left-right or front-back movements of Tilt
Cho et al. propose a photo browsing system for mobile devices.
Device), or for any navigation function along two or three axes.
They compared three types of interaction: a tilt-based interaction
Tilt Device is not applied in our test application, because the
technique, an iPod wheel and a button-based browser to browse
application does not use binary ratings.
and search photos efficiently. The results show that the tilting
technique is comparable to the controllability of buttons, more 3.3 Test Application User Interface
interesting than the other techniques and performed better than the
In the test application, the user can use a search interface to select
iPod wheel [5]. Negulescu et al. examined the cognitive demands
among movie genres and find items. The search interface can be
of motion gestures, taps and surface gestures. They show that
reached from the start screen, main menu or through the options
these three techniques do not differ in reaction time. Moreover
menu. After searching, a list of corresponding items is shown
they found out that motion gestures result in much less time spent (Fig. 2, left). Users can scroll up and down the list, remove items
looking at the smartphone during walking than does tapping on from the list or select an item to display more details by using
the screen. Therefore motion gestures are advantageous in certain Single Tap. The item details screen (Fig. 2, right) shows
scenarios [6]. Rico and Brewster applied a different focus on
information for the selected movie and allows for bookmarking
motion gestures for mobile devices. They found out that location
and rating the item. In addition, an options menu is available on
and audience have a significant influence on a user’s willingness
every screen to return to the search screen or main menu of the
to interact with a mobile device by using motion gestures [7].
application (Fig. 2, right). The following functions are available
3. DESIGNING THE TEST APPLICATION and implemented by at least two input options each:
3.1 Overview • Bookmark: The user can bookmark an item by using on-
We implemented the prototype application for Android 2.2 screen or options menu buttons (Fig. 2, right), or by
(Froyo) and tested it on a Google Nexus One smartphone with using the Double Tap gesture in the item details screen
Android. The goal of the test application was to provide different • Find Random Item: Accessible application-wide
input methods for functions typically found in recommender through the options menu or by using the Shake Device
systems to test which interaction patterns the user would chose in gesture
the successive study. The selection of functions in our application • Save Search Parameters: This function is available in
is not really specific to mobile recommenders and considers the search screen via an on-screen button or by a Double
recommenders in a wider sense, i.e. taking also "search" Tap in this screen
applications into account. The scenario for the prototype is a • Find Similar: The item details screen shows three
movie search and recommendation application that resembles the movies similar to the selected one ("similar to this
Internet Movie Database (IMDb) mobile application (see
http://www.imdb.com/apps).
1
We provided at least two different input methods for each On most systems, a dedicated software or hardware button opens
application function, either up the options menu
18
movie" part in Fig. 2, right). The user has the option to 4.2 Log File Analysis
find more similar items by using an on-screen button or 16 persons with mixed backgrounds participated in the study.
the Flick gesture Other than a few users skipping a few tasks from the instruction
• Exclude Item: Available in the list view as an on-screen list, all subjects completed the given scenario. We first analyzed
button (Fig. 2, left) or via the Flick gesture the log file to understand which input options the users chose to
• Rate Item: Users can rate items in the item details screen complete a given task.
by selecting the "Rate" on-screen button (Fig. 2, right).
Then, a rating scale of 1 to 10 stars appears. The user Out of a total of 44 recorded usages, the Find Random function
can set his or her desired rating by either using the was initiated 26 times using the Shake Device gesture, and 18
rating scale as an on-screen button or applying the One- times using the options menu button (see Fig. 2, right). This
Finger-Hold Pinch (cf. Section 2.1) gesture. represents a 59.1% usage rate for the implemented gesture.
Interestingly, only one out of the 16 users elected to use both
available input methods; every other user exclusively used either
the gesture or the button for the three instances of Find Random in
our instruction list.
The Bookmark Item function is represented three times in the
scenario. The users chose to use the Double Tap gesture 27 out of
46 times (58.7%). However, at one instance in the scenario, the
activity in focus is the item list, which only implements
bookmarking via double tapping. In this case, 11 of 16 users
(68.8%) chose the Double Tap gesture, while the rest of the users
elected to take additional time to first open an item’s details page
and bookmark there. While the users were on an item’s details
page, they called only 16 of 35 (45.7%) instances of Bookmark
Item using the Double Tap gesture. All differences to 100% in this
paragraph are due to the uses of the on-screen bookmark button –
the options menu button was never used.
The use of the Save Search Parameters function was requested
only once in the scenario and can be called using Double Tap or
an on-screen button. This is the function with the clearest favorite
among the input methods: 15 out of 16 users (93.8%) chose the
Figure 2. List of recommendations (left). on-screen button.
Item details with options menu (right). The scenario contained two instances of the Exclude
Recommended Item function, operable via Flick gesture or an on-
4. USER STUDY screen button. 18 of 32 (56.3%) calls were made using gestural
interaction. A relatively high number of users used both input
4.1 Study Setup and Methodology methods for this task: 4 out of 16 participants (25%). This is even
We have conducted a user study to find out what input method for though the two instances of the Exclude Recommended Item task
a given function is preferred by the test users. The evaluation was occurred directly after each other in our task list.
performed with each of the participants individually. To start,
each user was given an explanation of the application and was Rate Item and Find Similar Item each occur two times in the
then allowed to practice navigating the different functions and scenario. For both, a clear preference towards the standard input
input methods for about ten minutes. The participants then had to method of an on-screen button can be seen: for Rate Item, only 10
perform a set of 18 instructions in the application in a certain of 32 instances (31.3%) were operated with the One-Finger-Hold
order. The list mentioned the required tasks only; the input Pinch gesture. Even more one-sided, the Find Similar Item
method to perform them was not specified. By doing so, we tested function was only initiated using Flick in 3 of 32 cases (9.4%).
which input method the test persons found more intuitive to use The remaining percentages represent instances of functions called
for a certain task. The beginning of the sequence of instructions via on-screen button.
read as follows: (1) Find Random Item, (2) Find Similar Item, (3)
Rate Item, (4) Open Main Menu, (5) Open My Recommendations, 4.3 Survey Results
(6) Exclude Item from Recommendations, and so on. Some of the In the first part of the survey we asked the participants how
requested functions appeared several times in the list, for example intuitive they find the input methods for the six functions on a
Find Random Item was requested three times. This was used to scale from 1 to 5. Figure 3 illustrates the results with a higher
test whether participants would change their preferred input number meaning "more intuitive". In general, the results
method for a particular function during the experiment. correspond to the log file very well: input methods that were
actually preferred and used by the participants received higher
We recorded every user action in a log file. After a test user grades for intuitivity. For example, the participants find the on-
completed the scenario, he or she had to fill out a survey screen buttons for Save Search and Find Similar very intuitive.
concerning his or her opinions about the input methods for the On the other hand, the Shake Device for Find Random Item,
requested instructions and about the handling of the gestures in Double Tap for Bookmark and Flick for Exclude Item gestures
particular. received higher grades in comparison with on-screen or option
menu buttons.
19
users preferred the simpler, easier to handle gestures over the
more complex ones. Complex gestures like One-Finger-Hold
Pinch must be carefully calibrated for ease of handling. Omitting
on-screen buttons is only an option in activities where content
space is rare, in our case the overview list of items. For the item
detail screen, simply touching a button was the favorite input
method most of the times. The options menu was not very popular
in any of the used cases. This is likely due to the fact that opening
the options menu is an extra effort that users do not tend to make
Figure 3. Average of users' ratings how intuitive each when other input methods are available.
function’s input method was.
While Double Tap for bookmarking items was received very well,
The next question was whether inclusion of an on-screen button the Double Tap gesture for Save Search Parameters was not very
was worth the necessary screen space for it. Our users mostly popular and received low grades for intuitivity. This may be due
were in favor of it: the majority of users denied this question for to the layout of the corresponding screens because users might
Exclude Item only (Fig. 4). Interestingly, this is the only on-screen have the fear of accidently tapping on other interface elements. In
button in the list view (Fig. 2). essence, the use of gestural interaction patterns seems to depend
on the actual screen and function detail. Interestingly, users did
not change their preferred input mode much during the test: they
mostly used the same method for the same task throughout the
scenario. Users with more experience with touchscreen devices
were more open towards gestures than users with less experience.
Future work includes studying in more detail how more complex
gestures can be introduced in mobile recommender systems to
improve user interaction. Moreover, a long-term study would be
interesting because user acceptance might change if smartphone
users get more and more used to complex motion gestures.
Figure 4. Screen space usage for on-screen buttons 6. REFERENCES
The goal of the next part of the survey was to determine the user’s [1] Konstan, J.A., and Riedl, J. 2012. Recommender systems:
favorite input method for each function. The distribution of from algorithms to user experience. User Model. User-Adapt.
choices for each function is shown in Fig. 5 and is comparable to Interact. 22, 1-2 (April 2012), 101-123. DOI=
the grades for intuitivity: interaction patterns that users perceived http://dx.doi.org/10.1007/s11257-011-9112-x.
as intuitive were chosen as favorite input method. [2] Ricci, F. 2011. Mobile recommender systems. J. of IT &
Tourism. 12, 3 (April 2010), 205-231. DOI=
http://dx.doi.org/10.3727/109830511X12978702284390.
[3] Saffer, D. 2008. Designing Gestural Interfaces. O'Reilly,
Sebastopol.
[4] Schulze, F., Woerndl, W., and Ludwig, M. 2012. A gesture-
based interface for map-based exploratory search on tablets.
In Proc. Mobility and Web Behavior Workshop, MobileHCI
'12 Conference (San Francisco, CA, September 21 – 24,
Figure 5. Selecting only one input option for each function 2012).
We also asked the test users about their prior experience with [5] Cho, S.J., Murray-Smith, R. and Kim, Y.B. 2007. Multi-
touchscreen devices and analyzed whether it would relate to context photo browsing on mobile devices based on tilt
differences in the results. The most significant difference was that dynamics. In Proc. of the 9th International Conference on
62.5% of the users with more prior touchscreen experience rated Human Computer Interaction with Mobile Devices and
the Shake Device gesture as intuitive, while only 12.5% did so Services (Singapore, September 09 – 12, 2007). MobileHCI
among the users with less experience. We noted a similar '07. ACM, New York, NY, 190-197. DOI=
difference regarding the Flick gesture. http://doi.acm.org/10.1145/1377999.1378006.
Concerning the ease of handling of the four gestures, the [6] Negulescu, M., Ruiz, J., Li, Y., and Lank, E. 2012. Tap,
participants considered all gestures, except One-Finger-Hold swipe, or move: attentional demands for distracted
Pinch (OFHP), as easy to handle in general. One of the problems smartphone input. In Proc. of the Int. Working Conference on
with OFHP was that lifting a finger while adjusting the desired Advanced Visual Interfaces (Capri Island, Italy, May 21 – 25,
rating for item ends the rating process. In addition, the calibration 2012). AVI '12. ACM, New York, NY, 173-180. DOI=
for the rating scale of one to ten stars was difficult. So this gesture http://doi.acm.org/10.1145/2254556.2254589.
might be more suitable for simpler tasks with fewer options. [7] Rico, J., and Brewster, S. 2010. Usable gestures for mobile
interfaces: evaluating social acceptability. In Proc. of the
5. DISCUSSION AND CONCLUSION SIGCHI Conference on Human Factors in Computing
The results of the study presented in this work may be used to Systems (Atlanta, GA, April 10 – 15, 2010). CHI '10. ACM,
improve the design of user interfaces for mobile recommender New York, NY, 887-896. DOI=
systems and other similar applications. Our study showed that http://doi.acm.org/10.1145/1753326.1753458.
20