Touch or shake? The interaction effect between hand gesture and reward setting on the enjoyment of gamified marketing

Abstract

Gamification has recently become a popular tool in supporting company's marketing activities The effectiveness of gamification in the context of marketing promotion is largely due to the nature of games: highly interactive process, uncertain outcome, and overall enjoyable experience which is constituted by the former two. In this paper, we focus on mobile hand gestures which is a new interaction modality in mobile gamification, aiming to provide some initial insight to our understanding how the interaction between hand gestures-based process and reward outcome setting influences consumer experience of enjoyment. A mobile-based online experiment was conducted and provides evidence to bolster our prediction. That is when reward outcome is set as uncertain (vs. certain), game process with motion gestures like shaking (vs. surface gestures) will increase consumer enjoyment in gamified marketing, such interaction effect is mediated by perceived control. Findings of this research will provide implications for both marketing practitioners and mobile game designers.

Introduction

Gamification, defined as the use of game design elements in non-game contexts, has recently become a popular tool in supporting company’s marketing activities (Deterding et al. 2011; Huotari and Hamari 2012) . The effectiveness of gamification in the context of marketing promotion is largely due to the nature of games: highly interactive process, uncertain outcome, and enjoyable overall experience which is constituted by the former two (McGonigal 2011). To be specific, traditional marketing promotion often distribute reward straightforward without the interaction process, and the reward magnitude is fixed and known to target consumers. Differently, gamified marketing promotion usually requires an interactive process of completing tasks before distribute reward. Besides, final outcome of reward is often set as uncertain with probabilistic magnitude (Shen, Fishbach, and Hsee 2015). For example, “complete the task and you will win a mysterious gifts!” or “do your best to win the biggest prize ”. A burgeoning number of brands or companies have started to consider creating better consumer experience through gamification designing (Lucassen and Jansen 2014) . However, as the most basic and vital components in games, how should the process design of interaction modality be aligned with the outcome design of reward? In this research, we focus on a newly developed modalities of interaction in mobile gamification – hand gesture-based interaction. Such as surface gestures that are based on touching-sensor technology (e.g. touching, scrolling, and swiping), and motion gestures that are based on motionsensor technology (e.g. shaking, tilting, and rotating) (Ruiz, Li, and Lank 2011; Wobbrock, Morris, and Wilson 2009). Examples are pervasive in gamified marketing practices. In the mobile application for Domino’s Pizza, consumers can activate a pizza slot machine game by shaking the phone, and the game will randomly choose one topping for consumers (Forbes 2012). Rarely used in traditional marketing channels (e.g. TV, desktop, digital signage), these mobile interactive gestures have brought a unique process experience. Despite much work on hand gestures, still little is known about the role of such new element in game process and its relationship with reward outcome in gamified marketing.

Therefore, the current paper aims to provide some initial insight to our understanding how the interaction between hand gestures-based process and reward outcome setting influences consumer experience of enjoyment. The findings of this research will provide implications for both marketing practitioners and mobile game designers on gamified marketing design, especially the integration of hand gestures into gamification design. 2.

Literature Review

2.1. Hand Gestures and Bodily Involvement

Different from traditional input with keyboard or mouse, mobile platforms enable various new modalities via sophisticated sensor technologies (Wobbrock et al. 2009). Gesture-based interactions have been widely integrated into mobile games (Sirlantzis, Mentzelopoulos, and Protopsaltis 2015). One input modality involves surface gestures, such as clicking, dragging, and moving objects on the screen of a mobile device, which allows users to interact directly with the object on a touch-sensing screen in 2D space (Wobbrock et al. 2009). Another input modality involves a set of motion sensors (e.g. accelerometers, gyroscopes, orientation sensors), where users can engage in motion gestures in 3D environment, like shaking, tilting, or rotating a mobile phone (Daiber, Li, and Krüger 2012; Ruiz et al. 2011) . Except for the distinction of input technology, we further propose that surface and motion gestures are different at physical characteristics. As suggested by Ruiz et al. (2011), one aspect to explore characteristics of hand gestures is from their physical perspective. Physical characteristics may be understood from the level of kinematic impulse (e.g. low, moderate or high), number of motor axis (e.g. motion occurs around a singleaxis, tri-axis etc.), level of complexity (e.g. simple or compound). In a similar vein, Rempel, Camilleri, and Lee (2014) distinguish hand posture features of gestures based on two interaction interfaces. We therefore posit that the level of body involvement differs between surface and motion gestures. On the touch-sensing interface, surface gestures require moving of fingertips, it is precisely manipulated through fingers with mainly thumb, index and sometimes middle fingers, such as pinching, flicking and swiping (Tucker and Ellis 1998). While gestures in 3D-space decouple the hand from a touch-sensing interface, motion gestures such as shaking, tilting and rotating mobile devices, are performed with a little movement of fingers but large palm contact area and more body involvement, including fingers, hands, waist, and even arms. In a sense, motion gestures usually require more physical involvement and requires more effort than surface gestures. Some research argue that users will prefer interaction process with less effort, however, we propose an opposite game scenario which involves reward outcome setting of uncertainty (Cutright and Samper 2014) .

2.2. Reward Uncertainty and Perceived Control

Reward outcome setting, as part of the basic game design, plays a vital role in influencing consumer’s motivation and valuation of the marketing promotion games. According to Shen et al. (2015), the setting of reward certainty denotes rewards with a fixed and known magnitude, (e.g., a 100% chance of getting X), while reward uncertainty includes situations in which at least two potential reward magnitudes are available no matter whether or not the winning probability is known (i.e., a 50% chance of getting X and a 50% chance of getting Y, both X and Y are positive gains). Similarly, Lee and Qiu (2009) have differentiated two reward settings based on whether or not prizes are clearly notified to the game participants. In the certain reward setting, there is only one certain reward, whereas uncertain reward settings provides several potential rewards and participants are unclear about what they are going to get. Based on these definitions, in the current paper reward certainty is defined as 100% chance of getting one certain reward after finish the task, and uncertainty is defined as situations in which more than one potential reward is available no matter the probability of wining the reward is known or not, game participants are uncertain about which exact reward they would get before they finish the game tasks.

Earlier research suggests that when facing uncertainty people tend to feel deprived of personal control. More recent work has pointed out that people have a natural tendency to restore perceived control (Cutright and Samper 2014) . That is, feelings of control deprivation will lead people to strive for control restoration. The desire of regaining control is analogous to motivation, which is to reduce the discrepancy between the current and expected state when one feels lack of control in the process of goal pursuit (Carver and Scheier 2001) . Similar work on the relationship between uncertainty and motivation also suggests that when people purse a reward with uncertain magnitude, an increase in motivation will result in illusions of control, leading to a belief that good results are for hard working people (Langer 1975).

One way to solve the discrepancy and restore feelings of control is to exert effort (Cutright and Samper 2014) . Effort is regarded as the primary means to get sense of control. Prior research has shown that the more effort people devote, the more they suggestively believe that they can control the outcomes with the help of the effort, and the greater sense of efficacy people experience (Carver and Scheier 2001; Higgins 2012; Lee and Qiu 2009; Schunk 1983) . In the marketing literature, evidence has been accumulated that when people feel low in personal control, they tend to prefer high-effort products in order to get self-empowering and re-establish their senses of control (Cutright and Samper 2014) . To sum up, there is a psychological need to acquire senses of control when facing the uncertain reward setting and exerting more effort is one way to satisfy such need.

Conceptual Framework

As analyzed earlier, motion gestures usually require more physical involvement compared to surface gestures, given that it elicits movement of hand, waist and even arm. The more is body involved, the more bodily effort is invested. In the context of our paper, when the reward setting is uncertain, it requires high effort to reduce the feeling of uncertain, and thus reestablish the perceived personal control. Hence, the high bodily-involved nature of motion gestures renders certain congruency between reward uncertainty and motion gesture. Indeed, much work in the literature mentions player’s sense of control in games, which is one of key factors that influence gaming experience (Komulainen et al. 2008; Korhonen, Montola, and Arrasvuori 2009; Sánchez et al. 2012). One widely recognized model of flow proposes that perceived control will influence enjoyment, especially in the context of games, where players’ control originates from game interface and input devices (Csikszentmihalyi 1990; Sweetser and Wyeth 2005) . The ability to exercise certain sense of control over actions during games will further induce some sense of enjoyment. Therefore, we hypothesize as below: H1: When reward is set as uncertain (vs. certain), using motion gestures like shaking (vs. surface gestures) will increase game enjoyment.

H2: The interaction effect between hand gesture and reward setting is mediated by perceived personal control.

Experiment

The purpose of this experiment was to examine H1 and H2. In this research, we selected commonly used gestures of touch and shake, which belongs surface gesture and motion gesture respectively. We designed a lucky wheel game, in which participants either touch a button on the wheel or shake the smartphone to reveal the final reward. We manipulated reward uncertainty by varying the number of potential rewards. To be specific, reward certainty condition offered one fixed reward, whereas reward uncertainty condition offered two potential rewards. In this experiment, we also recorded the duration of gesture interaction as an index of effort investment. 4.1.

Method

One hundred and thirteen university students were recruited online as previous experiments. We used a 2 (reward certain vs. uncertain) ×2 (touch vs. shake) between-subjects design. Participants were told that they were going to play a lucky wheel game to win a reward coupon. Depending on the gesture condition, participants were instructed to either touch a button or shake the smartphone to start and end the coupon game. Half of the participants were assigned to the reward certain condition, where they were told that they have a 100% chance of getting a mysterious reward. The remaining half of participants were assigned to the reward uncertain condition, where they were told that they will have a 100% chance of getting a mysterious reward, but uncertain which one of the two potential rewards. The two rewards are of equal value. We had all the participant win the Coke. Finally, participants filled a questionnaire assessing perceived control and game enjoyment by a 9-point scales. As a behavioral measure of effort, we also recorded the interaction time of each participant, which started from the first gesture input to the last gesture (i.e., the time since when there was no more input within 3 seconds).

4.2. Results

Game enjoyment. We firstly completed a 2 ×2 ANOVA on game enjoyment. Results showed no significant main effects of either hand gesture or reward uncertainty (Fs < 1). However, the interaction between gesture and reward uncertainty was reliable(p = .023). When the final reward was uncertain, participants who used motion gesture evaluated the coupon game as more enjoyable than those who used surface gesture (Mmotion = 4.33, Msurface = 3.01; F(1, 113) = 5.22, p = .024). However, there is no difference between two gestures in terms of game enjoyment when the reward was certain (F(1,113) =1.00, p =.319). The result shows that H1 is supported.

Perceived control. Similarly, another 2 ×2 ANOVA was conducted on perceived control. Neither the main effect exists. But there was a significant interaction of the two variables on perceived personal control (F(1,109) = 7.11, p = .009). To be specific, when the reward was uncertain, motion gesture lead to higher levels of perceived personal control as compared with surface gesture (Mmotion = 5.03, Msurface = 3.23; F(1,109) = 9.46, p = .003). Such difference between two gestures was attenuated when the reward was certain (F < 1).

Moderated mediation analysis. Based on the above results, we further conducted a moderated mediation analysis through Bootstrapping with 5000 samples. Results suggest a significant indirect effect of the interaction between gesture and on game enjoyment (95% CI = [.52, 3.38]). In addition, the mediating effect of perceived control was only reliable when the final reward was uncertain (95% CI = [.54, 2.61]). In summary, H2 is supported.

Effort investment. We recorded the interaction time as an indicator of participants’ effort investment in the game. A 2×2 ANOVA showed a significant interaction between hand gesture and reward uncertainty on interaction time, F(1, 109) = 4.93, p = .028. Significance between two gestures only found in the reward uncertain condition, participants invested more effort with motion gesture than surface gesture (Mmotion = 1384ms, Msurface = 413ms; F(1,113)=5.65, p=.019). This indicates that the more people feel in control, the more they are willing to perform the behavior (Cutright and Samper 2014) .

We further added a new experiment group (N=55) to explore if there is any difference on effort investment when the probability varies under the condition of reward uncertain. We set the probability distribution of 40%-60%. Paired comparisons of shaking gesture had done between each two of the three conditions. Results shows in Table 1. However, paired comparisons of touching gesture shows no significant difference between three conditions (p>.5). Therefore, only the game mechanics of whether the reward is certainty or uncertainty affects the level of effort investment, but the probability will not matter, due to the probability neglect (Rottenstreich and Kivetz 2006; Shen et al. 2015). Last, we conducted a linear regression analysis of the relationship between effort investment of shaking gesture and perceived personal control. The result shows that effort positively influence the perception of personal control (β=.207, t=2.74, p=.007, R2adj=.037), which further explains how the participants get the feeling of control in the mobile coupon game through the shaking gestures.

Discussion

The results has proved our hypotheses and further provided evidence to bolster the interactive effect between hand gesture and reward uncertainty on game enjoyment. Specifically, motion gesturebased process bring greater game enjoyment under the uncertain reward outcome. In addition, perceived personal control was shown to mediate the relationship. Moreover, interaction time, as a behavioral measure, more directly demonstrates the influence of gesture and reward uncertainty on effort investment, which serves as an effective way to retain personal control. These findings suggest that people are sensitive to the general setting of the reward, like certainty or uncertainty. Consistently with prior work, we observe that the more effort participant devote, the more control they seem to have (Higins 2012).

Prior researches on gamification mainly focus on the traditional game design elements, such as badges, points, leaderboards etc., and explored each element isolated. The current study extends the research stream of gamification by exploring the role of hand gestures, which is a new emerging but indispensable design element in mobile context, and we further explore how game design elements aligned together. Besides, the findings of underlying psychological mechanism of perceived control driven by motion gestures under uncertain situation deepens our understanding of the two major types of hand gestures, which contribute to the literature of human-computer interaction studies. This research also provide implications for both marketing practitioners and mobile game designers. The results highlight that to enhance consumer’s game enjoyment in gamified marketing design, hand gestures should be aligned with reward outcome setting by considering the psychological nature of each gestures. For brands or products that provide a randomized reward situation, shaking will work better than mere touching for it will give customer a sense of control, and will thus increase the enjoyment in the participation. More broadly, firms and designers may follow the psychological nature of gestures, that any gestures that designed to be more bodily involved may increase the perception of control in the game when facing uncertainties. Therefore, gamified marketing designs should not only be limit to the researched gestures.

The major limitations in our research is that the mobile coupon game was designed in a very simple form to reduce the interference of irrelevant game elements in the experiment. However, this also reduces the overall aesthetics and somewhat influences the enjoyment of the game. Hand gesture is not a new topic in psychology nor in HCI. However, there are a relatively limited number of psychological studies of hand gesture under HCI context. With the fast development of mobile technology, as well as VR, more interactive gestures and wearables are integrated in the gamification. Whether and how these newly invented gestures will affect the mobile gamification effectiveness is worth exploring in the future.

Higgins, E.Tory. 2012. Beyond Pleasure and Pain: How Motivation Works. Oxford University Press. Huotari, Kai and Juho Hamari. 2012. “Defining Gamification - A Service Marketing Perspective.” P. 17 in Proceeding of the 16th International Academic MindTrek Conference on - MindTrek ’12. Komulainen, J., J. Tkatalo, M. Lehtonen, and G. Nyman. 2008. “Psychologically Structured Approach to User Experience in Games.” Pp. 487–90 in Nordic Conference on Human-Computer Interaction 2008,. Korhonen, Hannu, Markus Montola, and Juha Arrasvuori. 2009. “Understanding Playful User Experience through Digital Games.” Pp. 274–85 in International Conference on Designing Pleasurable Products and Interfaces.

Langer, E. J. 1975. “Illusion of Control.” Journal of Personality & Social Psychology 32(2):311–28. Lee, Yih Hwai and Cheng Qiu. 2009. “When Uncertainty Brings Pleasure: The Role of Prospect Imageability and Mental Imagery.” Journal of Consumer Research 36(4):624–33.

Lucassen, Garm and Slinger Jansen. 2014. “Gamification in Consumer Marketing - Future or Fallacy?” Procedia - Social and Behavioral Sciences 148(2011):194–202.

McGonigal, Jane. 2011. Reality Is Broken: Why Games Make Us Better and How They Can Change the World.

Rempel, David, Matt J. Camilleri, and David L. Lee. 2014. “The Design of Hand Gestures for HumanComputer Interaction: Lessons from Sign Language Interpreters.” International Journal of Human Computer Studies 72(10–11):728–35.

Rottenstreich, Yuval and Ran Kivetz. 2006. “On Decision Making without Likelihood Judgment.” Organizational Behavior and Human Decision Processes. Ruiz, Jaime, Yang Li, and Edward Lank. 2011. “User-Defined Motion Gestures for Mobile Interaction.” P. 197 in Proceedings of the 2011 annual conference on Human factors in computing systems - CHI ’11. ACM Press.

Sánchez, JoséLuis González, Francisco Luis Gutiérrez Vela, Francisco Montero Simarro, and Natalia Padilla-Zea. 2012. “Playability: Analysing User Experience in Video Games.” Behaviour & Information Technology 31(10):1033–54.

Schunk, Dale H. 1983. “Ability versus Effort Attributional Feedback: Differential Effects on Self-Efficacy and Achievement.” Journal of Educational Psychology 75(6):848–56.

Shen, Luxi, Ayelet Fishbach, and Christopher K. Hsee. 2015. “The Motivating-Uncertainty Effect: Uncertainty Increases Resource Investment in the Process of Reward Pursuit.” Journal of Consumer Research 41(5):1301–15.

Sirlantzis, Konstantinos, Markos Mentzelopoulos, and Aristidis Protopsaltis. 2015. “Human Computer Interaction Using Gestures for Mobile Devices and Serious Games: A Review Sotirios Spanogianopoulos.” in Proceedings of 2014 International Conference on Interactive Mobile Communication Technologies and Learning, IMCL 2014.

Skinner, B. F. 1969. Contingencies of Reinforcement: A Theoretical Analysis. Appleton-Century-Crofts. Sweetser, Penelope and Peta Wyeth. 2005. “GameFlow: A Model for Evaluating Player Enjoyment in Games.” Computer Entertainment 3(3):3–3.

Tucker, Mike and Rob Ellis. 1998. “On the Relations between Seen Objects and Components of Potential Actions.” Journal of Experimental Psychology: Human Perception and Performance 24(3):830–46. Wobbrock, Jacob O., Meredith Ringel Morris, and Andrew D. Wilson. 2009. “User-Defined Gestures for Surface Computing.” P. 1083 in Proceedings of the 27th international conference on Human factors in computing systems - CHI 09. ACM Press.

References

Carver , Charles S. and Michael F. Scheier . 2001 . “Optimism, Pessimism, and Self-Regulation.” Optimism & Pessimism: Implications for Theory, Research, and Practice 31-51.

Csikszentmihalyi , Mihaly. 1990 . Flow:The Psychology of Optimal Experience .

Cutright , Keisha M. and Adriana Samper . 2014 . “ Doing It the Hard Way: How Low Control Drives Preferences for High-Effort Products and Services. ” Journal of Consumer Research 41 ( 3 ): 730 - 45 .

Daiber , Florian, Lianchao Li , and Antonio Krüger. 2012 . “ Designing Gestures for Mobile 3D Gaming .” P.

1 in Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia - MUM '12.

Deterding , Sebastian, Dan Dixon, Rilla Khaled, and Lennart Nacke . 2011 . “From Game Design Elements to Gamefulness.” P. 9 in Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek ' 11 .