What makes a trophy hunter? An empirical analysis of Reddit discussions

Abstract

In this paper, an empirical data-driven analysis of online discussions of meta-game reward systems is carried out. The data is collected from one of the biggest online discussion forums called Reddit and a text-mining technique called topic modeling is employed. Over 46000 discussion threads from the two most relevant subreddits /r/xboxachievements and /r/Trophies are analyzed and the results of topic modeling shows not only interesting topics but also the (dis)similarity between two text sources the temporal trends of topics. We have found that the volume of related discussions shows an ongoing trend. The topic model results have also revealed that some game genres are more prevalent than others and the (dis)similarities between text sources have been also discovered.

Keywords

• trophy hunting
• Reddit
• topic modeling

Introduction

It has been noted that games can also be gamified [ 10 ], which has attracted researchers’ attention [ 9,14 ]. Early work on applying badge systems to non-game contexts have also used game achievement systems as a source of inspiration [ 19 ]. One of the gamification design implementations is a meta-game reward system, which awards visual indicators for completion of tasks and acts as an overarching game in which players obtain rewards through playing across different games [ 5 ]. In game industries, meta-game reward systems have been widely used such as badges (STEAM), achievement points (Xbox) or trophies (PlayStation), etc. [ 12, 15, 30 ].

Reddit, a major participatory culture platform [ 17 ], wh has more than 138,000 active subreddits (each represents a focused sub-community) and more than 330 million users. Due to the number of discussions, the text content has been a valuable resource in various fields [ 16, 22 ]. There are two subreddits related to our interests: /r/xboxachievements, a place to “show off achievements” has more than 3 thousand users and /r/Trophies has more than 25 thousand “trophy hunters”. They both yield an abundance of discussions that reflect the gamers or potential gamers’ perception. However, despite containing valuable information, they have not been well explored.

This work performs a large-scale data-driven analysis of gamers’ interests and attitudes toward two reward systems with comprehensive data collection of discussion threads in the above described two sources. We use topic modeling to computationally extract themes of discussion from the huge corpus. How the prevalence of the extracted themes varies over the two subreddits and over time are further estimated.

The rest of the paper is structured as follows. Section 2 describes related work. Section 3 describes data collection and text mining methods. Section 4 describes the resulting extracted themes. Section 5 provides analysis and discussions. Conclusions, limitations, and opportunities can be found in Section 6. 2

Related work

Cruz et al. [ 5 ] investigated meta-reward systems for Xbox and Playstation and how they affect game players’ experience. They collected discussions from focus groups of 36 console players and used Self-determination theory (SDT), Organismic integration theory (OIT), Cognitive evaluation theory (CET) and Signaling theory (ST) as the framework to analyze the discussions. SDT, OIT, CET disclose an individual’s motivations [ 26 ] and ST provides a framework to understand how the information is sent and interpreted when communicating [ 6 ]. Many studies shed light on importance of game reward systems towards players’ enjoyment and motivation. Jakobsson, based on study of the Xbox 360 achievement system, indicates it not only works as a reward system but also as a multiplayer online game in which all Xbox players participate compulsorily [ 11 ]. Wang and Sun suggest players enjoy game rewards and react to the motivation they provide while indicating that reward mechanisms foster intrinsic motivation and give extrinsic rewards based on relevant psychological theories [ 27 ]. By studying the reward system of World of Warcraft, Rapp emphasizes importance of rewards in shaping player experiences but suggests to consider reward designs carefully to “surpass the exclusive employment of extrinsic motivators in gamification design” [ 24 ]. Many studies obtained positive outcomes adopting design similar to game reward systems in gamification and serious games [ 8, 20 ].

One example of using empirical data is the work of Wells et al. [ 28 ] who analyzed 30,227 PlayStation user profiles to explore how factors including, PlayStation Plus subscriptions, player regions, etc., influence earning achievements. Using data from Steam, O'Neill indicates achievements may incentivize more playtime from players who would have otherwise played less [ 21 ]. Players' achievements data and data mining methods are also used in analyzing their performance in serious games towards enhancement of players' motivation and engagement [ 2 ]. Many studies have chosen Reddit as their data source when researching users' opinions on various topics [ 13, 29 ]. We didn’t find works analyzing text of online discussion regarding meta-game reward systems, perhaps due to difficulties when it comes to a large amount of data. We tackle the issue by a computer-aided solution explained in Section 3.

We aim to answer three research questions. RQ1. Is discussion of the meta-game reward system an ongoing trend? RQ2. What kind of games are more mentioned concerning meta-game reward systems? RQ3. What are similarities & dissimilarities of discussion between Xbox and PS about the meta-game reward system? 3

Method

Data Collection and Processing. Reddit threads from the two subreddits between 2014 and 2019 (until the 17th of October) were collected via the pushshift API1. The text was lemmatized and stop words removed. Each submission with its comments was aggregated as a document. We eliminated submissions without comments as they did not evoke discussions. Documents with less than five words were filtered out after processing. In total, we obtained 1384 documents from /r/xboxachievements and 45145 documents from /r/trophies, more details shown in Table 1.

Text Mining. We used a text mining technique called topic modeling to analyze the collected online discussions. It is a probabilistic model that represents each document as a mixture of several latent topics. Each topic speaks for a specific semantic theme within the data collection. The topics are not provided by a human expert but are learned by an algorithm in a data-driven manner. Topic models such as Latent Dirichlet Allocation [ 3 ] have been used in domains including game studies (e.g. [ 7 ]).

The basic representations found by a topic model are: 1) The strength (prevalence) of the latent topics; each topic has a prevalence in each document, represented as proportions that sum to one, and can be summarized over different data subsets. 2) The vocabulary of the topics, represented as a distribution over typical words in each topic. In this work, a more advanced approach called Structural Topic Model (STM, [ 25 ]) is employed. It is a topic model where the strength of topics within a document can depend on document-level covariates, enabling us to model both text content of the documents, and how they are affected by annotations. We take the document source (subreddit) and submission time (initial time of the discussion series) as document-level covariates. The two covariates are taken to model (dis)similarity between two subreddits and the temporal dynamics. Compared to conducting topic modeling in two sources separately (e.g. [ 1 ]), this provides an integrated solution via directly treating the effect of the source as a learned parameter in the model.

We use held-out likelihood as the criterion to decide the number of topics K, the user-specified parameter of STM. For efficiency we first searched from K = 10 to 500 with an interval of 5; the best setting was K = 355. We investigated exhaustively from K = 351 to 359 and K = 355 was still the best. We set up 10 initializing models with K = 355; the model with best semantic coherence [ 18 ] was adopted as the final model. 4

Results

tics of the top words and example documents of the topic (having high prevalence of the topic). Topics whose top words prominently contained words for specific game titles or characters were denoted as specific topics about those games; such topics might concern multiple games, such as a game series, if often discussed together. While separation of topics on specific games into genres cannot be made fully conclusively, each topic was tagged with one or more genres. Non-game-specific topics were labeled according to semantic themes they contained. Topics with no clear semantic theme were labeled as linguistic topics that provide nuance to the Reddit discussion but which are not a focus of our analysis. Note that each topic may weakly contain other thematic content besides the main one marked as their label.

The /r/Trophies subreddit introduced a bot ‘PSNTrophyBot’ in 2018 which describes a game’s trophies, for example, “God of War has 37 trophies: 22 bronze, 8 silver, 6 gold, and platinum...”. The bot adds ‘I am a bot’ to the end of its posts; thus topics where ‘bot’ was among the top words were considered related to bot activity and are shown in the online full table but not focused on in later analysis. A selection of found topics are in Tables 2 - 32. We discuss selected examples next.

Game related topics. 105 of the 355 topics (30% of the topics) turned out to be game-specific (see online full table). The most discussed game-specific topics were about games PlayerUnknown’s Battlegrounds, the ‘Infamous’ series, point and click adventure games from Telltale Games, the ‘Far Cry’ series, Uncharted the Lost Legacy and Conan Exiles, the ‘Assassin’s Creed’ series, the Lego video games series, the ‘God of War’ series for Playstation portable, Demon’s Souls and Bloodborne, and Rocket League and Teslagrad. We do not present their top words in Tables 2-3 as they are game-specific: for example, the Telltale Games topic contained words like ‘walk’, ‘telltale’, ‘episode’, ‘dead’, ‘tale’, ‘batman’, ‘borderland’, and ‘season’ referencing games like ‘The Walking Dead’ and ‘Tales from the Borderlands’. However, examples of game-specific time courses are shown in Fig. 1 to demonstrate the rise and fall of a game’s popularity for trophy-hunting: PlayerUnknown’s Battlegrounds (Plot (a), Fig.1) is slowly declining, whereas Grim Fandango and Day of the Tentacle (Plot (b), Fig.1) have a strong Playstation peak in 2017. See the full set of plots for other games: e.g. Tomb Raider has little change on Playstation but has a peak around 2016 on Xbox, and Uncharted a broad Playstation peak around 2017. The broader the peaks, the more enduring the game and its reward mechanism are for trophy hunters; the strongest ones can serve as examples for future design.

2 Full table at shorturl.at/hrtH5, all time plots at shorturl.at/ t/uFGN. Topic Boss fights and weapon upgrades Shooting and kills Fan hopes for sequels Purchasing modes and preordering Getting trophies Milestone Frustration and perseverance Enjoyment and motivation Multiplayer gaming Bugs in games Memories and nostalgia Attempting and memorizing moves and routes

Reward related topics include going for certain types of rewards, earning rewards, reward difficulties, congratulates others, and discussing rewards themselves. Some share feelings of accomplishment, topics Earning trophies and happiness, Milestone, Enjoyment and motivation indicate pride and celebration. In Milestone, a user stated: “...I’d share my 100% of this as I hit level 100 yesterday, the game is consuming my life at the moment!”. Some topics imply the achievement difficulty level, e.g. Easiness and achievability and Difficulty of getting trophies; topics like Achieving trophies after long time and Dedication required from players imply perseverance and commitment. In Easiness and achievability, a user stated: “...I just kept at it until I got em. Otherwise, the trophies in both Pac-Man games are pretty simple...”. Topics about feelings (Fun of playing) and fandom (Memories and nostalgia indicate players’ memorable experiences of the games, and lifestyle (Time allocation to playing, relationship) indicates players’ commitment to playing as part of their life. Wastefulness and pointlessness expresses a feeling of time waste, and

Frustration with difficulty and perseverance, Games becoming boring or stale,

and Grinding and monotony indicate mixed feelings for the reward system. A user stated in Wastefulness and pointlessness: “...played both games over 200 hours and fully enjoyed them but can never waste time collecting pointless...”. Seek information about a trophy reflects information sharing among users.

Gameplay related topics. These topics indicate players hunting for trophies aim to find information to excel and to be aware of the games’ problems and updates, including gameplay experiences (Multiplayer gaming, Fighting moves: combos, countering and blocking and Boss fights and weapon upgrades) and technical aspects (Patches and glitches, Bugs in games, Saving and restoring progress, corruptions and backups). Enjoyment and recommendation of entertainment and Fan hopes for sequels link individual games to larger entertainment culture. Buying in packs for discounts, Downloadable content and expansions and Purchasing modes and pre-ordering indicate financial planning of trophy hunters.

Temporal Dynamics. Trends of selected topics are in Fig 1. E.g. Grinding and monotony on Playstation peaks in 2016 but is rising on Xbox in 2018-2019; Deciding to try a platinum trophy peaks on Playstation in 2017. Dotted lines represent 95% confidence intervals; non-overlapping intervals of two curves suggest significantly different prevalences. Some topics are Domain-specific. Game-specific topics

PlayerUnknown’s BattleGrounds and the “Tomb Raider” series are more preva

lent in /r/xboxachievements whereas Grim Fandango, Day of the Tentacle, “Spider-Man” series and Uncharted The Lost Legacy, Conan Exiles are more prevalent in /r/Trophies. Besides, topics Messaging and invites, Completion percentage,

Requesting and appreciating help, are found to be more mentioned in

/r/xboxachievements whereas Games becoming boring or stale, Grinding and monotony, Time allocation to playing, Deciding to try a platinum trophy and Congratulations are more mentioned in /r/Trophies. Attempting moves and routes has an attention shift, as the trends have opposite directions in the two sources. 5

Discussion

As shown in Table 1, the data volume is drastically growing indicating growth of the communities and impact of meta-game reward systems on gameplay and social media, so we answer our RQ1: the trend is clearly ongoing.

A sizable proportion of topics were game-specific indicating which games are most discussed, answering RQ2. Action-adventure games were the most discussed genre (topics with that genre tag had a total prevalence 0.069), followed by first-person shooters (0.022), role-playing games (0.022) and platformers (0.020). The rest had a prevalence of under 0.010. Discussions on multi-player games (e.g. PlayerUnknown’s Battlegrounds) can be connected to the statement by Jakobsson [ 11 ] that the reward system can itself be a massive multiplayer online game. Topics such as Requesting and appreciating help, Congratulations are related to the relatedness in SDT.

The (dis)similarity of focus in the two subreddits is learned by STM and can be evaluated from its parameters, answering RQ3. Topics that are more prevalent in /r/xboxachivements such as Messaging and invites, Completion percentage and Request, appreciate help are more related to pragmatic means of achieving scores.

While /r/Trophies has a wider variety of discussions, from perceived emotions (Games becoming boring or stale, Grinding and monotony and Congratulations) to decision making (Time allocation to playing, Deciding whether to try a platinum trophy and Difficulty of getting trophies, note that, although the last two topics contain PlayStation-specific words such as “platinum” and “trophy”, other terms (“decide”, “tricky”) can convey general semantic meanings so that they may also appear in /r/xboxachievements threads). This could be due to that PlayStation has different levels of rewards from Bronze to Platinum. This layered design enhances the complexity and potentially enrichs the online discussions. The r/xboxachievements subreddit is also smaller by comparison which means some of the discussion on Xbox achievements has spread out more to other venues, such as r/xbox, a more general discussion venue on Xbox related topics.

Conclusions, limitations, and opportunities

We presented a data-driven topic modeling study of player’s interests and attitudes to reward systems based on two subreddit corpora. Our qualitative analysis of text content was assisted by computational text mining. The results were both interesting on their own and supported human interpretation and efficient analysis of a large amount of text data. We found semantically meaningful topics on game series, trophy-hunting techniques, appreciation of achievements, dissatisfaction with difficulty, grinding, and time use, and their temporal fluctuation. Appreciation or frustration revealed by the topics can help promote a similar appreciation and avoid frustration in future gamification design. The topics provide an understanding of players’ attitudes and opinions regarding gamification (i.e., badge and achievement systems) of games. The topics show players commit game achievement-related behaviors, e.g., expressing frustration, asking for help, appreciating help, and etc. via online forums. Benefits of such behaviors towards continuous engagement in achievement pursuing and game playing can be useful for other gamification applications.

This work used only Reddit discussions; other large-volume venues include the Playstation Forum and Xbox achievement discussion forum. This can lead to a bias as most Reddit users are from the United States. More categorical analysis can be conducted in future by combining data sources, e.g., Twitter, Steam Community, etc., or comparing to other methods, e.g., Non-Negative Matrix Factorization (NMF) [ 23 ].

Deepened understanding of player behavior regarding game trophies and achievements can facilitate the design of personalized gamification, and application of such computational methods towards the understanding of player communities can enrich game studies. We expect the computational approach will inspire similar studies, especially in game studies. The discovered topics and temporal trends can further be used to establish research questions or directions in related fields.

Acknowledgments References

The work was supported by Academy of Finland decisions 312395, 313748 and 327352.

References

1. Ammari , T. , Schoenebeck , S. , Romero , D.M.: Pseudonymous parents: Comparing parenting roles and identities on the mommit and daddit subreddits . In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . pp. 1 - 13 ( 2018 )

2. Benmakrelouf , S. , Mezghani , N. , Kara , N.: Towards the identification of players' profiles using game's data analysis based on regression model and clustering . In: Proc. ASONAM 2015 . pp. 1403 - 1410 . ACM ( 2015 )

3. Blei , D.M. , Ng , A.Y. , Jordan , M.I. : Latent Dirichlet allocation . In: Advances in neural information processing systems . pp. 601 - 608 ( 2002 )

4. Cooper , H.E. , Camic , P.M. , Long , D.L. , Panter , A. , Rindskopf , D.E. , Sher , K.J.: APA handbook of research methods in psychology , Vol 1 : Foundations, planning, measures, and psychometrics . American Psychological Association ( 2012 )

5. Cruz , C. , Hanus , M.D. , Fox , J.: The need to achieve: Players' perceptions and uses of extrinsic meta-game reward systems for video game consoles . Computers in Human Behavior 71 , 516 - 524 ( 2017 )

6. Donath , J.: Signals in social supernets . Journal of Computer-Mediated Communication 13 ( 1 ), 231 - 251 ( 2007 )

7. Faisal , A. , Peltoniemi , M. : Establishing video game genres using data-driven modeling and product databases . Games and Culture 13 ( 1 ), 20 - 43 ( 2018 )

8. Goh , D.H.L. , Pe-Than , E.P.P. , Lee , C.S.: Perceptions of virtual reward systems in crowdsourcing games . Computers in Human Behavior 70 , 365 - 374 ( 2017 )

9. Hamari , J. , Eranti , V. :Frameworkfordesigningandevaluatinggameachievements. In: Digra conference. Citeseer ( 2011 )

10. Huotari , K. , Hamari , J.: Gamification” from the perspective of service marketing . In: Proc. CHI 2011 Workshop Gamification ( 2011 )

11. Jakobsson , M.: The achievement machine: Understanding xbox 360 achievements in gaming practices . Game Studies 11 ( 1 ), 1 - 22 ( 2011 )

12. Jakobsson , M. , Sotamaa , O. , Moore , C. , Begy , J. , Consalvo , M. , Gazzard , A. , Medler , B. , Williams , P. , Nesbitt , K.V. , Eidels , A. , et al.: game reward systems . Game Studies 11 ( 1 ) ( 2011 )

13. Kilgo , D.K. , Yoo , J.J. , Sinta , V. , Geise , S. , Suran , M. , Johnson , T.J.: Led it on reddit: An exploratory study examining opinion leadership on reddit . First Monday 21 ( 9 ) ( 2016 )

14. Kominiarczuk , M. :Gamifiedgameplay:Gamificationingamedesign.Gamification. Critical Approaches ( 2015 )

15. Lewis , I., de Salas, K. , Wells , L. : Features of achievement systems . In: Proc. CGAMES'2013 USA . pp. 66 - 73 . IEEE ( 2013 )

16. Lu , J. , Sridhar , S. , Pandey , R. , Hasan , M.A. , Mohler , G.: Investigate transitions into drug addiction through text mining of reddit data . In: Proc. KDD . pp. 2367 - 2375 . ACM ( 2019 )

17. Massanari , A. : Playful participatory culture: Learning from reddit . AoIR Selected Papers of Internet Research 3 ( 2013 )

18. Mimno , D. , Wallach , H.M. , Talley , E. , Leenders , M. , McCallum , A. : Optimizing semantic coherence in topic models . In: Proc. EMNLP . pp. 262 - 272 ( 2011 )

19. Montola , M. , Nummenmaa , T. , Lucero , A. , Boberg , M. , Korhonen , H.: Applying game achievement systems to enhance user experience in a photo sharing service . In: Proc. MindTrek . pp. 94 - 97 . ACM ( 2009 )

20. Nagle , A. , Wolf , P. , Riener , R. , Novak , D. : The use of player-centered positive reinforcement to schedule in-game rewards increases enjoyment and performance in a serious game . International Journal of Serious Games 1 ( 4 ), 35 - 47 ( 2014 )

21. O 'Neill , M. , Vaziripour , E. , Wu , J. , Zappala , D. : Condensing steam: Distilling the diversity of gamer behavior . In: Proc. 2016 internet measurement conference . pp. 81 - 95 . ACM ( 2016 )

22. Park , A. , Conway , M. , Chen , A.T. : Examining thematic similarity, difference, and membership in three online mental health communities from reddit: a text mining and visualization approach . Computers in human behavior 78 , 98 - 112 ( 2018 )

23. Pauca , V.P. , Shahnaz , F. , Berry , M.W. , Plemmons , R.J.: Text mining using non- negative matrix factorizations . In: Proceedings of the 2004 SIAM International Conference on Data Mining . pp. 452 - 456 . SIAM ( 2004 )

24. Rapp , A. : From games to gamification: A classification of rewards in world of warcraft for the design of gamified systems . Simulation & Gaming 48 ( 3 ), 381 - 401 ( 2017 )

25. Roberts , M.E. , Stewart , B.M. , Airoldi , E.M.: A model of text for experimentation in the social sciences . J Am Stat Assoc 111 ( 515 ), 988 - 1003 ( 2016 )

26. Ryan , R.M. , Deci , E.L. : Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being . American psychologist 55(1) , 68 ( 2000 )

27. Wang , H. , Sun , C.T.: Game reward systems: Gaming experiences and social meanings . In: DiGRA Conference . vol. 114 ( 2011 )

28. Wells , L. , Cauchi-Saunders , A. , Lewis , I. , Monsif , L. , Geelan , B. , de Salas , K. : Mining for gold (and platinum): Playstation network data mining . In: Proc. 2016 Annual Symposium on Computer-Human Interaction in Play . pp. 304 - 312 . ACM ( 2016 )

29. Weninger , T. : An exploration of submissions and discussions in social news: Mining collective intelligence of reddit . Soc Netw An Mining 4 ( 1 ), 173 ( 2014 )

30. Zulkifly , A. , Lewis , I., de Salas, K. : Do achievement systems undermine intrinsic player motivation? In: Proc. CGAMES'2013 USA . pp. 19 - 22 . IEEE ( 2013 )