Team ORG @ GameStory Task 2018 Mathias Lux1 , Michael Riegler2,3 , Duc-Tien Dang-Nguyen4 , Marcus Larson5 , Martin Potthast6 , and Pål Halvorsen2,3 1 Alpen-Adria-Universität Klagenfurt, Austria; 2 SimulaMet, Norway; 3 University of Oslo, Norway; 4 University of Bergen, Norway; 5 ZNIPE.TV; 6 Universität Leipzig, Germany mlux@itec.aau.at,michael@simula.no,ductien.dangnguyen@uib.no,marcus@znipe.se, martin.potthast@uni-leipzig.de,paalh@simula.no ABSTRACT This paper describes the approach of the organizers’ team for a submission to the GameStory task at MediaEval 2018. Goal of the task is to provide a summary of a match of Counter Strike: Global Offensive (CS:GO), a popular e-sports game, that boils down a long game to it’s most important events and delivers a story on the progress of the match. Our approach was to provide match statistics and overlay them with events and highlights of the game. We Figure 1: Timeline used for visualization with the first 8 focused on ends of critical rounds, i.e. the rounds where one team rounds. The animation was build using a moving camera of took the lead over the other one, and kill streaks, where one player the rounds, each one resembled by one card. eliminated a substantial number of other players in short time. each round, we extracted from the metadata file the current stand- 1 INTRODUCTION ings, the money spent by each team and eventual kill streaks. For Broadcasting games over the internet has attracted more and more the video, we moved a virtual camera with a resolution of 1920x1080 users over the years [1, 2]. However, beside the technical challenges pixels over the stats, and four of the “stats cards” made up an entire of low-latency, high quality streaming [4], one also has to deal with screen. The virtual camera moves with 4 pixels per frames and 30 the large amount of available data. The GameStory task at Media- frames per second (fps), moving one card from one position to the Eval 2018 [3] is about summarizing matches or even tournaments next takes 4 seconds. Each card is visible for 16 seconds, which we in e-sports with particular focus on CS:GO. It is a very new task considered enough for being able to read the information. For the with a qualitative evaluation, meaning that players and experts take video, the colors of the stats card have been inverted, as white text a look at the videos submitted and judge on how well the videos on black background seemed to be more pleasant for the viewers can summarize a given CS:GO match. who we showed the video to. Within a CS:GO match multiple influences and events – or the For the money spent in each round, we extracted the items sequence thereof – can decide upon the outcome of a game. First, of bought by the players each round and computed the total amount course, the skill of the players has a huge impact. Especially visible of money spent per team. For that, we had to create a look-up are those players that take over an offensive role and eliminate table (a Python dictionary) based on data from the internet1 , which several players of the other team in short time. An event like that is allowed us to relate the item name to the price, e.g., “ak47”:2700, called a kill streak. Moreover, the decision on how to spend money “deagle”:300, “flashbang”:200, . . . at the start of a round is critical to the success over multiple rounds. For the kill streaks, we first extracted the kills per round and Our idea was to mainly analyze the development of the game grouped them by the players. We only considered kill streaks where over time and present events in addition to animated game statistics. a single player scored three or more eliminations to be displayed Focusing on the metadata, we created an animated timeline of the on the statistics in the animated timeline. We further computed the game and used videos from the streams at hand to overlay the time from the first to the last elimination to be able to find the kill events we deemed important. streaks, which happen in short time, for later use in the overlay videos. 2 APPROACH With the moving camera animation over the stats cards we then Our approach focused on analyzing and making sense of the meta- focused on extracting videos from the original streams provided data, where all events of the game were recorded, including what for the GameStory task for overlaying actual game action. The the players bought, who was killed by whom, when rounds started streams have a rather low resolution with 640x380 pixels, but could and ended, and if and when bombs were planted and defused. In be used in the left upper and left lower corners of the summary. a first step, we created an overview on the development of the Based on the kill streak data we extracted videos from the players game following the timeline of the rounds. Figure 1 shows our perspective if the kill streaks were not longer than the number of approach to visualizing the game stats and their development. For kills k × 6 in seconds, e.g., 18 seconds for 3 kills, or 24 seconds for 4 Copyright held by the owner/author(s). kills. The maximum of course is a kill streak of 5, as there are only MediaEval’18, 29-31 October 2018, Sophia Antipolis, France 1 http://counterstrike.wikia.com/, last visited 2018-10-16 MediaEval’18, 29-31 October 2018, Sophia Antipolis, France M. Lux et.al. 5.00 4.50 The submission gives a summary of the match at 4.00 hand. 3.50 The submission is entertaining. 3.00 2.50 The submission provides the flow and peak of a good story. 2.00 1.50 The submission provided an innovative way to present a 1.00 summary of an CS:GO match. 0.50 A summary like this submission can be applied to 0.00 games different from CS:GO. Figure 2: Frame of the final video showing the stats in in- verted colors and the overlay of a players view from a kill Figure 3: Review scores per run, averaged. 1 is strongly agree, streak in the left bottom corner. 5 is strongly disagree. Our run is titled Run03. 3 EVALUATION The jury noted that relative to the other submissions our video 5 opponents in the game and the players killed do not respawn in gave a good summary of the match and presented an innovative the same round. A frame of the final video with the overlay of a way to summarize the match. However, the reviewers noted that kill streak video can be seen in Figure 2. the overtime is not explicitly outlined and no in-game videos of In addition to the kill streak videos, we extracted the 10 seconds that particular period are given, although this is a critical part of a round for each round from the commentators view, as well of the game. The reviewers liked the mix of players’ view and as the map overview focusing on 5 seconds before to 5 seconds commentator’s view catching the excitement of the crowd, but all after the round end event in the metadata. For the final round, we in all, the video was too crowded with the animated stats. The clips took out 30 seconds, i.e., round end minus 10 to round end plus 20 seemed somewhat out of place and were not connected – visually seconds. or otherwise – to the rounds. Sound only came from the overlay While up to that point, most of the work done could be done videos, and the periods of silence in between were recognized as automatically, we then decided to tweak the results with manual missing audio by the reviewers. input. First of all, we switched from player and team id to the real names of the teams and the players and created the respective dictionaries. We then connected the player names to the streams 4 DISCUSSION & OUTLOOK with a dictionary. With the start times of the matches given for The approach we have chosen was mainly based on the metadata, each stream in terms of offset from the beginning, we could related and there only on the economy actions, the round end and the kill the UTC time-stamps from the metadata file to the respective time streaks. A lot more could have been done including on how the point in the stream. However, due to technical constraints, these round ended, i.e., by killing the entire enemy team, or by planting time points were not matching the actual events too well, so we or defusing the bomb, or by analyzing the strategy a team applied, synchronized them manually at the begin of the first round. The i.e. if they saved money by not outfitting their avatars, or if they offset for streams from the data given in the metadata ranged from tried to rush and win the round fast. With extraction of player 32 to 45 seconds. positions, we could have switched between viewpoints and would Moreover, we cut the final video manually. The animation was have multiple views of the same event. created automatically, but the overlays were done with OpenShot, While we extracted the videos from the map overview and the an open source non linear video editor. While the kill streak videos player position stream, the videos did not make it into the final could have been inserted automatically at the beginning of the version. The main problem was the synchronization as there was no respective round they took place, the selection of round end videos solid way to find the right time point in the map stream. A content was up to human decision. We focused on rounds were the tides based synchronization of the videos might have helped with that. were turning, i.e., when one team took over the other one, on rounds Furthermore, the overlay videos were cut solely based on time were teams that spent less money could still score a point, and on stamps. This gave the ill received effect of cut off audio for the the final round. final video. In the future, the audio characteristics should influence All kill streaks video sequences were presented in the bottom cutting decisions, i.e., by detecting speech or analyzing the audio left corner (see Figure 2), while all beside the last videos concerning envelope to not cut off videos in mid sentence. Moreover, in future the end of game rounds were presented in the top left corner. Only work, we aim to base the decision on for which rounds to show the last video concerning the end of the game was presented in full the commentator stream at the end on rules in contrast to manual frame. identification. Team ORG @ GameStory MediaEval’18, 29-31 October 2018, Sophia Antipolis, France REFERENCES [3] Mathias Lux, Michael Riegler, Duc-Tien Dang-Nguyen, Marcus Lar- [1] William A Hamilton, Oliver Garretson, and Andruid Kerne. 2014. son, Martin Potthast, and Pål Halvorsen. 2018. GameStory Task at Streaming on twitch: fostering participatory communities of play MediaEval 2018. In Working Notes Proceedings of the MediaEval 2018 within live mixed media. In Proceedings of the SIGCHI Conference on Workshop. Human Factors in Computing Systems. ACM, 1315–1324. [4] Karine Pires and Gwendal Simon. 2014. Dash in twitch: Adaptive [2] Mehdi Kaytoue, Arlei Silva, Loïc Cerf, Wagner Meira Jr, and Chedy bitrate streaming in live game streaming platforms. In Proceedings Raïssi. 2012. Watch me playing, i am a professional: a first study of the 2014 Workshop on Design, Quality and Deployment of Adaptive on video game live streaming. In Proceedings of the 21st International Video Streaming. ACM, 13–18. Conference on World Wide Web. ACM, 1181–1188.