=Paper=
{{Paper
|id=Vol-2670/MediaEval_19_paper_1
|storemode=property
|title=GameStory
Task at MediaEval 2019
|pdfUrl=https://ceur-ws.org/Vol-2670/MediaEval_19_paper_1.pdf
|volume=Vol-2670
|authors=Mathias Lux,Michael Riegler,Duc-Tien Dang-Nguyen,Johanna Pirker,Martin Potthast,Pål Halvorsen
|dblpUrl=https://dblp.org/rec/conf/mediaeval/LuxRDPPH19
}}
==GameStory
Task at MediaEval 2019==
https://ceur-ws.org/Vol-2670/MediaEval_19_paper_1.pdf
GameStory Task at MediaEval 2019
Mathias Lux,1 Michael Riegler,2 Duc-Tien Dang-Nguyen,4 Johanna Pirker,5
Martin Potthast,6 and Pål Halvorsen2,3
1 Alpen-Adria-Universität Klagenfurt, Austria; 2 SimulaMet, Norway; 3 Oslo Metropolitan University, Norway;
4 University of Bergen, Norway; 5 Graz University of Technology, Austria; 6 Leipzig University, Germany
mlux@itec.aau.at, michael@simula.no, ductien.dangnguyen@uib.no, johanna.pirker@tugraz.at,
martin.potthast@uni-leipzig.de, paalh@simula.no
ABSTRACT In-game streaming and e-sports, a lot of content is created. Be-
Game video streams are watched by millions, so that, meanwhile, sides videos, platforms like Twitch.tv or YouTube allow spectators
one can make a living from broadcasting and commenting video to interact with the players, influencing their gameplay. Altogether,
games, whereas some have become professional e-sports athletes. the data streams that can be collected for an individual player in-
E-sports leagues and tournaments have emerged worldwide, where clude video and audio, commentaries, game data and statistics,
players compete in controlled environments, streaming the matches interaction traces, viewer-to-viewer communication. The level of
online, and allowing the audience to discuss and criticize the game- detail of the data available, as well as its heterogeneity, render
play. In the GameStory task, held for the second time at MediaEval, video game streams a challenging subject to multimedia research,
we foster research into this exciting domain. Our focus is on an- allowing for a manifold of research questions.
alyzing and summarizing video game streams. With the help of New research fields like game analytics [2, 3] have emerged,
ZNIPE.tv, we compiled a high-quality dataset of a Counter-Strike: investigating the highly interactive and narrative nature of video
Global Offensive tournament alongside ground truth labels for two games [1]. Relating to summarization, while a lot of work has been
analysis tasks, forming a basis for summarization. done on videos and multi-modal summarization [7, 10, 12, 16], video
games have hardly been investigated so far. At MediaEval 2018,
GameStory was organized for the first time [8, 9]. Here, participants
1 INTRODUCTION were given the multi-player plus commentary streams for an e-
The e-sports industry has grown significantly in the past decade. sports match of Counter-Strike: Global Offensive (CS:GO),5 a game
Exemplified with one of the most successful games, League of with a solid population of players and viewers, and the task was to
Legends (LoL), in 2012, the number of concurrent online viewers generate an entertaining summary.
in an LoL championship for a single event exceeded one million.1 At GameStory 2019 we built on top of the 2018 dataset by adding
From 2016 to 2019, the peak viewer counts rose from 28.26 (2016) a particular subproblem of e-sports match summarization. Since the
to 106.27 (2017) to 205.11 million (2018) viewers.2 The video game data are comprised of video streams from multiple perspectives on a
streaming industry competes with traditional sports events for top given match, as well as a commentators’ stream, we ask participants
viewership counts.3 to identify critical moments by finding replays in the commenta-
E-sports has been compared to traditional sports a lot. Like in tors’ stream and by aligning them with the source video clip in
traditional sports, leisure activities, like playing soccer for fun, the players’ streams. Then, as a second task, following our 2018
may lead to professional training and organized competitions for summarization task, participants are asked to create a multi-modal
athletes. In addition, Freeman and Wohn [5] posit that e-sports is summary that provides a captivating story of the match’s progress.
defined by the spectatorship and the governing bodies like the ESL The latter task is open-ended in the sense that there is no ground
Gaming Network.4 Hamilton et al. [6] found that streaming games truth, but participants can be creative and identify critical moments
focuses on social engagement and community building, which con- themselves, whereas the former task may guide participants in
trasts traditional sports, where the focus lies on the highest levels solving the latter.
of play. E-sports appears to be in-between game streaming and
traditional sports. Though it depends on a participatory commu- 2 BACKGROUND AND DATASET
nity, especially streams with a large amount of viewers struggle
to maintain meaningful social engagement. Seo and Jung [11] see CS:GO is a first-person shooter (FPS) game and, as an e-sports
e-sports at the heart of consumer communities with consumers also game, it has very strict rules of- play. Two teams, the terrorists and
being players interacting with the game beyond the game interface. the counter-terrorists, with five players each, compete in a virtual
3D world, called map. Matches consist of several rounds and players
1 https://web.archive.org/web/20130608053015/http://www.riotgames.com/articles/
only re-spawn in between rounds. Depending on the success of
20130509/549/league-legends-season-two-championship; all URLs in this paper have
been last accessed on July 26, 2019, and been archived at the Internet Archive. players and teams, players get awarded virtual money. With that
2 https://escharts.com/tournaments/lol money, players can outfit their avatars with weapons, ammo, tools,
3 https://onlinebusiness.syr.edu/blog/esports-to-compete-with-traditional-sports/
4 https://www.eslgaming.com/
and armor in-between rounds.6
Copyright 2019 for this paper by its authors. Use
5 https://csgo-stats.com/
permitted under Creative Commons License Attribution
6 http://www.tobyscs.com/csgo-economy-guide/
4.0 International (CC BY 4.0).
MediaEval’19, 27-29 October 2019, Sophia Antipolis, France
MediaEval’19, 27-29 October 2019, Sophia Antipolis, France M. Lux, M. Riegler, D-T. Dang-Nguyen, J. Pirker, M. Potthast, P. Halvorsen
Our dataset was recorded at the CS:GO Intel Extreme Masters of video frames:
(IEM) ESL tournament in Katowice 2018. According to the ESL rules, A∩B
J (A, B) = ,
a typical CS:GO match is decided in a best-of-30 fashion. After A∪B
15 rounds, the teams switch sides, i.e., terrorists become counter- where A denotes the set of consecutive frames identified as a replay
terrorists and vice versa. If two teams end up with a draw after clip, and B the set of consecutive frames of the actual replay from
30 rounds, the teams play overtime to determine the winner. Strate- the ground truth.
gies typically span over multiple rounds, including different sets We consider a replay to be successfully found if J (A, B) > t for
of constraints, e.g., teams can afford to buy what they need, teams threshold t, using two thresholds t 1 > 0.5 and t 2 > 0.75. Using
have to save money, etc. In general, a winning condition is to elim- the Jaccard index, we calculate precision, recall, and the F1 score
inate all opponents. In a DE_Map round, as we have them in our for the set of replays to be identified. Should a given replay be
dataset, the goal for the terrorist team is to plant a bomb at one of identified more than once, only the clip with the highest Jaccard
some specific locations and protect it from being defused (until the index is counted. In a second step, we determine the goodness of the
end of the round), while the counter terrorist team has to defuse match between true positive replay clips and original player streams.
the bomb. In these maps, a bomb going off or the prevention of that Again for we use the Jaccard index to determine the overlap of the
event is an additional winning condition. source segment from the ground truth with the segment found in
The data consists of twelve video streams along with metadata. the run. To quantify the degree of overlap for all found replays we
Ten files give the view of the players with the in-game audio streams. average for all found replay segments.
One file gives the commentator stream, where a professional cutter The evaluation of the second task is based on a jury of experts,
selected the parts of the player views to be shown as well as videos including CS:GO players and game researchers. The jury members
from the audience, the teams’ players, and the commentator pro- watch the summaries individually and independent of each other
vides the spoken content mixed with recordings from the game and with the tasks of summarizing and arguing both strong and weak
audience cheering. The last one shows the map from above with points of each submission, as well as rating them on a 5-point Likert
icons indicating the position of players. A metadata file indicates scale (strongly agree to strongly disagree) concerning the following
the start and end of games and the content in the commentators’ statements:
stream. JSON files, one for each match in the dataset, capture player (1) The submission gives a summary of the match at hand.
activity and events in addition to the raw video. Events range from (2) The submission is entertaining.
kills, deaths, starts, and ends of rounds to what the players bought (3) The submission provides the flow and peak of a good story.
at the beginning of rounds and when a bomb was set or a grenade (4) The submission provided an innovative way to present a
was thrown and went off. The data covers three days of the tourna- summary of a CS:GO match.
ment and is split into training (two days) and test (one day) sets. A
(5) A summary like this submission can be applied to games
ground truth for where to find replays in the commentators’ stream
other than CS:GO.
is given for the first day of the training dataset. Additionally, for all
the video streams, we provide synchronization data as the actual
video is off up to 40 seconds from the time stamps given in the
4 DISCUSSION AND OUTLOOK
metadata. GameStory is at the forefront of research on data-driven analysis
of video games and game streaming, a domain that has hardly been
addressed within multimedia computer science to date. Here, huge
3 TASKS AND EVALUATION amounts of content and data are generated by millions of players
Compared to sports summaries [4, 13–15], video games are not and viewers daily, by both amateurs and professional creators and
focused on a small number of attention points like, for instance, the producers. Their highly interactive nature makes the outcome of
ball and the two goals on a soccer field. Rather, video games com- games mostly unpredictable. Research in this area is still in its
prise multiple views and concurrent events within a well-defined infancy but has the potential for a high social impact. Players and
game world (map), which can change in-between games, but stays viewers are often young, and games have become an important
the same in one single game. Typically, game statistics only con- part of youth culture, sometimes having a strong and long-lasting
vey active and obvious events, but miss those with semantics on a influence on people’s lives. With GameStory, we seek to explore this
tactical level, including fake tactics, intentional misses, intentional exciting new direction of research. In the coming years, our goal
risk-taking (e.g., a player avatar’s death or re-spawn), an the like. is to diversify and grow the tasks with a combination of objective,
With that in mind, we defined two tasks for GameStory 2019: quantitative analysis tasks (e.g., finding kill streaks, synchronizing
streams, or identifying relations between consecutive wins and
(1) Find all replays in the commentators’ streams and locate economy in the game streams) and freestyle synthesis tasks, where
the source clips in the respective player streams. the analysis technology can be readily employed.
(2) Create a short and captivating summary with a maximum
length of five minutes of a single match. ACKNOWLEDGMENTS
We thank Michael Wutti for providing the scores and the sync
The evaluation of the first task is based on the overlap of the points, Natascha Rauscher, Shivi Vats, and Simon Bernard for cre-
found clips with the ground truth. To determine if a given replay ating the ground truth, and Sabrina Kletz for the helpful discussion
has been successfully found, we employ the Jaccard index in terms of the evaluation metrics.
GameStory 2019 MediaEval’19, 27-29 October 2019, Sophia Antipolis, France
REFERENCES
[1] Jason Bryce and Jason Rutter. 2002. Spectacle of the deathmatch: Character
and narrative in first-person shooters. ScreenPlay: Cinema/videogames/interfaces
(2002), 66–80.
[2] Anders Drachen, Magy Seif El-Nasr, and Alessandro Canossa. 2013. Game
analytics–the basics. In Game analytics. Springer, 13–40.
[3] Magy Seif El-Nasr, Anders Drachen, and Alessandro Canossa. 2013. Game
Analytics: Maximizing the Value of Player Data. Springer Publishing Company,
Incorporated.
[4] Farhad Bayat Farshad Bayat, Mohammad Shahram Moin. 2014. Goal Detection
in Soccer Video: Role-Based Events Detection Approach. International Journal
of Electrical and Computer Engineering (IJECE) 4, 6 (Dec 2014), 979–988.
[5] Guo Freeman and Donghee Yvette Wohn. 2017. eSports as an emerging research
context at CHI: Diverse perspectives on definitions. In Proceedings of the 2017 CHI
Conference Extended Abstracts on Human Factors in Computing Systems. ACM,
1601–1608.
[6] William A Hamilton, Oliver Garretson, and Andruid Kerne. 2014. Streaming on
twitch: fostering participatory communities of play within live mixed media. In
Proceedings of the 32nd annual ACM conference on Human factors in computing
systems. ACM, 1315–1324.
[7] Zhuo Lei, Ke Sun, Qian Zhang, and Guoping Qiu. 2016. User Video Summarization
Based on Joint Visual and Semantic Affinity Graph. In Proceedings of the 2016
ACM workshop on Vision and Language Integration Meets Multimedia Fusion
(iV&L-MM). 45–52. https://doi.org/10.1145/2983563.2983568
[8] Mathias Lux, Pål Halvorsen, Duc-Tien Dang-Nguyen, Håkon Stensland, Manoj
Kesavulu, Martin Potthast, and Michael Riegler. 2019. Summarizing E-sports
matches and tournaments: the example of counter-strike: global offensive. In Pro-
ceedings of the 11th ACM Workshop on Immersive Mixed and Virtual Environment
Systems. ACM, 13–18. https://doi.org/10.1145/3304113.3326116
[9] Mathias Lux, Michael Riegler, Duc-Tien Dang-Nguyen, Marcus Larson, Martin
Potthast, and Pål Halvorsen. 2018. GameStory Task at MediaEval 2018. In Working
Notes Proceedings of the MediaEval 2018 Workshop. http://ceur-ws.org/Vol-2283/
MediaEval_18_paper_8.pdf
[10] Jia-Yu Pan, Hyungjeong Yang, and C. Faloutsos. 2004. MMSS: multi-modal story-
oriented video summarization. In Proc. of IEEE International Conference on Data
Mining (ICDM). 491–494. https://doi.org/10.1109/ICDM.2004.10033
[11] Yuri Seo and Sang-Uk Jung. 2016. Beyond solitary play in computer games: The
social practices of eSports. Journal of Consumer Culture 16, 3 (2016), 635–655.
[12] Rajiv Ratn Shah, Anwar Dilawar Shaikh, Yi Yu, Wenjing Geng, Roger Zim-
mermann, and Gangshan Wu. 2015. Eventbuilder: Real-time multimedia event
summarization by visualizing social media. In Proceedings of the 23rd ACM inter-
national conference on Multimedia (ACM MM). ACM, 185–188.
[13] H. C. Shih. 2017. A Survey on Content-aware Video Analysis for Sports. IEEE
Transactions on Circuits and Systems for Video Technology PP, 99 (2017), 1–1.
https://doi.org/10.1109/TCSVT.2017.2655624
[14] Yih-Ming Su and Shu-Jiun Liang. 2008. A novel highlight event decision approach
for baseball videos. In Proceedings of IEEE International Symposium on Consumer
Electronics. 1–2. https://doi.org/10.1109/ISCE.2008.4559549
[15] Min Xu, Ling-Yu Duan, Changsheng Xu, M. Kankanhalli, and Qi Tian. 2003.
Event detection in basketball video using multiple modalities. In Proceedings of
the joint International Conference on Information, Communications and Signal
Processing and Pacific Rim Conference on Multimedia, Vol. 3. 1526–1530 vol.3.
https://doi.org/10.1109/ICICS.2003.1292722
[16] Y. Yuan, T. Mei, P. Cui, and W. Zhu. 2017. Video Summarization by Learning
Deep Side Semantic Embedding. IEEE Transactions on Circuits and Systems for
Video Technology PP, 99 (2017), 1–1. https://doi.org/10.1109/TCSVT.2017.2771247