Game video streams are watched by millions, so that, meanwhile, one can make a living from broadcasting and commenting video games, whereas some have become professional e-sports athletes. E-sports leagues and tournaments have emerged worldwide, where players compete in controlled environments, streaming the matches online, and allowing the audience to discuss and criticize the gameplay. In the GameStory task, held for the second time at MediaEval, we foster research into this exciting domain. Our focus is on analyzing and summarizing video game streams. With the help of ZNIPE.tv, we compiled a high-quality dataset of a Counter-Strike: Global Ofensive tournament alongside ground truth labels for two analysis tasks, forming a basis for summarization.
The e-sports industry has grown significantly in the past decade. Exemplified with one of the most successful games, League of Legends (LoL), in 2012, the number of concurrent online viewers in an LoL championship for a single event exceeded one million.1 From 2016 to 2019, the peak viewer counts rose from 28.26 (2016) to 106.27 (2017) to 205.11 million (2018) viewers.2 The video game streaming industry competes with traditional sports events for top viewership counts.3
E-sports has been compared to traditional sports a lot. Like in
traditional sports, leisure activities, like playing soccer for fun,
may lead to professional training and organized competitions for
athletes. In addition, Freeman and Wohn [
In-game streaming and e-sports, a lot of content is created. Besides videos, platforms like Twitch.tv or YouTube allow spectators to interact with the players, influencing their gameplay. Altogether, the data streams that can be collected for an individual player include video and audio, commentaries, game data and statistics, interaction traces, viewer-to-viewer communication. The level of detail of the data available, as well as its heterogeneity, render video game streams a challenging subject to multimedia research, allowing for a manifold of research questions.
New research fields like game analytics [
At GameStory 2019 we built on top of the 2018 dataset by adding a particular subproblem of e-sports match summarization. Since the data are comprised of video streams from multiple perspectives on a given match, as well as a commentators’ stream, we ask participants to identify critical moments by finding replays in the commentators’ stream and by aligning them with the source video clip in the players’ streams. Then, as a second task, following our 2018 summarization task, participants are asked to create a multi-modal summary that provides a captivating story of the match’s progress. The latter task is open-ended in the sense that there is no ground truth, but participants can be creative and identify critical moments themselves, whereas the former task may guide participants in solving the latter. 2
CS:GO is a first-person shooter (FPS) game and, as an e-sports game, it has very strict rules of- play. Two teams, the terrorists and the counter-terrorists, with five players each, compete in a virtual 3D world, called map. Matches consist of several rounds and players only re-spawn in between rounds. Depending on the success of players and teams, players get awarded virtual money. With that money, players can outfit their avatars with weapons, ammo, tools, and armor in-between rounds.6 5https://csgo-stats.com/ 6http://www.tobyscs.com/csgo-economy-guide/ M. Lux, M. Riegler, D-T. Dang-Nguyen, J. Pirker, M. Pothast, P. Halvorsen Our dataset was recorded at the CS:GO Intel Extreme Masters (IEM) ESL tournament in Katowice 2018. According to the ESL rules, a typical CS:GO match is decided in a best-of-30 fashion. After 15 rounds, the teams switch sides, i.e., terrorists become counterterrorists and vice versa. If two teams end up with a draw after 30 rounds, the teams play overtime to determine the winner. Strategies typically span over multiple rounds, including diferent sets of constraints, e.g., teams can aford to buy what they need, teams have to save money, etc. In general, a winning condition is to eliminate all opponents. In a DE_Map round, as we have them in our dataset, the goal for the terrorist team is to plant a bomb at one of some specific locations and protect it from being defused (until the end of the round), while the counter terrorist team has to defuse the bomb. In these maps, a bomb going of or the prevention of that event is an additional winning condition.
The data consists of twelve video streams along with metadata. Ten files give the view of the players with the in-game audio streams. One file gives the commentator stream, where a professional cutter selected the parts of the player views to be shown as well as videos from the audience, the teams’ players, and the commentator provides the spoken content mixed with recordings from the game and audience cheering. The last one shows the map from above with icons indicating the position of players. A metadata file indicates the start and end of games and the content in the commentators’ stream. JSON files, one for each match in the dataset, capture player activity and events in addition to the raw video. Events range from kills, deaths, starts, and ends of rounds to what the players bought at the beginning of rounds and when a bomb was set or a grenade was thrown and went of. The data covers three days of the tournament and is split into training (two days) and test (one day) sets. A ground truth for where to find replays in the commentators’ stream is given for the first day of the training dataset. Additionally, for all the video streams, we provide synchronization data as the actual video is of up to 40 seconds from the time stamps given in the metadata. 3
Compared to sports summaries [
With that in mind, we defined two tasks for GameStory 2019: (1) Find all replays in the commentators’ streams and locate the source clips in the respective player streams. (2) Create a short and captivating summary with a maximum length of five minutes of a single match.
The evaluation of the first task is based on the overlap of the found clips with the ground truth. To determine if a given replay has been successfully found, we employ the Jaccard index in terms of video frames:
J (A, B) = A ∩ B ,
A ∪ B where A denotes the set of consecutive frames identified as a replay clip, and B the set of consecutive frames of the actual replay from the ground truth.
We consider a replay to be successfully found if J (A, B) > t for threshold t , using two thresholds t1 > 0.5 and t2 > 0.75. Using the Jaccard index, we calculate precision, recall, and the F1 score for the set of replays to be identified. Should a given replay be identified more than once, only the clip with the highest Jaccard index is counted. In a second step, we determine the goodness of the match between true positive replay clips and original player streams. Again for we use the Jaccard index to determine the overlap of the source segment from the ground truth with the segment found in the run. To quantify the degree of overlap for all found replays we average for all found replay segments.
The evaluation of the second task is based on a jury of experts, including CS:GO players and game researchers. The jury members watch the summaries individually and independent of each other with the tasks of summarizing and arguing both strong and weak points of each submission, as well as rating them on a 5-point Likert scale (strongly agree to strongly disagree) concerning the following statements: (1) The submission gives a summary of the match at hand. (2) The submission is entertaining. (3) The submission provides the flow and peak of a good story. (4) The submission provided an innovative way to present a summary of a CS:GO match. (5) A summary like this submission can be applied to games other than CS:GO. 4
GameStory is at the forefront of research on data-driven analysis of video games and game streaming, a domain that has hardly been addressed within multimedia computer science to date. Here, huge amounts of content and data are generated by millions of players and viewers daily, by both amateurs and professional creators and producers. Their highly interactive nature makes the outcome of games mostly unpredictable. Research in this area is still in its infancy but has the potential for a high social impact. Players and viewers are often young, and games have become an important part of youth culture, sometimes having a strong and long-lasting influence on people’s lives. With GameStory, we seek to explore this exciting new direction of research. In the coming years, our goal is to diversify and grow the tasks with a combination of objective, quantitative analysis tasks (e.g., finding kill streaks, synchronizing streams, or identifying relations between consecutive wins and economy in the game streams) and freestyle synthesis tasks, where the analysis technology can be readily employed.
We thank Michael Wutti for providing the scores and the sync points, Natascha Rauscher, Shivi Vats, and Simon Bernard for creating the ground truth, and Sabrina Kletz for the helpful discussion of the evaluation metrics.