Automated Killstreak Extraction in CS:GO Tournaments Michael Wutti Alpen-Adria-Universität Klagenfurt, Austria mwutti@edu.aau.at ABSTRACT This paper describes a system for automated killstreak extraction in CS:GO tournaments. It uses a metadata and content-based approach for identification of the killstreaks. Inaccurate timestamps in the provided metadata of the tournament resulted in almost unusable detection results. Therefore, a content-based technique has been developed to extract the correct highlight videos. The results are merged into a summary video with a multi-view perspective of Figure 1: Displacement of killstreak start: t... killstreak be- the event stream, the actor’s view and the views of the victims. gin extracted from the game log, δ ... maximum displacement Due to the content-based approach, the multi-view video could be from the actual killstreak begin, t ′ ... possible range of cor- synchronised nearly frame accurate. rect killstreak begin 1 INTRODUCTION by processing the game log. This list is used to identify all players E-sports events such as the CS:GO World Championship 2018 in who were involved in a killstreak (actor and victims) and to store Katowice are becoming increasingly popular. One of the key chal- the start time and the duration of the killstreak. lenges of such an event is to summarise it in an exciting and capti- vating manner. E-sports events are rich in multimedia data such as video and audio streams, player details, commentator descriptions 3 EXTRACTING KILLSTREAKS and comments from viewers. One of the tasks in the MediaEval With the generated model all killstreaks with a maximum duration 2018 Workshop [2] is to find a way how E-sports matches ramp of 20 seconds and a minimum killstreak length of three players up, evolve and play out over time. This paper shows an approach were identified as candidates for the summary. As seen in Figure to cover a subset of all the different aspects and possibilities for 1 the main difficulty in extracting the killstreaks is to identify the summarising E-sports events. The developed software is written start time of the killstreak. The begin of a killstreak t extracted from in Python and uses a couple of image processing and machine the game log may differ up to δ = 40 seconds from the timestamps learning frameworks such as OpenCV [1] and scikit-learn [3] to in the stream metadata. This means that the correct start point t ′ automatically extract killstreaks from the provided videos. The final could be in the range of: goal in the GameStory Task of MediaEval 2018 is to summarise the t − δ ≤ t′ ≤ t + δ last match of the tournament (approx. one hour) in less or equal To find the correct timestamp, an additional video processing step than five minutes. Every team is allowed to submit three different has been implemented. First, the beginning of the round in which versions of the highlight video. In the next sections the process the killstreak was occurred has been identified. Second, based on of summarising a CS:GO match by extracting the most valuable the round begin and the information in the model the beginning of killstreaks is presented. the killstreak has been found. 2 META DATA PROCESSING 3.1 Detecting Round Start Two different types of metadata are provided by the task organis- To determine the correct start of a killstreak, the model retrieves ers. The stream metadata consists of information about the video the number of kills that were done before the killstreak has started. streams including timestamps and duration of the matches. The Based on this number k the timestamp of the first kill from the game log contains a detailed listing of all events such as purchases, killstreak can be determined. But due to the potential displacements kills and the winning team for each round. Unfortunately, the times- of the timestamps, there can be two consecutive rounds in the tamps in the stream metadata may differ up to 40 seconds from the search window t − δ ≤ t ′ ≤ t + δ in which an equal number of kills timestamps in the game log which is a typical problem in multime- were occurred. Therefore, it is necessary to first identify the round dia systems. In order to use the information in the game log, it is begin of a killstreak. As an indication for the correct round start, therefore necessary to find out the exact time of an event. Based the difference between the timestamps of the match begin and the on the game log a model has built which contains additional infor- (target kill + δ ) is added to the given video position of the target mation about the game score for each round and keeps track of the match. Starting from that point the match begin is being searched. overall game state. Additionally, a killstreak list has been generated As seen in Figure 2 the match score is displayed in the upper part of the video. Two Regions of Interest (ROI) are extracted where the Copyright held by the owner/author(s). MediaEval’18, 29-31 October 2018, Sophia Antipolis, France score numbers are displayed. A Support Vector Machine (SVM) is used to classify the ROIs. The SVM was trained with 400 samples of MediaEval’18, 29-31 October 2018, Sophia Antipolis, France M. Wutti Figure 2: Gameplay screenshot with match score and skull Figure 3: Screenshot of the multi-view killstreak summary. indication of achieved kills. 4 RESULTS AND DISCUSSION Three different summaries were created from the extracted videos. Every video consists of nine killstreaks, ordered by their corre- each number from 0-21 which where extracted beforehand from the sponding round numbers. Video one and two show the synchro- training set. Every frame in an extended search window with δ = 1 nised event view and killstreak actor view. As seen in Figure 3 the minute is processed by the SVM and after an amount of 60 matches last video is a multi-view summary and is made from the event view, with the target score, the timestamp of the round start tr is detected. actor view, and the views of all victims. The videos where watches The decision for choosing a SVM for the number detection problem by two to four reviewers from MediaEval and evaluated based on is based on the fast computation speed and satisfying precision that five criteria: match summary, entertainment value, flow and peak was achieved during development. of a good story, innovation and portability for other games. Some of the shortcomings of the reviewers should now be briefly addressed. The summary covers only the killstreaks greater than three 3.2 Detecting Killstreak Start which were achieved during the match. This seems like a quick and A pattern matching approach is used for the detection of the kill- easy approach. But if the timestamps in the metadata files can differ streak begin. Given the information from the model, that n kills up to 40 seconds from the actual time, the provided information is were achieved before the killstreak has begun, the timestamp tk useless. Because the compensation of the timestamp difference was where (n + 1) kills were achieved is being searched. As seen in an unexpected time-consuming task, many ideas like the integration Figure 2 a kill is indicated with a skull at the left or right side of of overtime events and audio-based analysis could not be done. the screen. For that reason ten ROIs where a skull could possibly Another aspect which is not covered by the summary is the appear have been extracted and compared with a reference picture impact of money and the player’s purchases at round begin. An of the skull. The comparison was done with the L1-norm of the integration and analysis of this data could have produced additional normalised reference and ROI values. If the norm has fallen below highlight videos. For example, one could investigate if a team is a threshold value the kill counter is increased. At the moment the playing with a weak load-out against a team with strong equipment kill counter reaches (n + 1), tk is reached and can be used as the and still wins the round. In addition, sneak and melee kills could start point for extracting the video of the killstreak. be easily integrated by analysing the metadata. Nevertheless the metadata timestamps of such events must be synchronised with the actual event times. Other aspects like pitfalls and mistakes made 3.3 Extracting Videos by other players are definitely interesting and would make the The technique described above has first been applied to the event summary more exciting. stream. Later it has been used for extracting the killstreaks from the The algorithm presented in this paper is able to automatically player streams. To achieve a better control of the target video length detect and extract sequences of killstreaks from the whole tour- and a better overall viewing experience, a clipping-offset between nament data. Nevertheless, killstreaks are just a subset of events 5 and 20 seconds is applied before tk and after the killstreak end. which could be used for an exciting and captivating game story. In Because every timestamp tk is a result of a content-based video future work, this algorithm will be extended and adapted according processing step, the resulting video set of the player views and the to the feedback of the reviewers. event stream is synchronised nearly frame accurate. Based on the correct timestamps and the duration of the killstreaks, the videos ACKNOWLEDGMENTS are extracted using ffmpeg and the multi-view summary seen in Special thanks to Mathias Lux who has contributed as a supervisor Figure 3 is created with OpenCV. and coordinator of this project. GameStory: Video Game Analytics Challenge MediaEval’18, 29-31 October 2018, Sophia Antipolis, France REFERENCES [1] G. Bradski. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000). [2] Mathias Lux, Michael Riegler, Duc-Tien Dang-Nguyen, Marcus Lar- son, Martin Potthast, and Pål Halvorsen. 2018. GameStory Task at MediaEval 2018. In Working Notes Proceedings of the MediaEval 2018 Workshop. [3] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.