This paper describes a system for automated killstreak extraction in CS:GO tournaments. It uses a metadata and content-based approach for identification of the killstreaks. Inaccurate timestamps in the provided metadata of the tournament resulted in almost unusable detection results. Therefore, a content-based technique has been developed to extract the correct highlight videos. The results are merged into a summary video with a multi-view perspective of the event stream, the actor's view and the views of the victims. Due to the content-based approach, the multi-view video could be synchronised nearly frame accurate.
E-sports events such as the CS:GO World Championship 2018 in
Katowice are becoming increasingly popular. One of the key
challenges of such an event is to summarise it in an exciting and
captivating manner. E-sports events are rich in multimedia data such as
video and audio streams, player details, commentator descriptions
and comments from viewers. One of the tasks in the MediaEval
2018 Workshop [
Two diferent types of metadata are provided by the task organisers. The stream metadata consists of information about the video streams including timestamps and duration of the matches. The game log contains a detailed listing of all events such as purchases, kills and the winning team for each round. Unfortunately, the timestamps in the stream metadata may difer up to 40 seconds from the timestamps in the game log which is a typical problem in multimedia systems. In order to use the information in the game log, it is therefore necessary to find out the exact time of an event. Based on the game log a model has built which contains additional information about the game score for each round and keeps track of the overall game state. Additionally, a killstreak list has been generated by processing the game log. This list is used to identify all players who were involved in a killstreak (actor and victims) and to store the start time and the duration of the killstreak. 3
With the generated model all killstreaks with a maximum duration of 20 seconds and a minimum killstreak length of three players were identified as candidates for the summary. As seen in Figure 1 the main dificulty in extracting the killstreaks is to identify the start time of the killstreak. The begin of a killstreak t extracted from the game log may difer up to δ = 40 seconds from the timestamps in the stream metadata. This means that the correct start point t ′ could be in the range of:
t − δ ≤ t ′ ≤ t + δ To find the correct timestamp, an additional video processing step has been implemented. First, the beginning of the round in which the killstreak was occurred has been identified. Second, based on the round begin and the information in the model the beginning of the killstreak has been found. 3.1
To determine the correct start of a killstreak, the model retrieves the number of kills that were done before the killstreak has started. Based on this number k the timestamp of the first kill from the killstreak can be determined. But due to the potential displacements of the timestamps, there can be two consecutive rounds in the search window t − δ ≤ t ′ ≤ t + δ in which an equal number of kills were occurred. Therefore, it is necessary to first identify the round begin of a killstreak. As an indication for the correct round start, the diference between the timestamps of the match begin and the (target kill + δ ) is added to the given video position of the target match. Starting from that point the match begin is being searched.
As seen in Figure 2 the match score is displayed in the upper part of the video. Two Regions of Interest (ROI) are extracted where the score numbers are displayed. A Support Vector Machine (SVM) is used to classify the ROIs. The SVM was trained with 400 samples of each number from 0-21 which where extracted beforehand from the training set. Every frame in an extended search window with δ = 1 minute is processed by the SVM and after an amount of 60 matches with the target score, the timestamp of the round start tr is detected. The decision for choosing a SVM for the number detection problem is based on the fast computation speed and satisfying precision that was achieved during development. 3.2
A pattern matching approach is used for the detection of the killstreak begin. Given the information from the model, that n kills were achieved before the killstreak has begun, the timestamp tk where (n + 1) kills were achieved is being searched. As seen in Figure 2 a kill is indicated with a skull at the left or right side of the screen. For that reason ten ROIs where a skull could possibly appear have been extracted and compared with a reference picture of the skull. The comparison was done with the L1-norm of the normalised reference and ROI values. If the norm has fallen below a threshold value the kill counter is increased. At the moment the kill counter reaches (n + 1), tk is reached and can be used as the start point for extracting the video of the killstreak. 3.3
The technique described above has first been applied to the event stream. Later it has been used for extracting the killstreaks from the player streams. To achieve a better control of the target video length and a better overall viewing experience, a clipping-ofset between 5 and 20 seconds is applied before tk and after the killstreak end. Because every timestamp tk is a result of a content-based video processing step, the resulting video set of the player views and the event stream is synchronised nearly frame accurate. Based on the correct timestamps and the duration of the killstreaks, the videos are extracted using fmpeg and the multi-view summary seen in Figure 3 is created with OpenCV.
Three diferent summaries were created from the extracted videos. Every video consists of nine killstreaks, ordered by their corresponding round numbers. Video one and two show the synchronised event view and killstreak actor view. As seen in Figure 3 the last video is a multi-view summary and is made from the event view, actor view, and the views of all victims. The videos where watches by two to four reviewers from MediaEval and evaluated based on ifve criteria: match summary, entertainment value, flow and peak of a good story, innovation and portability for other games. Some of the shortcomings of the reviewers should now be briefly addressed.
The summary covers only the killstreaks greater than three which were achieved during the match. This seems like a quick and easy approach. But if the timestamps in the metadata files can difer up to 40 seconds from the actual time, the provided information is useless. Because the compensation of the timestamp diference was an unexpected time-consuming task, many ideas like the integration of overtime events and audio-based analysis could not be done.
Another aspect which is not covered by the summary is the impact of money and the player’s purchases at round begin. An integration and analysis of this data could have produced additional highlight videos. For example, one could investigate if a team is playing with a weak load-out against a team with strong equipment and still wins the round. In addition, sneak and melee kills could be easily integrated by analysing the metadata. Nevertheless the metadata timestamps of such events must be synchronised with the actual event times. Other aspects like pitfalls and mistakes made by other players are definitely interesting and would make the summary more exciting.
The algorithm presented in this paper is able to automatically detect and extract sequences of killstreaks from the whole tournament data. Nevertheless, killstreaks are just a subset of events which could be used for an exciting and captivating game story. In future work, this algorithm will be extended and adapted according to the feedback of the reviewers.
Special thanks to Mathias Lux who has contributed as a supervisor and coordinator of this project.