Positions and actions detection/classification are one of the main challenges in visual content analysis and mining. Sports video ofer such challenges due to the variety of scenes and actions they contain. Sports also provide a wide range of analysis, related to athletes' performances and tactics. We propose a series of 6 sports-related tasks, divided each into 2 sub-tasks for two sports, table tennis and swimming. Those tasks are a follow-up from the Sport Task and SwimTrack task from the 2022 MediaEval Benchmark.
We present SportsVideo, a series of six sports-related multimedia tasks, divided in sub-tasks for
table tennis and swimming. The dataset is a merge of the Sport Task (Table Tennis) [
By combining two diferent sports –table tennis and swimming– we aim to encourage an
approach that generalizes beyond a single sport type since the two sports are very diferent in
terms of the type of video, the type of events and the type of analysis required. Table tennis is a
fast-paced sport with a lot of action and a small field of view. Swimming is a slower sport with
a large field of view and a lot of occlusions. We expect participants to develop approaches that
can generalize to both, but also to other sports. Those tasks have been identified and designed
to be as independent as possible so that participants can choose to participate in one or more
tasks. But if combined, they can provide a more complete analysis of sports videos. Participants
are encouraged to release their code publicly with their submission. This year, similarly to
the Sport Task 2022 edition [
The main information in sports is related to positions of players in videos featuring diferent numbers of lanes in a swimming pool and sides around a table tennis (seen from various angles). They need to provide bounding boxes for identified players, and their results are evaluated using Average Precision (AP) at an IoU ratio of 0.25, counting true positives and negatives across the dataset.
Subtask 1.1 (table tennis) – To detect 2 or 4 players (depending if single or double) and track them during the video especially during double games where players have a lot of overlaps, from videos recorded from various angles (e.g., side, corder).
Subtask 1.2 (swimming) – To detect up to 8 swimmers in the pool from static videos (recorded
from the side of the pool). A baseline is provided in [
Another key information in sports video is related to events, in particular strokes, which are key for performance evaluation. In swimming, stroke events indicate the pace of swimming for Freestyle, Backstroke, and Butterfly are triggered when the right hand enters the water, while for Breaststroke, when the head reaches its highest point.
Subtask 2.1 (table tennis) – To detect when a player is performing a stroke (i.e. a ball hit with the racket) using close-up videos. The goal is to detect the exact frame when the ball is hit by the racket. Videos are provided with the ball annotated. Evaluation is based on the F1-score, which measures the harmonic mean of precision and recall.
Subtask 2.2 (swimming) – To detect each time a swimmer is achieving a repeated motion for
each swimming style (for freestyle, backstroke, and butterfly stroke once the swimmer’s right
hand enters the water; for breaststroke once the head is at its highest point). Swimmers’ strokes
are identified after the underwater phase and until the swimmer finishes the race, excluding
underwater phases for races longer than 50m. Video clips of cropped swimmers are provided,
and evaluation is based on Of-By-One Accuracy, which measures the proportion of correctly
estimated stroke counts within a tolerated error of one stroke in the dataset. A baseline is
provided in [
The goal of this task is to classify the type of stroke performed by a player in table tennis and swimming. Participants need to categorize a collection of shortened table tennis stroke videos, each containing either a single stroke or no stroke. There are 20 potential stroke categories and an extra category for non-strokes. Two sets with annotations are given: a training set with 807 videos and a validation set with 230 videos. The challenge is to classify a non-annotated test set comprising 118 videos, with the assurance that the trimmed videos in each set are derived from the same untrimmed videos but captured at diferent time instances without temporal overlap. Subtask 3.1 (table tennis) – To classify diferent strokes in table tennis from trimmed videos in which only one stroke is present. There are 3 diferent categories of strokes, services, forehand and backhand. For services we have 6 diferent classes. For forehand and backhand we have 5 classes. For a total of 16 classes and one non-stroke class.
Subtask 3.2 (swimming) – To classify diferent swimming styles (Freestyle, Backstroke, Breaststroke, Butterfly).
Sports videos in general, and the one we provided, are usually recorded from the side. This tasks asks participants to find the absolute homography matrix for each frame in the dataset. The precision of this projection is evaluated using Intersection over Union (IoU), with two metrics: IoU for the visible pool parts and IoU for the entire pool, including parts outside the camera’s ifeld of view.
Subtask 4.1 (table tennis) – To detect the table position for a given video frame of a whole table tennis. The dataset contains 54 annotated images with homography matrix from TV broadcasts of table tennis matches.
Subtask 4.2 (swimming) – To detect pool position for a given video frame. The dataset is
provided with 500 annotated images with homography matrix from [
Sports are highly multi-modal events. Sound is an important modality that can be used to detect events. They can either be used as oficial cues (e.g., buzzer sound in swimming) or as additional cues (e.g., ball bounce in table tennis). In this task, participants are asked to detect sound events in table tennis and swimming videos. In table tennis, the ball bounces on the table at each stroke.
Subtask 5.1 (table tennis) – Ball hits indicate the pace of the game. The goal is to detect the exact frame when the ball bounces on the table. Videos are provided with the ball annotated and evaluation is based on the F1-score.
Subtask 5.2 (swimming) – A buzzer sound (preceded by an "on your mark") signals the start. Participants are asked to find the time of this sound within audio files extracted from live videos. These files may or may not include the buzzer sound, which can occur at various points during the recording. This task is challenging because the sound might be recorded from a considerable distance and could be accompanied by significant background noise.
In most sports, the outcome is presented on a scoreboard, featuring race time for each swimmer (and possibly extra data like reaction time) or the current score of the game. These scoreboards are typically either physical LCD screens on the wall or close to the referee. Digital versions shown on TV broadcasts.
Subtask 6.1 (table tennis) – To recognise the score of the match. In table tennis, the score of a match can be embedded in the broadcast video or it can be shown by referees with scoreboards. When score is embedded in stream video, names of players are also displayed. Subtask 6.2 (swimming) – The task here involves extracting swimmers’ names, lane numbers, and race results (times) from screenshots of these scoreboards. The images and scoreboard coordinates will be provided, with the localization aspect already addressed. During swimmer competitions, after each race, results are displayed on digital boards. The goal is to recognise characters of these boards to obtain the results of races.
We thank the people involved in the previous versions of this challenge. This project was partially funded by the ANR NePTUNE, grant number ANR-19-STHP-0004 and the FFTT partnership convention.