The Medico: Multimedia for Medicine Task is running for the sixth time as part of MediaEval 2022. This year, the task focuses on automatically detecting and tracking spermatozoa in video recordings of human semen. We provide a dataset with 20 video recordings of spermatozoa with manually annotated bounding-box coordinates and a set of sperm characteristics. The task consists of four subtasks.
The 2022 Medico task tackles the challenge of tracking spermatozoa cells in video recordings of untreated semen samples. Manual evaluation of sperm motility using a microscope is timeconsuming and expensive, and requires experts who have extensive training. Furthermore, the validity of manual sperm analysis becomes unreliable due to limited reproducibility and high inter-personnel variations due to the complexity of tracking, identifying, and counting sperm in fresh samples. The existing computer-aided sperm analyzer systems are not working well enough for application in a real clinical setting due to unreliability caused by the consistency of the semen sample. Therefore, we need to research new methods for automated sperm analysis.
The goal is to encourage task participants to track individual spermatozoa in real-time and combine diferent data sources to predict common measurements used for sperm quality assessment, specifically the motility of the spermatozoa.
We hope that this task will encourage the multimedia community to aid in the development of computer-assisted reproductive health and discover new and clever ways of analyzing multimodal datasets. In addition to good analysis performance, an important aspect is also the eficiency of the algorithms due to the fact that the assessment of the sperm is performed in real-time and therefore requires real-time feedback. Furthermore, adding explainability to predictions of machine learning algorithms can improve the trust between medical experts and machine learning solutions.
For the task, we will provide a dataset that contains in total 20 30-second videos, a set of sperm characteristics (hormones, fatty acids data, etc.), frame-by-frame bounding box annotations, some anonymized study participants-related data, and motility and morphology data following the WHO recommendations.
Based on this data, the participants will be asked to solve the following four subtasks, where subtasks 3 and 4 are optional: • Subtask 1: The goal of this subtask is real-time tracking of sperm cells in a given semen video. Tracking should be performed by predicting bounding box coordinates with the similar format to the bounding box coordinates provided with the development datasets. In this task, models should track sperm in each frame of a provided video in real-time.
Therefore, frames per second is an important factor to measure. • Subtask 2: The goal of this task is to predict sperm motility1 in terms of the percentage of progressive and non-progressive spermatozoa. The prediction needs to be performed patient wise, resulting in one value per patient per predicted attribute. Sperm tracking or bounding boxes predicted in task 1 are required in order to solve this task. • Subtask 3: This task focuses on identifying the fastest sperm cells with corresponding average speed and highest top speed. One specific challenge with this subtask is that the video also changes the view of the sample. This happens because the sample is moved below the microscope to observe the complete sample area. Therefore, the tracking has to be performed per viewpoint on the sample. • Subtask 4: This subtask builds on task 1, 2 and 3. The goal is to provide an explanation of the model predictions to convince domain experts about the correctness of the results. There are not any specific requirements for this task, and participants can choose for which subtask they want to provide the explanations. A report should be provided that can be given to medical domain experts.
For subtasks 2 and 3, the participants are encouraged to consider the temporal aspect of the videos. This is important due to the fact that single-frame-based analysis will not be able to catch the movement of the sperm (motility), which contains important information to perform proper predictions on subtasks 2 and 3.
The 2022 Medico task uses the data set VISEM [
For each patient, we include a video of live sperm (video and extracted frames), manually
annotated bounding box details for each spermatozoon, a set of measurements from a standard
semen analysis for the whole sample, a sperm fatty acid profile, the fatty acid composition of
serum phospholipids, study participants-related data, and WHO analysis data. The bounding
box coordinates are provided in two separate folders, one has bounding box coordinates in
YOLO format [
All study participants agreed to donate their data for the purpose of science and provided the necessary consent for us to be able to distribute the data (checked and approved by the Norwegian data authority and ethical committee).
For the first sub-task, sperm cell tracking, the mAP (mean average precision) will be calculated as follows: = 1 ∑︁ =1 (1) with two diferent threshold values of 0.5 and 0.5 : 0.95. For the second range of threshold values, the average of incremental threshold values between 0.5 and 0.95 by 0.05 will be calculated. Finally, a fitness value will be calculated as follows: _ = 0.1 × 0.5 + 0.9 × 0.5:0.95. (2)
In addition to the mAP and fitness values, we will calculate frames per second (FPS) for the ifrst task to evaluate the speed of predictions. For the evaluation of the second task, we calculate the mean squared error and mean absolute error. Additionally, the time required to predict the output of the subtask 2 is also calculated.
The optional third and fourth tasks will be evaluated using manual evaluation with the help of three diferent experts within human reproduction. A fixed hardware setup will be used for all evaluations to measure the speed of predictions.
Previous studies of applying deep learning solutions [4, 5, 6] in sperm analysis show the potential of applying deep learning techniques to predict morphology and motility levels of given sperm videos. However, the high values of mean average errors of the studies imply that pure video input to deep learning models makes predictions less accurate. In this regard, we provide additional manually annotated bounding boxes for three classes, namely sperm, sperm clusters and small or pinhead sperms. We believe that these additional localization details will help to improve the deep-learning-based sperm analysis, which again will help future assisted human reproduction. [4] S. A. Hicks, J. M. Andersen, O. Witczak, V. Thambawita, P. Halvorsen, H. L. Hammer, T. B. Haugen, M. A. Riegler, Machine learning-based analysis of sperm videos and participant data for male fertility prediction, Scientific reports 9 (2019) 1–10. [5] V. Thambawita, P. Halvorsen, H. Hammer, M. Riegler, T. B. Haugen, Extracting temporal features into a spatial domain using autoencoders for sperm video analysis, arXiv preprint arXiv:1911.03100 (2019). [6] V. Thambawita, P. Halvorsen, H. Hammer, M. Riegler, T. B. Haugen, Stacked dense optical flows and dropout layers to predict sperm motility and morphology, arXiv preprint arXiv:1911.03086 (2019).