Fishing is one of the most important food sources globally. Commercial fishing can potentially be more eficient, precise, and accountable, and if artificial intelligence should be one of the remedies for improvement, one need a better understanding of inner details and what processes are happening on a ifshing trawler. The NjordVid task aims to encourage researchers to tackle this challenge in addition to preserving the privacy of the people working on these boats. The participants are asked to detect events in videos taken on the fishing trawler and to enhance privacy for the people visible in the videos.
Surveillance on board fishing vessels has been argued to be a necessity for sustainable fishing
practices and for our ability to fight fraud in the fishery industry [
With this competition we hope to achieve a better understanding of the processes that happen on a fishing trawler and in addition we want to encourage the community to work on this important topic.
The Njord dataset [
For each video, we have labeled bounding boxes around people, other boats, nets, and fish. The temporal annotations consist of when scene changes occur, when overlays are turned on and of, when Events of Interest (EoI) occur, and when the intro plays. We also have labels that denote whether it is daytime or nighttime, and, due to the videos being from a live-stream, labels for parts of the videos that are before the introduction and after the end of the relevant live-stream. The bounding boxes for fish label groups of fish due to the scenes on deck showing ifsh being far away from the camera. The bounding boxes for the nets both label nets in use and those lying in heaps on deck.
The dataset is organized as follows. The videos directory contains a subdirectory for each annotated video that contains the video in .mp4 format and two annotation files, one file for the bounding box annotations and one file for the timeline annotations. The two annotation files are structured as csv-formatted files using a semi-colon as the delimiter. The bounding box contains one line per bounding box annotation with the following seven values; class, frame number, center x-position, center y-position, the bounding box’s width, and the bounding box’s height. The width and height have been normalized by dividing each by the video’s width and height, respectively. The timeline annotation file contains one line per annotated class and includes the following two values; the class of the frame and the frame number of the corresponding video. The videos directory also contains an unannotated subdirectory containing all videos that have not been annotated yet.
The dataset Njord is publicly available under the CC BY-NC 4.0 International license.
The NjordVid task consists of two diferent subtasks, which can be tackled independently depending on your research area of interest. The dataset consists of 198 surveillance videos from a fishing trawler, of which 71 are annotated with bounding boxes and temporal annotations. The goal of the task is to both gain insight into what is happening on fishing vessels and also investigate methods for preserving the privacy of the fishing crew. 3.1. Subtask 1 Detection of events on the boat: The participants are asked to detect events on the boat like people moving, fish caught, etc. In addition to simple detection of the events we also ask the participants to provide an interestingness score which relates to how uncommon the event is. The score should be between 0 to 1 where 0 determines a very common event and 1 a very uncommon event. 3.2. Subtask 2 Privacy of onboard personnel: For this task the participants are asked to develop methods to preserve the privacy of the people working on the boat, which includes anything that can identify the person (face, nametags, etc). At the same time the privacy preserving measurements should have as little impact on the analysis as possible.
For the evaluation of subtask 1 we will use the standard metrics Precision, Recall, F1 score and Matthew correlation coeficient. The interestingness score provided by the participants will be used to weight the resulting scores.
For subtask 2 we will have a group of manual evaluators checking the privacy aspects on the test dataset (basically is the person still identifiable by a human observer or not). In addition we will calculate some metrics before and after the method was applied. Specifically, we will apply an object detection model and evaluate with classic regression metrics before and after the privacy-preserving method is applied.
In this section we present some baseline results obtained by training a simple object-tracking model using YOLOv5 on the development dataset. Table 1 shows the performance metrics based on the ground truth given in the development dataset and Figure 1 provides some example images with resulting bounding boxes.
The task focuses on the exploration of a completely unknown area where automatic multimedia analysis can have an important impact. We hope that the task will lead to new insights and research questions in addition to inspiring researchers to work on this important topic. For the future we envision a more complex and multimodal dataset that also contains sensor readings and other additional information.
We particularly thank Hermes staf and owners for relevant discussions, meetings, and for allowing us to annotate and publish the Njord dataset and use it for MediaEval.