=Paper=
{{Paper
|id=None
|storemode=property
|title=Overview of the MediaEval 2013 Visual Privacy Task
|pdfUrl=https://ceur-ws.org/Vol-1043/mediaeval2013_submission_93.pdf
|volume=Vol-1043
|dblpUrl=https://dblp.org/rec/conf/mediaeval/BadiiEP13
}}
==Overview of the MediaEval 2013 Visual Privacy Task==
https://ceur-ws.org/Vol-1043/mediaeval2013_submission_93.pdf
Overview of the MediaEval 2013 Visual Privacy Task
Atta Badii1, Mathieu Einig1, Tomas Piatrik2
1 2
University of Reading Queen Mary,
Intelligent Systems Research Laboratory, University of London
School of Systems Engineering Multimedia & Vision Research Group
United Kingdom United Kingdom
{atta.badii; m.l.einig}@reading.ac.uk tomas.piatrik@elec.qmul.ac.uk
ABSTRACT
T his paper describes the Visual Privacy T ask (VPT ) 2013, its
3. VISUAL PRIVACY TASK
T his task explores how image processing, computer vision and
scope and objectives, related dataset and evaluation approach.
scrambling techniques can deliver technological solutions to
some visual privacy problems [2] [3] [4]. T he goal of privacy
1. INTRODUCTION protection is to prevent potential access to information, the
Advances in artificial intelligence and video surveillance have led divulgement of which can amount to a (perceived) intrusion of
to increasingly complex surveillance systems of rising scale and an individual’s privacy. T he extent of such a (perceived) loss of
capabilities. T his ubiquity and intelligence poses great threats to privacy depends on the individual as well as the context and as
privacy, and new mitigation technologies must be found to such can only be determined by reference to the user (“ data-
ensure an appropriate level of privacy protection. T he Visual subject”) in each case. Context-specific privacy protection
Privacy T ask (VPT ) aims at exploring how image processing, constitutes an interesting extension of this VPT task which is
computer vision and scrambling techniques can deliver planned to be included in future challenges. T he goal of this
technological solutions to some visual privacy problems. T he VPT is to propose methods whereby persons featured in digital
evaluation is performed using both video analytics algorithms imagery can be obscured so as to render them unrecognisable.
and user studies so as to provide both subjective and objective Privacy level variations may also be triggered by detected
evaluation of privacy protection techniques. T he manner in anomalies, critical events, and alerts etc. or be based on prior
which the privacy of individual actors appearing in a video scene official permission granted by higher authorities to suspend the
may need to be protected can, at least partially, depend on their masking of the identity of an individual in specific cases. Since
context-specific preferences for privacy. T he context can the resulting partially obscured videos would still have to convey
include their behaviour and interaction with each other and/or some video information to be worth viewing, an optimal balance
with any objects in the scene. Effective privacy protection must should be struck so that despite the extent of such masking of
model any such context-dependent personal privacy preferences the facial identity as may be necessary, the categorical identity
and this is a challenging extension of this VPT for the future. of any masked actors e.g. humans can still be recognisable to the
viewer. T hus identity obscuring techniques should not result in
2. THE VPT 2013 DATASET artefacts that are ‘socially inappropriate/offensive’ and
T he data set consists of videos collected from a range of unacceptable to the human users. T he participants should also
standard and high resolution cameras and contains clips of demonstrate that their choice of obscuring technique is such that
different scenarios showing one or several persons walking or the resulting obscured (e.g. pixelated) faces do not tend to fixate
interacting in front of the cameras. People may also carry a viewers’ attention thus distracting the viewer and/or adversely
specific items which could potentially reveal their identity and impacting the acceptability-usability of any obscured/scrambled
may therefore need to be filtered appropriately. For this year, images, from the perspective of both the data-subject as well as
people can carry backpacks, umbrellas, wear scarves, and can be other viewers. Participants are provided with videos containing
seen fighting, pickpocketing or simply walking around. People faces from different camera angles. T he ground truth consists of
may be at a distance from the camera or near the camera, annotations of persons’ images, including face, hair, visible skin
making their faces vary considerably in pixel size and quality. regions, as well as their personal accessories.
T he videos have variable ambient lighting with half of the clips
recorded at night. T he dataset contains 22 video clips and 3.1 Objective metrics
associated annotations in xml form. T he videos include indoor, T he objective metrics are computed automatically with a
outdoor, day-time and night-time environments, showing people mixture of object detection and matching in order to evaluate
interacting or performing various actions. T he clips are in the the impact of the filtering on the privacy and intelligibility.
mpeg format with a resolution of 1920x1080 pixels at 25 Some additional image quality measures will be taken into
frames per second. Publications arising from experiments account in order to give credit to filters resulting in visually-
performed using PEViD must acknowledge its publishers [1]. pleasant masking.
Copyright is held by the author/owner(s).
3.1.1 Face Detection
MediaEval 2013 Workshop, October 18-19, 2013, Barcelona,
A face detection algorithm will be run on the obscured videos
Spain
submitted for the evaluation using the Viola-Jones face detection
from OpenCV library. Ideally, no faces should be found, since UI-REF based privacy protection requirements. T his subjective
they all should be obscured. T he faces found by the face evaluation will take into account three main aspects of any
detection algorithm are matched against the ground truth to obscured (element of) image, namely intelligibility, privacy, and
avoid including the false positives of the detection algorithm. appropriateness. In the context of surveillance scenarios,
questions related to whether a person wears personal items that
3.1.2 Object Tracking can be used for identification e.g. (branded) backpack, scarf, etc,
T he intelligibility is measured by applying the Histogram of will be considered as relevant to privacy and intelligibility.
Oriented Gradient as a human detector taking the video images
as input. Successful detections of a human means that even
although the sensitive areas may have been obscured, the
resulting video could still carry sufficient visible clues for Video
Analytics including tracking. T hese detections are compared
against the detections from the raw video.
3.1.3 Person Re-identification
A visual model of the un-filtered images of persons as featured in
the video set will be developed and matched against the privacy-
filtered versions of the images of the same persons as selected
from the submission set. T he matching process will be
implemented in two ways so as to provide the basis for a Merit
Criterion Framework for Privacy Impact Assessment based on Figure 1. Sample frame from the VPT Data Se t [1]
Efficacy, Consistency, Disambiguity and Intelligibility PIAF[5];
as follows: i) by building a visual model from the original T he visual appropriateness of the obscured images will be
unfiltered image in each case and then attempting to match this evaluated based on the various aspects such as pleasantness,
against the respective filtered image, and, ii) by building the distraction, and user acceptance for video surveillance, etc. T he
model from the filtered image and attempting to match it visual appropriateness criterion will essentially follow a UI-REF
against the respective unfiltered original set. A low re- based evaluation methodology [6]. T his metricates: i) the
identification score arising from the above matching cycles categoric “ recognisability” of an obscured image as a member of
would indicate a higher Efficacy privacy protection afforded by a particular species, and, ii) the obscuring Effects, and Side-
the privacy filtering techniques as deployed in each case. Effects on the perception of the image by a viewer, and, iii) the
Consistency, Disambiguity and Intelligibility properties of the extent of any resulting negative or positive emotions or Affects
deployed Privacy Filtering approach will also be assessed by or distraction in the mind of the viewer of an image that has
comparing the filtered visual model to the filtered instances of been subjected to such obscuring (indignity/stigma). Insights
the target person(s) in the image set. A high score would from this user study will serve as a baseline for refining the
indicate that the filtered video still carries sufficient information metrics and shall inform the design of the future privacy tasks.
to enable an observer to perform tasks such as person tracking
across images from the CCT V network without finding out the
4. ACKNOWLEDGMENTS
T his Visual Privacy MediaEval task was supported by the
person’s identity. T he framework has been extended to enable
European Commission under contracts FP7-261743 VideoSense.
the video-context-sensitive thresholding of the Merit Criteria.
T his provides a powerful benchmarking mechanism for the 5. REFERENCES
spectrum of possible privacy filtering techniques, in terms of [1] Korshunov P. & Ebrahimi T ., “ PEViD: privacy evaluation
their optimisation of the trade-offs (identity maskability video dataset”. Applications of Digital Image Processing
/trackability) across the specific criteria to suit the objectives of XXXVI, San Diego, California, USA, August 25-29, 2013.
the video processing with privacy protection and surveillance by
best balancing the resulting Efficacy, Consistency, Disambiguity [2] Dufaux, F. & Ebrahimi, T ., “ Scrambling for Privacy
and Intelligebility impacts of particular privacy filtering Protection in Video Surveillance Systems,” IEEE
techniques as deployed in arbitrary situated video-contexts and T ransaction on Circuits and Systems for Video T echnology,
UI-REF based privacy requirements. Vol. 18, Nr. 8 (2008), p. 1168-1174
[3] Dufaux, F. & Ebrahimi, T ., “ A framework for the
3.1.4 Metric for Visual Appropriateness validation of privacy protection solutions in video
Obscuring of the image of persons and their accessories will be surveillance,” 2010 IEEE International Conference on
evaluated using SSIM and PSNR metrics for image quality based Multimedia and Expo (ICME), pp.66-71, 19-23 July 2010.
on the human eye perception of salience in the image. A [4] Senior, A., “ Privacy Protection in a Video Surveillance
successful privacy filtering system should have a minimal impact System,” Privacy Protection in Video Surveillance,
on the global quality of the image with modifications occurring Springer, 2009
only on the sensitive areas which should be thus anonymised.
[5] Badii, A, Einig, M, Al-Obaidi, Ducournau, A, “ T he Merit
Criteria Framework for Impact Assessment of Privacy
3.2 User Study for Assessment of
Filtering T echnologies, based on Efficacy, Consistency,
Appropriateness of Visual Privacy Filtering Disambiguity and Intelligibility of Privacy Protection”,
A random subset of videos from the submitted runs will also be Working Paper UoR-ISR-VS-2013-3, May2013.
evaluated through a user study aimed at developing a deeper
understanding of user perceptions of appropriateness in terms of [6] Badii, A, “ UI-REF Methodology”, articles available online.