This paper presents our solution to the 2013 MediaEval Visual Privacy Task [3]. We apply cartoon-like e ects to captured video such that identities of persons are protected while behavioural information and hence system intelligibility are maintained. We present our processing pipeline which includes additional protection steps such as re-colouring or additional blurring and discuss early evaluation results.
Privacy in visual surveillance is one of the outstanding open issues currently investigated by several research groups. Privacy protection goes along with a loss of intelligibility (i.e., utility of the video's content) since lters such as blanking or pixelization reduce the amount of visible information. An ideal approach should achieve a tradeo which preserves behavioural information while protecting identities.
The 2013 MediaEval Visual Privacy Task [
With the limited robustness of state-of-the art object
detection techniques in mind, we designed our approach to be
extendable to cope with misdetections and implicit privacy
leakage channels [
Our approach consists of four sequential steps (cp.
Figure 1) including (1) pre-processing, (2) item-recolouring,
(3) cartooning and (4) additional face blurring. The
individual steps are discussed throughout Sections 2.1 to 2.4. Our
prototype is implemented in C++ using OpenCV [
Before applying the privacy lters, some preprocessing is required. First, the motion regions are detected by using background extraction. A preliminary blurring with a 7 7 size kernel is done before the motion region detection in order to reduce noise. Secondly, the edges are detected with a Sobel edge detector. Both the motion regions and the edges are used as masks in the privacy lter. 2.2
Before applying the cartooning e ect, all personal items are recoloured in order to distort their original characteristics as much as possible. Instead of simply using the rectangular bounding box from the annotation, we merge it with the motion region mask by using a bitwise-AND operation. This way the colour change is done only to the item and the viewer's attention is not distracted by the sharp edges of a recoloured regular rectangle. The colour change itself is performed by shifting the hue level by 180 . The RGB (RedGreen-Blue colour model) frame is converted to HLS (HueLightness-Saturation colour model) beforehand and back to RGB after the hue shift. 2.3
The cartooning e ect is applied to the ROI (Region of
Interest) in the image frame, namely to the bounding boxes
of people and personal items. A preliminary blur is also
applied in order to reduce noise and achieve visually better end
results. Furthermore, the ROIs are again merged with the
contour mask as described earlier in order to avoid viewer
distraction. The cartooning is done in two steps:
1. Applying a Mean Shift Filter [
In cases where faces are relatively big (i.e., close to the camera), cartoonization does not provide su cient privacy protection. Therefore, we introduced an additional, adaptive face blurring e ect. A Gaussian blur is applied to the faces in an elliptical ROI with a kernel size that is proportional to the size of the face. The elliptical ROIs have gradient edges in order to avoid viewer distraction.
A central aspect of video surveillance is the detection of unusual events such as abandoned luggage. Even though it is not required in MediaEval's Visual Privacy Task, we integrated an extra feature into our processing pipeline which prevents protection from supposedly critical image regions. Critical events leading to uncovering of such regions are: A bag or backpack is dropped (i.e., not moving). An umbrella or a bottle is used as a weapon in a ght.
A wallet is being stolen.
This function further enhances intelligibility and helps the operator of the surveillance system in determining when to react.
(a) Original. (b) Modi ed. (c) Original. (d) Modi ed. indicates that a stronger cartooning e ect is required to provide adequate privacy in case of human observers.
Our Approach: Objective Score Our Approach: Subjective Score
Average (9) Objective Score
Average (9) Subjective Score Intelligibility
Privacy
Appropriateness 5.
The performance bottleneck of our approach, that currently prevents real-time applications, is OpenCV's Mean Shift Filter which is relatively slow. There are alternative and possibly faster algorithms that can produce a similar cartooning e ect which will be explored in the future work.
Assuming that such a cartooning algorithm su ciently hides facial identity, it would be possible to apply it globally to the whole image. This way we do not depend on feature detectors that typically are not accurate and reliable enough.
Based on the evaluation criteria of the Visual Privacy Task we created a tool that compares original and ltered videos in terms of privacy and intelligibility. In future work we will use it for a comprehensive evaluation of global approaches.