=Paper=
{{Paper
|id=None
|storemode=property
|title=Serious Fun: Cartooning for Privacy Protection
|pdfUrl=https://ceur-ws.org/Vol-1043/mediaeval2013_submission_74.pdf
|volume=Vol-1043
|dblpUrl=https://dblp.org/rec/conf/mediaeval/Erdelyi0R13
}}
==Serious Fun: Cartooning for Privacy Protection==
https://ceur-ws.org/Vol-1043/mediaeval2013_submission_74.pdf
Serious Fun: Cartooning for Privacy Protection
Ádám Erdélyi, Thomas Winkler Bernhard Rinner
Institute of Networked and Institute of Networked and Institute of Networked and
Embedded Systems Embedded Systems Embedded Systems
Alpen-Adria Universität Alpen-Adria Universität Alpen-Adria Universität
Klagenfurt Klagenfurt Klagenfurt
Lakeside Park B02b, Lakeside Park B02b, Lakeside Park B02b,
9020 Klagenfurt, Austria 9020 Klagenfurt, Austria 9020 Klagenfurt, Austria
adam.erdelyi@aau.at thomas.winkler@aau.at bernhard.rinner@aau.at
ABSTRACT
This paper presents our solution to the 2013 MediaEval
Visual Privacy Task [3]. We apply cartoon-like effects to
captured video such that identities of persons are protected
while behavioural information and hence system intelligi-
bility are maintained. We present our processing pipeline
which includes additional protection steps such as re-colour-
ing or additional blurring and discuss early evaluation re-
sults.
1. MOTIVATION AND GOALS
Figure 1: Pipeline of the proposed privacy filter.
Privacy in visual surveillance is one of the outstanding
open issues currently investigated by several research groups.
Privacy protection goes along with a loss of intelligibility required. First, the motion regions are detected by using
(i.e., utility of the video’s content) since filters such as blank- background extraction. A preliminary blurring with a 7 × 7
ing or pixelization reduce the amount of visible information. size kernel is done before the motion region detection in
An ideal approach should achieve a tradeoff which preserves order to reduce noise. Secondly, the edges are detected with
behavioural information while protecting identities. a Sobel edge detector. Both the motion regions and the
The 2013 MediaEval Visual Privacy Task [3] provides a edges are used as masks in the privacy filter.
set of video sequences, the PEViD data-set [5], together
with annotations of sensitive regions (e.g., faces, persons,
2.2 Recolouring Items
carried items) in separate data files. The privacy protection Before applying the cartooning effect, all personal items
algorithms developed by participants should obscure people are recoloured in order to distort their original characteris-
and their personal belongings while maintaining sufficient tics as much as possible. Instead of simply using the rectan-
information for observers to monitor behaviour and actions. gular bounding box from the annotation, we merge it with
With the limited robustness of state-of-the art object de- the motion region mask by using a bitwise-AND operation.
tection techniques in mind, we designed our approach to be This way the colour change is done only to the item and
extendable to cope with misdetections and implicit privacy the viewer’s attention is not distracted by the sharp edges
leakage channels [6]. Our solution focuses on the whole body of a recoloured regular rectangle. The colour change itself is
of monitored people including their carried items instead of performed by shifting the hue level by 180◦ . The RGB (Red-
obscuring their faces only. Green-Blue colour model) frame is converted to HLS (Hue-
Lightness-Saturation colour model) beforehand and back to
RGB after the hue shift.
2. PRIVACY PROTECTION APPROACH
Our approach consists of four sequential steps (cp. Fig- 2.3 Cartooning
ure 1) including (1) pre-processing, (2) item-recolouring, The cartooning effect is applied to the ROI (Region of In-
(3) cartooning and (4) additional face blurring. The individ- terest) in the image frame, namely to the bounding boxes
ual steps are discussed throughout Sections 2.1 to 2.4. Our of people and personal items. A preliminary blur is also ap-
prototype is implemented in C++ using OpenCV [1]. For plied in order to reduce noise and achieve visually better end
parsing the provided annotation files we use pugixml [2]. results. Furthermore, the ROIs are again merged with the
contour mask as described earlier in order to avoid viewer
2.1 Preprocessing distraction. The cartooning is done in two steps:
Before applying the privacy filters, some preprocessing is
1. Applying a Mean Shift Filter [4] with a spatial window
radius of 20 and a colour window radius of 40. This
Copyright is held by the author/owner(s). makes the image smooth and reduces the number of
MediaEval 2013 Workshop, October 18-19, 2013, Barcelona, Spain colours as if it was drawn like a cartoon.
2. To enhance the cartoon look, the original image is re- indicates that a stronger cartooning effect is required to pro-
stored along object contours. A bitwise weighted copy vide adequate privacy in case of human observers.
from the original frame to the processed frame is per-
formed using the gradient mask from the Sobel edge Our Approach: Objective Score Average (9) Objective Score
0.9
detector. This makes the image less blurry after the Our Approach: Subjective Score Average (9) Subjective Score
Mean Shift Filter and more similar to a cartoon where 0.8
contours of the drawings are usually emphasized. Op- 0.7
tionally, a single colour (e.g., black) could be used for
contours instead of copying the original image. How- 0.6
ever, this was found to be too distractive due to the 0.5
added noise.
0.4
2.4 Additional Face Blurring
0.3
In cases where faces are relatively big (i.e., close to the
camera), cartoonization does not provide sufficient privacy 0.2
protection. Therefore, we introduced an additional, adap- 0.1
tive face blurring effect. A Gaussian blur is applied to the
faces in an elliptical ROI with a kernel size that is propor- 0
tional to the size of the face. The elliptical ROIs have gra- Intelligibility Privacy Appropriateness
dient edges in order to avoid viewer distraction.
Figure 3: Evaluation scores for intelligibility, privacy and
3. EXTENDED FUNCTIONALITY appropriateness. Blue bars show the scores for our proposed
approach; Yellow bars show the average scores over the 9
A central aspect of video surveillance is the detection of Privacy Task participants. The plain bars show the result
unusual events such as abandoned luggage. Even though it for objective (i.e., automated) evaluation while the hatched
is not required in MediaEval’s Visual Privacy Task, we in- bars show the results for subjective (i.e., human) evaluation.
tegrated an extra feature into our processing pipeline which
prevents protection from supposedly critical image regions.
Critical events leading to uncovering of such regions are: 5. CONCLUSION AND FUTURE WORK
• A bag or backpack is dropped (i.e., not moving). The performance bottleneck of our approach, that cur-
• An umbrella or a bottle is used as a weapon in a fight. rently prevents real-time applications, is OpenCV’s Mean
• A wallet is being stolen. Shift Filter which is relatively slow. There are alternative
This function further enhances intelligibility and helps the and possibly faster algorithms that can produce a similar
operator of the surveillance system in determining when to cartooning effect which will be explored in the future work.
react. Assuming that such a cartooning algorithm sufficiently
hides facial identity, it would be possible to apply it globally
4. RESULTS to the whole image. This way we do not depend on feature
detectors that typically are not accurate and reliable enough.
Figure 2 presents two pairs of frames (original and modi- Based on the evaluation criteria of the Visual Privacy Task
fied) which show the results of the applied cartooning effect. we created a tool that compares original and filtered videos
The re-colouring of personal items is nicely visible at the in terms of privacy and intelligibility. In future work we will
scarf and the bag in Figure 2b. use it for a comprehensive evaluation of global approaches.
6. REFERENCES
[1] OpenCV – Open Source Computer Vision.
http://opencv.org (last visited: Sept. 2013).
[2] pugixml – Light-weight, simple and fast XML parser
for C++ with XPath support. http://pugixml.org
(last visited: Sept. 2013).
[3] A. Badii, M. Einig, and T. Piatrik. Overview of the
MediaEval 2013 Visual Privacy Task. In Proceedings of
the MediaEval Workshop, Barcelona, Spain, 2013.
(a) Original. (b) Modified. (c) Original. (d) Modified.
[4] Yizong Cheng. Mean shift, Mode Seeking, and
Figure 2: Comparison of original frame and cartoonized Clustering. IEEE Transactions on Pattern Analysis and
frame with re-colouring (bag and scarf) and face blur. Machine Intelligence, 17(8):790–799, 1995.
Figure 3 visualizes the evaluation results provided by the [5] P. Korshunov and T. Ebrahimi. PEViD: Privacy
Visual Privacy Task organizers. In the objective (i.e., auto- Evaluation Video Dataset at Applications of Digital
mated) evaluations our approach consistently achieves higher Image Processing XXXVI. In Proceedings of SPIE,
scores than the average over all 9 participating teams in all volume 8856, San Diego, CA, 25-29 August 2013.
three evaluation categories (intelligibility, privacy and ap- [6] M. Saini, P. K. Atrey, S. Mehrotra, and M. S.
propriateness). For the subjective (i.e., human) evaluation Kankanhalli. Considering Implicit Channels in Privacy
our approach is well above average for intelligibility and ap- Analysis of Video Data. IEEE Communications Society
propriateness but below average for privacy protection. This E-Letters, 6(11):27–30, 2011.