Privacy Protection Filter Using StegoScrambling in Video Surveillance Natacha Ruchaud Jean Luc Dugelay Eurecom Eurecom 450 Route des Chappes 450 Route des Chappes Biot Sophia antipolis, France Biot Sophia antipolis, France ruchaud@eurecom.fr dugelay@eurecom.fr ABSTRACT age. This paper introduces a new privacy filter adopted in the Privacy filter presented in [2] fails to be near-lossless re- context of the DPT (Drone Protect Task) at MediaEval versible unlike scrambling [4]. Nevertheless, scrambling fails Benchmark 2015. Our proposed filter protects privacy by to recognize events easily because of the amount of noise. visually replacing sensitive RofI (Regions of Interest) by its Our proposed filter conceals privacy information and keep shapes. A combination of steganography and scrambling is the comprehensibility of the video in order to detect events. used in order to make this filter. Once the scrambling is applied on the pixels of the RofI, its MSB (Most Signifi- 2. STEGOSCRAMBLING FILTER cant Bit) are hidden in the LSB (Least Significant Bit) of a RofI (e.g. people, vehicles or accessories bounding boxes) cover image. Our filter fulfils four criteria defined by DPT: are previously annotated in the database [2] used for DPT [1]. near-lossless reversibility, intelligibility, appropriateness and To hide information, an XOR is computed between the six anonymization. We benchmarked the filter on the last three MSBs of the RofI and the random numbers generated with criteria and we get good results: 40 % for intelligibility and a PRG (pseudorandom generator) controlled by a seed, as appropriateness, and 60 % for anonymization. expressed in the equation 1. 1. INTRODUCTION XORImg(i) = Rof I(i) ⊕ RandN ums(i), ∀i (1) Due to the growing of video surveillance systems and the significant improvement of automatic recognition tools, pri- with i the bit position and each bit ∈ {0, 1}. vacy protection techniques became a necessity. Moreover, In parallel, cover images are computed to replace RofI and these systems benefit from image sensors progress (e.g. peo- keep the possibility to recognize events. An edge detector ple are recognized far away from the camera). and a Kmeans clustering [3] (limited to two clusters, similar Here examples of already existing systems protecting pri- to a binarization) are applied in the RGB space of the RofI vacy: pixelization, blurring or black masking with FacePix- containing people. An AND is computed between the edges elizer 1 on Google plus, ObscuraCam 2 on Android, and also of the RofI and the resulting clusters of the images on each scrambling in JPEG compression with Scrambling JPEG pixel, by multiplying them as shown in the equation 2. tool 3 . Besides, people are working on methods to hide iden- tity such as morphing [5], warping [5] and scrambling [6], CoverImg = EdgeImg. ∗ KmeansClustering, (2) but they are complex and the degradation they apply on the images prevent any usage for security purpose (lack of with .* the Element-by-element multiplication. intelligibility). The convex hull image from the binary cover image for The use-case scenario designed for the challenge was Car people is generated in order to become the RofI containing Park Security. The goal was the creation of privacy filtering people. RofI containing car or accessories use(s) only K- solutions for drone videos related to public safety. They are means clustering as cover image. evaluated by following four criteria: Next, the 2 MSBs of the cover image, where the pixels i) protection of privacy, intensity is either 192 or 0, are inserted in the 2 MSBs of ii) intelligibility for the visual quality in order to recognize the resulting image. Finally, the 6-bit of the XOR image, events on the result (i.e. people walking, running, fighting, where pixels intensity is between 0 and 63, are integrated in stealing...), the LSB of the resulting image as shown in the equation 3. iii) appropriateness to see if the result is good looking, Therefore, only cover images are visible by viewers in order iv) possibility to reverse to come back to the original im- to recognize events. 1 http://www.facepixelizer.com/ 5 7 2 X X https://guardianproject.info/apps/obscuracam/ P rotectedImg = XORImg(i)∗2i + CoverImg(i)∗2i , 3 http://ltslinux18.epfl.ch/scramble/ i=0 i=6 (3) Figure 1 illustrates the workflow of the proposed method Copyright is held by the author/owner(s). and Figure 2 shows an example of an entire privacy image. MediaEval 2015 Workshop, September 14-15, 2015, Wurzen, Germany ePassphrase H RofI HH HH  Edge pixel, b0 +e 1 1 b0 7 b0 6 b0 5 b0 4 b0 3 b0 2 H No-edge pixel, b0 +e 0 0 b0 7 b0 6 b0 5 b0 4 b0 3 b0 2 eSeed To recover the original pixel, an XOR is computed between 6 MSB of the RofI Cover image in 2-Bit the same random number than previously, and the six LSBs of the protected pixel. Recovered pixel b7 b6 b5 b4 b3 b2 X X Random 6-Bit ⊕XOR The process is mostly reversible because the two LSBs, de- 2-Bit = 2 MSB noted b0 and b1 are lost. 6-Bit XOR image 3. EVALUATION RESULTS We tested our proposed filter on different video sequences 6-Bit = 6 LSB from DronesProtect dataset [2]. The guidelines of the Medi- aEvel 2015 DroneProtect Tasks [1] are followed to perform Privacy Image the evaluation. This evaluation is based on the human- perceived and interpretation of the resulting privacy filtered videos in terms of level of privacy, intelligibility and appro- Figure 1: Workflow of the proposed process priateness. The aim of the challenge is to find a trade-off between pri- vacy and visual quality of the protected image. Indeed, the higher is the protection, the lower is the level of information (see intelligibility and appropriateness). Two human evaluator groups are selected. In the first group, people come from surveillance security domain (R & D), and in the second group they come from any other domain (Naive). In Table 1, we report the average results of our filter. We obtained positive feedbacks from the jury and especially for the privacy protection. Indeed, according to the results 60 % of privacy is well protected. However, we got 40 % for Figure 2: Illustration of the proposed filter intelligibility and appropriateness; this shows a lack in our filter for recognizing events properly. This can be explained because the edges detector makes mistakes and also colors of RofI are turned to black and white. It is planned as fu- To recover the original RofI, the inverse process is applied ture work to improve the edges detection method with a as shown in the equation 4. Two LSBs are removed from the new design for the cover image, in order to be better tai- original RofI, thus, a maximum error rate of 3 may be pro- lored to release more information and having a better event duced between an original pixel and a recovered pixel. This recognition. error implies no impact for human vision and is negligible for machines. Table 1: Average results (%) 7 X Evaluation Privacy Intelligibility Pleasantness Recovered = (P rotectedImg(i − 2) ⊕ RandN ums(i)) ∗ 2i Category 1 (R&D) 0.63 0.37 0.36 i=2 Category 2 (Naive) 0.57 0.43 0.48 (4) Average (%) 0.6 0.4 0.4 2.1 Pixel example One pixel is considered with 8-bit from MSB to LSB. Original pixel b7 b6 b5 b4 b3 b2 b1 b0 4. CONCLUSIONS We presented a new privacy filter applied on videos in a For each pixel of the RofI, only the MSB bits between 2 car park from drone. The novelty of the work is to com- and 7, are preserved. An XOR is computed between the bine a scrambling to encrypt privacy-sensitive RofI, and a MSB of the original pixel and a random number. The result steganography to hide this scambled RofI in a cover image is denoted b0 . represented by its edges. XORpixel, b0 b0 7 b0 6 b0 5 b0 4 b0 3 b0 2 X X The bits of b0 are shifted in the six LSBs. XORpixel, b0 X X b0 7 b0 6 b0 5 b0 4 b0 3 b0 2 5. REFERENCES [1] A.Badii, P.Koshunov, H.Oudi, T.Ebrahimi, T.Piatrik, Two MSBs of a white and black pixel are represented by V.Eiselein, N.Ruchaud, C.Fedorczak, JL.Dugelay, and e6 = 1 and e7 = 1 for the former, and e6 = 0 and e7 = 0 for D. Vazquez. Overview of the mediaeval 2015 drone the latter. Finally, the two MSBs of e are added with the protect task. In MediaEval 2015 Workshop, Wurzen, six LSBs of b0 . The result is denoted, protected pixel. Germany, Sept, 2015. [2] M. Bonetto, P. Korshunov, G. Ramponi, and T. Ebrahimi. Privacy in mini-drone based video surveillance. In Workshop on De-identification for privacy protection in multimedia, number EPFL-CONF-206109, 2015. [3] H. Fradi, Y. Yan, and J.-L. Dugelay. Privacy protection filter using shape and color cues. In MediaEval 2014 Workshop, Barcelona, Spain, October, 2014. [4] J. Hu and F. Han. A pixel-based scrambling scheme for digital medical images protection. Journal of Network and Computer Applications, 32(4):788–794, 2009. [5] P. Korshunov and T. Ebrahimi. Towards optimal distortion-based visual privacy filters. In IEEE International Conference on Image Processing, number EPFL-CONF-197087, 2014. [6] A. Melle and J.-L. Dugelay. Scrambling faces for privacy protection using background self-similarities. In Image Processing (ICIP), 2014 IEEE International Conference on, pages 6046–6050. IEEE, 2014.