This paper describes privacy protection method based on a false coloring approach for Drone Protect Task of MediaEval 2015. The aim is to obscure regions of a video that are privacy sensitive without sacri cing intelligibility and pleasantness. False coloring transforms the original colors of pixels using a color palette into a di erent set of colors in which private information is harder to recognize. The method can be applied globally to an entire frame of the video or to a speci c region of interest (ROI). The privacy protected output is expected to remain pleasant, and when needed, a close approximation of the original input can be recovered. Benchmarking evaluations on the mini-drone dataset show promising results, especially, for intelligibility and pleasantness criteria.
Video surveillance systems are being widely used to
protect the safety of public and private perimeters. An ideal
surveillance system should balance well between two
objectives: e ciently execute a security task (intelligibility ) and
carefully preserve subjects' privacy (privacy). The most
commonly used methods to protect privacy such as
blurring, masking, and pixelization do not achieve a good
balance. For this reason, second generation solutions such as
scrambling [
Furthermore, most methods strongly rely on e cient computer vision algorithms for instance when regions that require privacy protection must be automatically detected (e.g., faces, license plates, etc.). However, computer vision algorithms are known to fail at times. If a sensitive region is missed, even in a single frame, it will severely compromise privacy. Therefore, there is a need to develop robust and effective algorithms for privacy protection that can e ciently cope with situations when computer vision algorithms fail.
We propose to protect privacy via false coloring, which does not rely on computer vision and can be applied either on an entire frame or a region of interest. It is simple to Currently with Idiap research institute (Switzerland) 2.
The main idea in false color based privacy protection is in
transforming colors of pixels in a frame such that the
private information becomes unrecognizable while the impact
on intelligibility is kept as small as possible. Previous work
on false coloring has demonstrated the applicability of such
an approach for privacy protection against both human
observers and automatic face recognition algorithms [
This algorithm rst converts a color frame into grayscale. The pixel intensities of the grayscaled frame are then used as keys to a look-up a table that represents a color palette. Optionally, the grayscale frame can be compressed or quantized to further distort the visual information prior to table lookup. The pixel values of the original frame are then replaced by the values from the table. This algorithm can be applied on an entire frame or on one or more ROIs. The strength of the protection is controlled by the color distribution of the selected color palette (Figure 1).
The protected frames can be reversed to obtain a close approximation of the originals by performing an inverse table look-up. However, due to the initial grayscale conversion, the recovered frames will be in grayscale. Also, if the lookup table contains duplicate values, full recovery may not be possible due to the initial many-to-one mapping. Finally, the reversion is only possible if one knows the properties of the color map used during protection. Thus, security can be enhanced by utilizing a custom color palette.
We applied false coloring to the annotated ROIs of the
provided mini-drone dataset [
A sample result is shown in Figure 2, where (a) represents an original video frame, (c) its privacy protected version, and (e) its recovered version. The close-up views can be observed in the bottom row. It can be noted in (c) and (d) that the face of the individual is represented in the DEF color scale, whereas his body is represented in the RBS color scale. The vehicle, on the other hand, is represented in the LOCS color scale based on expert annotations.
The recovered ROIs shown in (e) and (f) do not contain color information and have some artifacts near the ROI boundaries. This is due to the compression of the protected frames. As the compression works at block rather than pixel level, the false colored pixels a ect the colors of the neighboring pixels and are not corrected during the inverse look-up.
The MediaEval benchmarking results reported in Table 1 show that our intelligibility and pleasantness scores are above the average of all submissions whereas the privacy level score is below the average. This can be explained by the fact that false coloring is a point operation and, unlike most other methods, it does not introduce structural distortions. Nevertheless, privacy level could be improved by using custom color palettes that are better tailored to privacy protection.
Intelligibility FC AVG 0.76 0.59 0.75 0.58 0.755 0.59
Pleasantness FC AVG 0.73 0.60 0.75 0.61 0.74 0.60 4.
In this paper, we described a simple and e ective method for protecting privacy using false coloring. Benchmarking evaluations indicated high preference for the pleasantness and intelligibility of this method, whereas it was found to be less e ective for preserving privacy. Future work will investigate designing custom color scales to improve privacy protection and the quality of reversibility while enhancing security. 5.
The work was conducted in the framework of FP7 NoE VideoSense and TUBITAK project number 114E445.