In the paper, the image series forgery detection algorithm based on the analysis of camera pattern noise is proposed. Distribution characteristics of the camera pattern noise are obtained by extracting the noise component of images from the non-tampered image series. A noise residual of a forgery image is compared with the camera pattern noise. We compare various noise filtering algorithms to choose the one that achieves the best performance of the proposed method. The proposed algorithm is tested both on examples of copy-move forgeries and forgery fragments which were inserted from an image not included in the image series.
Image time series describes a scene dynamic. Analysis of image series allows predicting an image that may be next in the image series, as well as to conclude the authenticity of the image. There are several approaches to detect forged images. These approaches can use temporal and spatial correlations [1], unique artifacts of compression, and, finally, unique artifacts left by the camera. Methods using temporal and spatial correlations are divided into two categories. The methods belonging to the first category are based on pixel analysis of images [2-5] while the methods from the second category use the object level of images [6].
In the conditions of availability of many graphic editors and ease of their use, even an ordinary user does not require specialized knowledge and skills to falsify images. Forgeries can be made to add a new object to the scene captured by the camera or to hide the existing ones. Image series forgery detection has its distinctive features as compared to images matching since each image of an image series captures a scene at different moments. Two neighboring images of an image series can be captured under different lighting, weather or seasonal conditions. This paper proposes a forgery detection algorithm that is invariant to the conditions for obtaining images of a series.
This work consists of three parts. The first part deals with the model of the camera sensor noise and presents a method for extracting pattern noise. In the second part of the work, an algorithm for image forgery detection is proposed. The third part contains an experimental result of the proposed algorithm effectiveness. Experiments are focused on copy-move detection (fragments duplicated within one image) and copy-paste detection (fragments inserted from an image not included in the image series). 2. Model of a camera’s sensor noise When the camera sensor captures a uniformly lit scene, the output image will contain a certain number of pixels, slightly different in brightness from the rest. This fact is related to random noise components, such as readout noise (the magnitude of the matrix signal fluctuations relative to the average signal value) or shot noise (random fluctuations of voltages and currents relative to their average value), and a deterministic component - pattern noise. Pattern noise is present in each image, fixed by the sensor, and remains approximately the same for different images captured by the sensor.
The output image of the camera can be represented as follows [7]:
y(i, j) fi, j x(i, j) (i, j) c(i, j) (i, j) (
A F noise filter is used [8], [9] to extract the high-frequency component of camera noise. For each image of the sequence Ii (n, m) , it is possible to define the pattern noise matrix Wi (n, m) as follows:
Wi (n, m) Ii (n, m) F Ii (n, m) .
Estimation of pattern noise matrices W1(n, m),W2 (n, m),...,WL (n, m) set can be performed using a
matrix of per-element expected values and a matrix of per-element dispersion values. These matrices
can be calculated using (
E{W0 ,W1,...,WL} E(n, m) 1 L1
Wi (n, m) .
L i0 D{W0 ,W1,...,WL} D(n, m) 1 L1 Wi (n, m) E(n, m)2 .
L i0
Expected values and dispersion are calculated for every pixel of every dimension.
(
The main requirement for a noise filter is the high quality of filtering areas around the edges of
objects. This requirement is imposed so that the noise matrices contain the least amount of scene
traces. The median filter, the Lee filter [10], the Gauss filter, the non-local mean filter [11] and the
bilateral filter were chosen in the work.
3. Forgery detection algorithm
After obtaining pattern noise distribution characteristics of the camera, the noise component of the
suspicious image distortion is extracted. Let IF (n, m) be a suspicious image that captures the same
scene with the same camera. The image is not included in the image series used in (
WF (n, m) IF (n, m) F IF (n, m) . (
Pattern noise distribution characteristics of the camera are obtained in the way described in the
previous part of this work.
3.1. Calculation of similarity between the noise of suspicious image and camera pattern noise
The pattern noise matrix of the suspicious image WF (n, m), n 1, N, m 1, m is calculated using the
formula (
Set of Mahalanobis distance dM (wi, j ,μi, j ) calculated for every wi, j forms a Mahalanobis distance matrix DM WF ,E . Next, the matrix DM is averaged in the window whose size does not exceed the size of the forged region to carry off peak values caused by random noise. 3.2. Creating a binary mask based on the distance matrix The task of creating a binary mask is solved by choosing a threshold and threshold processing on the Mahalanobis distance matrix.
The threshold is selected based on the analysis of the Mahalanobis distance matrices total histogram. The cumulative histogram is created by aggregating Mahalanobis distance matrices histograms of authentic images Ii (n, m),i 1, L .
Neyman-Pearson criterion is used to select a value of the threshold T . The probability of a falsepositive p0 is fixed, and the value of the threshold T is chosen to minimize the probability of a falsenegative p1 . 3.3. Binary mask post-processing In the work, post-processing of a binary mask includes selecting connected regions on the mask [12] and filtering them by size. A connected area is considered forged if its square exceeds 1 / 1000 of the original image square. A minimal convex hull is constructed around each of forged region. Then the space inside the minimal convex hulls is filled. 4. Experiments The experiments were carried out on a standard PC (Intel Core i5-4460, 16 GB RAM).
Ten image time series were used as the object of experiments. Every image series includes 15 authentic images and two forged images. All images were represented in the RGB space. One forged image included a copy-move and a fragment of another image was inserted into the second image. All images had a size of 4032 3024 .
Figure 1 illustrates the camera pattern noise extracted by: (a) - the median filter, (b) - the Lee filter, (c) - the Gauss filter, (d) - the non-local mean filter and (e) - the bilateral filter. The images of camera pattern noise have been converted to grayscale and transformed by linear enhancement to the range of [0, 255] . 1 2 3 4 5 6 7 8 9 10 # d) e) Figure 1. Camera pattern noise extracted by: a - the median filter; b - the Lee filter; c - the Gauss filter; d - the non-local mean filter and e - the bilateral filter.
The first part of the experiments was aimed at determining the effectiveness of the copy-move detection by the proposed algorithm. 4.1. Effectiveness of copy-move detection The results of experiments aimed at detecting copy-move fragments are shown in Table 1. An example of an image containing copy-move, as well as the result of detecting it using a bilateral filter, is shown in Figures 2 (a) and 2 (b), respectively.
b)
Figure 2. The result of copy-move detection in the image.
The results of experiments aimed at detecting copy-paste fragments are shown in Table 2. An example of an image containing copy-paste, as well as the result of detecting it using a bilateral filter, is shown in Figure 3 (a) and 3 (b), respectively. 5. Conclusion The image series forgery detection algorithm based on the camera pattern noise analysis has been proposed in the paper. The conducted research has allowed determining the most suitable noise filter in the sense of the selected metric F1 - bilateral filter. Also, experiments have shown the Lee filter is not suitable for solving the problem of copy-move and copy-paste fragments detection. The proposed algorithm allows detecting copy-move fragments with the average F1 value of 0.85 if the bilateral noise filter was used for pattern noise extraction. The average F1 value of 0.87 is reached for copypaste fragments detection with the bilateral noise filter also.
This work was supported by RFBR according to the research project № 18-01-00748-а in part of
"Introduction" and (