=Paper=
{{Paper
|id=Vol-2391/paper35
|storemode=property
|title=The image series forgery detection algorithm based on the camera pattern noise analysis
|pdfUrl=https://ceur-ws.org/Vol-2391/paper35.pdf
|volume=Vol-2391
|authors=Nadezhda Evdokimova,Vladislav Myasnikov
}}
==The image series forgery detection algorithm based on the camera pattern noise analysis ==
https://ceur-ws.org/Vol-2391/paper35.pdf
The image series forgery detection algorithm based on the
camera pattern noise analysis
N I Evdokimova1 and V V Myasnikov1,2
1
Samara National Research University, Moskovskoe Shosse 34А, Samara, Russia, 443086
2
Image Processing Systems Institute of RAS - Branch of the FSRC "Crystallography and
Photonics" RAS, Molodogvardejskaya street 151, Samara, Russia, 443001
e-mail: nadezh.evdokimova@gmail.com, vmyas@geosamara.ru
Abstract. In the paper, the image series forgery detection algorithm based on the analysis of
camera pattern noise is proposed. Distribution characteristics of the camera pattern noise are
obtained by extracting the noise component of images from the non-tampered image series. A
noise residual of a forgery image is compared with the camera pattern noise. We compare
various noise filtering algorithms to choose the one that achieves the best performance of the
proposed method. The proposed algorithm is tested both on examples of copy-move forgeries
and forgery fragments which were inserted from an image not included in the image series.
1. Introduction
Image time series describes a scene dynamic. Analysis of image series allows predicting an image that
may be next in the image series, as well as to conclude the authenticity of the image. There are several
approaches to detect forged images. These approaches can use temporal and spatial correlations [1],
unique artifacts of compression, and, finally, unique artifacts left by the camera. Methods using
temporal and spatial correlations are divided into two categories. The methods belonging to the first
category are based on pixel analysis of images [2-5] while the methods from the second category use
the object level of images [6].
In the conditions of availability of many graphic editors and ease of their use, even an ordinary user
does not require specialized knowledge and skills to falsify images. Forgeries can be made to add a
new object to the scene captured by the camera or to hide the existing ones. Image series forgery
detection has its distinctive features as compared to images matching since each image of an image
series captures a scene at different moments. Two neighboring images of an image series can be
captured under different lighting, weather or seasonal conditions. This paper proposes a forgery
detection algorithm that is invariant to the conditions for obtaining images of a series.
This work consists of three parts. The first part deals with the model of the camera sensor noise and
presents a method for extracting pattern noise. In the second part of the work, an algorithm for image
forgery detection is proposed. The third part contains an experimental result of the proposed algorithm
effectiveness. Experiments are focused on copy-move detection (fragments duplicated within one
image) and copy-paste detection (fragments inserted from an image not included in the image series).
V International Conference on "Information Technology and Nanotechnology" (ITNT-2018)
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
2. Model of a camera’s sensor noise
When the camera sensor captures a uniformly lit scene, the output image will contain a certain number
of pixels, slightly different in brightness from the rest. This fact is related to random noise
components, such as readout noise (the magnitude of the matrix signal fluctuations relative to the
average signal value) or shot noise (random fluctuations of voltages and currents relative to their
average value), and a deterministic component - pattern noise. Pattern noise is present in each image,
fixed by the sensor, and remains approximately the same for different images captured by the sensor.
The output image of the camera can be represented as follows [7]:
y(i, j ) fi , j x(i, j ) (i, j ) c(i, j) (i, j) (1)
where (i, j ) is shot noise, (i, j ) is readout noise, с с(i, j ) is fixed pattern noise (FPN),
x x(i, j ) is image of the scene in the absence of any noise and f i , j is a multiplicative coefficient
characterizing photo-response nonuniformity (PRNU).
2.1. Extraction of sensor pattern noise
To reduce the contribution of the random noise (i, j ) and (i, j ) to the determined noise
component, an image time series Ii (n, m), i 1, L, n 1, N , m 1, M , of the same scene captured by the
same camera is used.
A F noise filter is used [8], [9] to extract the high-frequency component of camera noise. For each
image of the sequence Ii (n, m) , it is possible to define the pattern noise matrix Wi (n, m) as follows:
Wi (n, m) Ii (n, m) F I i (n, m) . (2)
Estimation of pattern noise matrices W1 (n, m),W2 (n, m),...,WL (n, m) set can be performed using a
matrix of per-element expected values and a matrix of per-element dispersion values. These matrices
can be calculated using (3) and (4), respectively. Both the matrix of expected values and the matrix of
dispersion values have the same dimensions and depth as the pattern noise matrix Wi (n, m) and the
original images Ii (n, m) accordingly.
1 L 1
E{W0 ,W1 ,...,WL } E (n, m) Wi (n, m) . (3)
L i 0
1 L 1
D{W0 ,W1 ,...,WL } D(n, m) Wi (n, m) E (n, m) .
2
(4)
L i 0
Expected values and dispersion are calculated for every pixel of every dimension.
2.2. Selection of a noise extraction filter
The main requirement for a noise filter is the high quality of filtering areas around the edges of
objects. This requirement is imposed so that the noise matrices contain the least amount of scene
traces. The median filter, the Lee filter [10], the Gauss filter, the non-local mean filter [11] and the
bilateral filter were chosen in the work.
3. Forgery detection algorithm
After obtaining pattern noise distribution characteristics of the camera, the noise component of the
suspicious image distortion is extracted. Let I F (n, m) be a suspicious image that captures the same
scene with the same camera. The image is not included in the image series used in (3) and (4). The
pattern noise matrix of the suspicious image is determined as follows:
WF (n, m) I F (n, m) F I F (n, m) . (5)
The image forgery detection algorithm can be introduced as follows:
Obtaining pattern noise distribution characteristics of the camera using an image series;
Obtaining the pattern noise matrix of the suspicious image. It is needed to use the same noise
filter with the same parameters;
V International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 259
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
Evaluation of similarity between the pattern noise of the suspicious image and the pattern
noise of the camera;
Creating of binary mask and post-processing of it.
Pattern noise distribution characteristics of the camera are obtained in the way described in the
previous part of this work.
3.1. Calculation of similarity between the noise of suspicious image and camera pattern noise
The pattern noise matrix of the suspicious image WF (n, m), n 1, N , m 1, m is calculated using the
formula (2). If images of the image series Ii (n, m), i 1, L, have three channels, then every element of
the pattern noise matrix can be presented as a vector wi , j wijR , wijG , wijB , i 1, N , j 1, M . The
T
similarity between the noise of the suspicious image and camera pattern noise distribution is
characterized by the Mahalanobis distance. The Mahalanobis distance is calculated for every element
w i , j of pattern noise matrix and the corresponding element μi , j E (i, j ) ijR , ijG , ijB of expected
T
values matrix using (6).
w μ B w μ ,
T 1
d M ( w i , j , μi , j ) i, j i, j ij i, j i, j (6)
where B ij is a covariance matrix.
Set of Mahalanobis distance d M (wi , j , μi , j ) calculated for every w i , j forms a Mahalanobis distance
matrix DM WF , E . Next, the matrix DM is averaged in the window whose size does not exceed the
size of the forged region to carry off peak values caused by random noise.
3.2. Creating a binary mask based on the distance matrix
The task of creating a binary mask is solved by choosing a threshold and threshold processing on the
Mahalanobis distance matrix.
The threshold is selected based on the analysis of the Mahalanobis distance matrices total
histogram. The cumulative histogram is created by aggregating Mahalanobis distance matrices
histograms of authentic images Ii (n, m), i 1, L .
Neyman-Pearson criterion is used to select a value of the threshold T . The probability of a false-
positive p0 is fixed, and the value of the threshold T is chosen to minimize the probability of a false-
negative p1 .
3.3. Binary mask post-processing
In the work, post-processing of a binary mask includes selecting connected regions on the mask [12]
and filtering them by size. A connected area is considered forged if its square exceeds 1/ 1000 of the
original image square. A minimal convex hull is constructed around each of forged region. Then the
space inside the minimal convex hulls is filled.
4. Experiments
The experiments were carried out on a standard PC (Intel Core i5-4460, 16 GB RAM).
Ten image time series were used as the object of experiments. Every image series includes 15
authentic images and two forged images. All images were represented in the RGB space. One forged
image included a copy-move and a fragment of another image was inserted into the second image. All
images had a size of 4032 3024 .
Figure 1 illustrates the camera pattern noise extracted by: (a) - the median filter, (b) - the Lee filter,
(c) - the Gauss filter, (d) - the non-local mean filter and (e) - the bilateral filter. The images of camera
pattern noise have been converted to grayscale and transformed by linear enhancement to the range of
[0,255] .
V International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 260
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
a) b) c)
d) e)
Figure 1. Camera pattern noise extracted by: a - the median filter; b - the Lee filter; c - the Gauss
filter; d - the non-local mean filter and e - the bilateral filter.
The first part of the experiments was aimed at determining the effectiveness of the copy-move
detection by the proposed algorithm.
4.1. Effectiveness of copy-move detection
The results of experiments aimed at detecting copy-move fragments are shown in Table 1. An example
of an image containing copy-move, as well as the result of detecting it using a bilateral filter, is shown
in Figures 2 (a) and 2 (b), respectively.
Table 1. F1 metric value of copy-move detection.
# Median filter Lee filter Gauss filter Non-local Bilateral filter
mean filter
1 0.24 0.03 0.56 0.46 0.87
2 0.36 - 0.43 0.64 0.93
3 0.29 - 0.37 0.25 0.77
4 0.44 0.02 0.42 0.33 0.69
5 0.38 0.01 0.30 0.15 0.86
6 0.42 - 0.14 0.14 0.87
7 - 0.03 0.19 0.31 0.74
8 0.49 - 0.27 0.21 0.94
9 - - 0.07 0.07 0.92
10 0.53 0.01 0.86 0.00 0.92
a) b)
Figure 2. The result of copy-move detection in the image.
V International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 261
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
4.2. Effectiveness of copy-paste detection
The results of experiments aimed at detecting copy-paste fragments are shown in Table 2. An example
of an image containing copy-paste, as well as the result of detecting it using a bilateral filter, is shown
in Figure 3 (a) and 3 (b), respectively.
Table 2. F1 metric value of copy-paste detection.
# Median filter Lee filter Gauss filter Non-local Bilateral filter
mean filter
1 0.25 - 0.30 0.21 0.95
2 0.44 - 0.43 0.15 0.96
3 - - 0.54 0.28 0.87
4 0.37 - 0.36 0.13 0.54
5 0.22 - 0.27 0.17 0.92
6 - - 0.01 0.01 0.95
7 0.21 - 0.26 0.30 0.91
8 0.13 - 0.47 - 0.86
9 0.81 - 0.83 0.21 0.93
10 - - 0.39 0.17 0.83
a) b)
Figure 3. The result of copy-paste detection in the image.
5. Conclusion
The image series forgery detection algorithm based on the camera pattern noise analysis has been
proposed in the paper. The conducted research has allowed determining the most suitable noise filter
in the sense of the selected metric F1 - bilateral filter. Also, experiments have shown the Lee filter is
not suitable for solving the problem of copy-move and copy-paste fragments detection. The proposed
algorithm allows detecting copy-move fragments with the average F1 value of 0.85 if the bilateral
noise filter was used for pattern noise extraction. The average F1 value of 0.87 is reached for copy-
paste fragments detection with the bilateral noise filter also.
6. References
[1] Christian A and Sheth R 2016 Digital Video Forgery Detection and Authentication Technique -
A Review International Journal of Scientific Research in Science and Technology 2 138-143
[2] Evdokimova N I and Kuznetsov A V 2017 Local patterns in the copy-move detection problem
solution Computer Optics 41(1) 79-87 DOI: 10.18287/2412-6179-2017-41-1-79-87
[3] Kuznetsov A V and Myasnikov V V 2016 A copy-move detection algorithm based on binary
gradient contours Computer Optics 40 284-293 DOI: 10.18287/2412-6179-2016-40-2-284-293
[4] Kuznetsov A V and Myasnikov V V 2014 A fast plain copy-move detection algorithm based on
structural pattern and 2D rabin-karp rolling hash Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8814
461-468
V International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 262
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
[5] Evdokimova N I and Myasnikov V V 2018 Detecting forgery in image time series based on
anomaly detection CEUR Workshop Proceedings 2210 184-192
[6] Hussain M, Chen D, Cheng A, Wei H and Stanley D 2013 Change detection from remotely
sensed images: From pixel-based to object-based approaches ISPRS Journal of Photogrammetry
and Remote Sensing 80 91-106
[7] Lukáš J, Fridrich J and Goljan M 2006 Detecting digital image forgeries using sensor pattern
noise Proceedings of SPIE - The International Society for Optical Engineering 6072
[8] Fahmy M F and Fahmy O M 2016 A new morphological based forgery detection scheme
National Radio Science Conference, NRSC, Proceedings 212-216
[9] Chen M, Fridrich J, Goljan M and Lukáš J 2008 Determining image origin and integrity using
sensor noise IEEE Transactions on Information Forensics and Security 3 74-90
[10] Soifer V A, Chernov A V, Chernov V M, Chicheva M A, Fursov V A, Gashnikov M V,
Glumov N I, Ilyasova N Y, Khramov A G and Korepanov A O 2009 Computer Image
Processing (VDM Verlag Dr. Müller)
[11] Buades A, Coll B and Morel J-M 2005 A non-local algorithm for image denoising Proceedings
IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR II 60-
65
[12] Solomon C and Breckon T 2011 Morphological Processing Fundamentals of Digital Image
Processing (John Wiley & Sons, Ltd) 197-234
Acknowledgments
This work was supported by RFBR according to the research project № 18-01-00748-а in part of
"Introduction" and (2) "Model of a camera’s sensor noise" and RF Ministry of Science and Higher
Education within the State assignment to the FSRC «Crystallography and Photonics» RAS
(Agreement 007-Г3/43363/26) in part of (3) "Forgery detection algorithm" - (4) "Experiments".
V International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 263