=Paper=
{{Paper
|id=Vol-2210/paper25
|storemode=property
|title=Detecting forgery in image time series based on anomaly detection
|pdfUrl=https://ceur-ws.org/Vol-2210/paper25.pdf
|volume=Vol-2210
|authors=Nadezhda Evdokimova,Vladislav Myasnikov
}}
==Detecting forgery in image time series based on anomaly detection==
https://ceur-ws.org/Vol-2210/paper25.pdf
Detecting forgery in image time series based on anomaly
detection
N I Evdokimova1 and V V Myasnikov1,2
1
Samara National Research University, Moskovskoe shosse 34, Samara, Russia, 443086
2
Image Processing Systems Institute - Branch of the Federal Scientific Research Centre
“Crystallography and Photonics” of Russian Academy of Sciences, Molodogvardeyskaya str.
151, Samara, Russia, 443001
Abstract. Increasing complexity of image forgery methods is an actual problem nowadays. This
problem rises due to the expansion of fields that use digital images in their work. Image time
series show the dynamics of the scene and allow it to be compared over time. This paper
proposes a new algorithm for detecting forgeries of the single digital image in an image time
series described a scene. This algorithm uses analysis of errors set that were computed during
reconstruction of the analyzed image using other images of series. The first part of the paper
describes the proposed algorithm consisted of three stages. The second part of the paper
describes forgery detection using morphological image filtering based on guided contrasting.
The third part of the paper contains comparison of considered algorithms and investigation
results of intra-image copy-move and inter-image copy-move detection.
1. Introduction
Image time series describe dynamic of an scene. Analysis of an image time series lets modeling
an image that can be next in the image time series. Also, it allows deciding authenticity of
the image. There are several approaches to image forgery detection. These approaches may
use unique artifacts left by the camera, unique artifacts arising after compression and, finally,
temporal and spatial correlations [1]. Methods that used temporal and spatial correlation can
divide into two categories. Techniques from the first category based on analysis of images pixel
data [5], [6], [7] whereas methods from the second category use object level of images [8].
Forgeries may be created to add a new object to the scene or to hide any existing. Image time
series forgery detection has its distinctive features compared to image matching. Every image
of an image time series is obtained at different moments of time. Two adjacent images of an
image time series can be captured under different conditions of illumination, weather or seasonal
conditions. In this paper, the algorithm invariant to the conditions for obtaining images of the
series is proposed.
The proposed algorithm uses a correlation between corresponding fragments of neighboring
images in the series. In this paper, the concept of anomaly applies for image series forgery
detection. In the global sense, an anomaly is a fragment of data that does not correspond to the
precisely defined concept of normal behavior [2]. In the sense of this paper, all fragments that
are marked as anomaly are considered forgeries.
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018)
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
This work consists of three parts. In the first part, an algorithm for image time series forgery
detection based on the anomalies detection are presented. An algorithm of forgery detection
using morphological image filtering based on guided contrasting [3] is given in the second part.
The third part contains the comparison of both algorithms presented in the first part and the
second part.
2. Forgery detection based on the anomalies detection
Let there is an image time series It (n1 , n2 ), t - image number in time series (t = 0, T , T ≥ 1).
Every image has the same size N1 × N2 (n1 ∈ [0, N1 ], n2 ∈ [0, N2 ]) and captures the same scene
at different moments of time.
For definiteness, it is assumed that the image I0 (n1 , n2 ) is checked for forgeries although it
may be located in the image time series anywhere. The fragments It (m1 , m2 ) of all images are
analyzed in the sliding window D(n1 , n2 ) ⊆ 0, N1 − 1 × 0, N2 − 1, (m1 , m2 ) ∈ D(n1 , n2 ).
2.1. Image fragment description
The fragments It (m1 , m2 ) are described in k steps, k = 2p , p > 2.
On the k = 1 step, the fragment I0 (m1 , m2 ) is reconstructed by linear combination of
corresponding fragments I1 (m1 , m2 ), ...IT (m1 , m2 ) for all possible positions of sliding window
D:
T
X
I0 (m1 , m2 ) ≈ αt It (m1 , m2 ) (1)
t=1
using mean squared deviation ε21 minimization:
T
1 X X
ε21 = (I0 (m1 , m2 ) − αt It (m1 , m2 ))2 → min . (2)
|D| (m ,m )∈D t=1
α1 ,...,αT
1 2
The last action on this step is calculating both types of errors the mean squared deviation and
the normalized mean squared deviation that defined by:
ε2
ε̃21 = N −1 N −11 . (3)
1
X 2
X
I0 (i, j)2
i=0 j=0
On the k = 2p step, every fragment It (m1 , m2 ), t = 1, T corresponding to window D location
is splitted into k fragments using k-means clusterization (by brightness) as shown in the Figure
1. New fragments that was constructed after clusterization can be denoted by Itj (n1 , n2 ),
j = 0, k − 1. Then fragment I0 (m1 , m2 ) is reconstructed by linear combination of fragments
Itj (m1 , m2 ):
T k−1
αtj Itj
X X
I0 ≈ (4)
t=1 j=0
using mean squared deviation ε2k minimization:
2
1
ε2k ∼ αtj Itj (m1 , m2 ) →
X X X
= I0 (m1 , m2 ) − min . (5)
|D| (m ,m )∈D 1≤t≤T 0≤j≤k−1 α01 ,...,αk−1
1 ,...,α0T ,...,αk−1
T
1 2
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 185
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
Then we calculate both types of errors the mean squared deviation and the normalized mean
squared deviation similar to (3):
ε2
ε̃2k = N −1 N −1k . (6)
1
X 2
X
2
I0 (i, j)
i=0 j=0
In the paper, this procedure performs for k = 4, 8, 16, so there is a set of three mean squared
3 5 12 3 5 0 0 0 0 0 0 12 0 0 0
1 14 2 = 1 0 2 + 0 0 0 + 0 14 0 + 0 0 0
4 9 17 4 0 0 0 9 0 0 0 0 0 0 17
Figure 1. Splitting of the fragment It(m1, m2) into k = 22 fragments.
deviation values and three normalized mean squared deviation values for every position of the
window D. Calculated values presents as x̄(n1 , n2 ) as follows:
T
x̄(n1 , n2 ) ≡ ε̃24 (n1 , n2 ), ε̃28 (n1 , n2 ), ε̃216 (n1 , n2 ) (7)
to form the image fragments feature vector. Mean squared deviation values ε2k are not
deliberately taken into account because they are directly used in the calculation of ε̃2k .
2.2. Statistic construction method
The obtained vectors x̄(n1 , n2 ) set represents in the coordinate system ε̃24 ε̃28 ε̃216 . This set locates
in the three-dimensional cube with sides equal to 1 as shown in Figure 2:
ε~162
1
1
1 ε~82
ε~42
Figure 2. The set of feature vectors in the coordinate system ε̃24ε̃28ε̃216.
2.3. Anomalies determination
There are no absolute static objects on images obtained in real conditions. This is due both to
noises of real cameras and the image compression on the path from the camera to the processing
system. It often leads to additional system distortions. Moreover, a scene may contain objects
that have specific dynamic characteristics although they are static in the global sense. For
example, it may be trees swaying in the wind.
As described above, it can be concluded that it is impossible to obtain a feature vector with
coordinates (0; 0; 0) after authentic image fragment representation by described above method.
It lets define a rule for assigning fragments corresponding to feature vectors (0; 0; 0) to anomalies.
This type of anomalies refers to fragments that were copied from one or several images of the
image time series.
On the other hand, the errors ε̃24 , ε̃28 , ε̃216 of an authentic fragment representation must have
values that do not exceed a certain threshold. Feature vectors not corresponded to this condition
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 186
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
are considered anomalies. This type of anomalies refers to fragments that were copied from the
same image or image not included in the image time series.
It is obvious the error value of the same fragment representation decreases with the clusters
number increasing. Therefore, it is justified to use different thresholds for ε̃24 , ε̃28 and ε̃216 . The
following relation should meet:
Tε̃2 ≥ Tε̃2 ≥ Tε̃2 (8)
4 8 16
Threshold values (8) are defined by analysis the distribution histograms as shown in Figure
(3). The first local histogram minimum is selected and considered a threshold value. So,
following threshold values were chosen for histograms shown in the Figure 3:
• Tε̃2 = 1.5 × 10−8 ,
4
• Tε̃2 = 1.5 × 10−8 ,
8
• Tε̃2 = 0.9 × 10−8 .
16
a b
c
Figure 3. Selecting the thresholds according to the distribution histogram: a) for ε̃24 , b) for ε̃28 ,
for ε̃216.
The cube with the feature vectors x̄(n1, n2) set (2) is divided into three areas:
1) Origin of the coordinate system;
2) A parallelepiped that is adjacent to the origin;
3) Rest area of the cube.
Per the above, feature vectors from the first area correspond to fragments that were copied
from one or several images of the image time series. Feature vectors from the second area refer
to authentic image regions. Feature vectors from the third area correspond to fragments that
were copied within one image or from an image not included in the image time series. This
splitting is shown in Figure 4.
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 187
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
ε~162
1
3
2
T16
1
T8
T4 1
1 ε~82
ε~42
Figure 4. Splitting the cube with set of vectors x̄(n1, n2) into three areas.
After extraction of feature vectors from the relevant area and labeling them as suspicious, the
corresponded binary mask is created. Then the mask is processed with a noise filter that removes
regions with square less than some value. After this, only feature vectors that correspond to
forgery regions are kept in the set of suspicious feature vectors.
3. Forgery detection using morphological image filtering based on guided
contrasting
Morphological image filtering technique based on guided contrasting was proposed in [3]. This
filtering technique makes available detecting changing between two images.
An image forgery detection algorithm based on guided contrasting can be performed in two
stages:
1. Background normalization based on guided contrasting;
2. Image forgery detection using normalized background image processing.
3.1. Background normalization based on guided contrasting
Let f - standard image and g - test image. Proposed in [3] background normalization algorithm
gives an opportunity to perform background normalization of the image g with considering
the shape of the image f . The procedure of background normalization is performed using the
window D(x, y). The procedure is applied to the pyramid of images with a constant size of the
window D(x, y) to ensure invariance to the window D(x, y) size.
Background normalization based on guided contrasting is as follows:
1. Construction of the pyramid representation f t = (f 0 , ...f t−1 ) and g t = (g 0 , ...g t−1 ) where
i−1 i−1
f 0 = f, g 0 = g and size(f i ) = size(f2 ) , size(g i ) = size(g2 ) , i = 1, t − 1.
2. Calculation of the filter (9) response φi (f i , g i )(x, y) for every pyramid level:
D(x,y) D(x,y)
φ(f, g)(x, y) = g0 (x, y)+ | K(f D(x,y) , g D(x,y) ) | (g(x, y) − g0 (x, y)), (9)
D(x,y)
where g D(x,y) (u, v) = g(x, y), if (u, v) ∈ D(x, y); 0, otherwise, g0 (x, y) = mean(g D(x,y) (x, y))
and K(f D(x,y) ,g D(x,y) ) is the local normalized correlation coefficient defined by (10).
D(x,y) D(x,y)
(f D(x,y) − f0 , g D(x,y) − g0 )
K(f, g) = D(x,y) D(x,y)
(10)
k f D(x,y) − f0 kk g D(x,y) − g0 k
3. Calculation of absolute difference ∆gfi between g i and corresponding filter response
φi (f i , g i )(x, y) for i = 1, t − 1.
4. Reconstruction of a difference image from the pyramid. It is performed from level t − 1
with averaging on every level by following:
∆mif (x, y) = {gfi (x, y), if i = t − 1; max(gfi (x, y), hi+1
f (x, y)), otherwise} (11)
where hif - twofold spatial increased image gfi .
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 188
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
Image m0f is the normalized background of the image g with considering the shape of the
image f .
3.2. Image forgery detection using normalized background image processing
Image g forgery detection performs by normalized background image m0f processing. This
processing carries out as following:
1. Image m0f binarization. For this procedure, thresholding an image with an opportunity to
set a threshold by the user was chosen.
2. Morphological filtering of the binary image from step 1 such as erosion and dilation in the
square window 3 × 3.
3. Segmentation and enumeration of all non-zero fragments.
4. Computation of an minimal convex hull of every suspicious segment and fill it.
5. Calculation of the local morphological correlation coefficient (MCC) KM n [4] for every
suspicious segment by formula (12) and comparison it with a threshold:
n k gFn χn (x, y) k
KM = (12)
kgk
where n is the number of an analyzed suspicious fragment, χn is the indicator function that has
value ”1” for pixels from analyzed suspicious fragment and ”0” otherwise and gFn defined by:
(g, χn )
gFn = (13)
k χn k2
6. Creating global mask with all forgery fragments which correspond to segments with MCC
greater than the threshold.
4. Experiments
The experiments were carried out on a desktop PC with Intel Core i5-4460 processor and 16
GB RAM.
Five image time series were obtained using the same camera. The camera was still all the time.
It has captured the scene and token image every 10 sec. As result of this procedure, there are five
image time series with six images in every series. Obtained images have 920 × 1380 dimension.
Next, every image has been transformed to gray-scale. These time series were chosen as the
objects of experiments.
Series consisted of six images were used for image forgery detection through the algorithm
based on anomaly detection. The algorithm based on guided contrasting used series from two
images that are first and last.
Copy-move various type embedding procedure was developed for forgeries generation.
Experiments of two type forgeries detection were carried out:
1. Copy-move within one image - intra-image copy-move.
2. Copy-move from another image of the image time series - inter-image copy-move.
4.1. Intra-image copy-move detection
The experiment results with duplicate taken from the same image are shown in the tables 1 and
2. Example of detection using these algorithms is shown in the Figure 5.
As shown in the tables 1 and 2, both algorithms give about the same results of F1 ( 0.66 is
mean F1 for the algorithm based on anomalies detection, and 0.59 is mean F1 for the algorithm
based on guided contracting). However, these algorithms reach these values on account of
different components as so as Precision and Recall. So, the algorithm based on anomalies
detection has high values of Recall. It means a greater portion of forgery pixels are detected
using the algorithm based on anomaly detection than using the algorithm based on guided
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 189
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
Table 1. Intra-image copy-move detection using the algorithm based on anomalies detection.
Series Precision Recall F1
1 0.55 0.84 0.66
2 0.53 0.74 0.62
3 0.62 0.85 0.72
4 0.55 0.68 0.61
5 0.63 0.78 0.70
Table 2. Intra-image copy-move detection using the algorithm based on guided contrasting.
Series Precision Recall F1
1 0.93 0.40 0.56
2 0.94 0.67 0.78
3 0.38 0.03 0.06
4 0.50 1 0.68
5 0.86 0.93 0.89
a b
c
Figure 5. Example of intra-image forgery detection: a - forgery image; b - result of detection
using the algorithm based on anomalies detection; c - result of detection using the algorithm
based on guided contrasting.
contrasting. On another hand, the algorithm based on guided contrasting has high values of
Precision. It means this algorithm has less false detection than the algorithm based on anomalies
detection.
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 190
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
4.2. Inter-image copy-move detection
The experiment results that were carried out on all image series with duplicate taken from
another image of the image series are shown in the tables 3 and 4. Example of detection using
these algorithms is shown in the Figure 6.
Table 3. Inter-image copy-move detection using algorithm based on anomalies detection.
Series Precision Recall F1
1 1 0.84 0.91
2 1 0.95 0.97
3 0.58 0.74 0.65
4 0.8 0.82 0.8
5 0.87 0.84 0.85
Table 4. Inter-image copy-move detection using algorithm based on guided contrasting.
Series Precision Recall F1
1 0. 0. 0.
2 0.75 0.004 0.009
3 0. 0. 0.
4 0.16 0.06 0.09
5 0 0 -
As shown in the tables 3 and 4, the algorithm based on guided contrasting doesn’t give
opportunity detecting inter-image copy-move forgeries while the algorithm based on anomalies
detection does it and give high values of F1.
5. Conclusion
The algorithm for image time series forgery detection based on anomaly detection was
proposed in this paper. Also, comparison of the proposed algorithm and the algorithm for
image forgery detection based on guided contrasting carried out. Experiments showed that
both algorithms have about the same quality of detection intra-image copy-move in the sense
of metric F1 (0.66 and 0.59 respectively). On another hand, experiments let to conclude that
the algorithm of image forgery detection based on guided contrasting doesn’t give opportunity
detecting inter-image copy-move, unlike proposed algorithm.
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 191
Image Processing and Earth Remote Sensing
N I Evdokimova and V V Myasnikov
a b
c
Figure 6. Example of inter-image forgery detection: a - forgery image; b - result of detection
using the algorithm based on anomalies detection; c - result of detection using the algorithm
based on guided contrasting.
6. References
[1] Christian A and Sheth R 2016 Digital video forgery detection and authentication technique -
a review International Journal of Scientific Research in Science and Technology 2(6) 138-143
[2] Chandola V, Banerjee A and Kumar V 2009 Anomaly detection: a survey ACM
Computing Surveys 41(3) 51-58
[3] Rubis A Yu, Lebedev M A, Vizilter Yu V and Vygolov O V 2016 Morphological image
filtering based on guided contrasting Computer Optics 40(1) 73-79 DOI:
10.18287/2412-6179-2016-40-1-73-79
[4] Pyt’ev Yu P and Chulichkov A I 2010 Methods of Morphological Image Analysis
(Moscow: Fizmatlit)
[5] Evdokimova N I and Kuznetsov A V 2017 Local patterns in the copy-move detection
problem solution Computer Optics 41(1) 79-87 DOI: 10.18287/2412-6179-2017-41-1-79-87
[6] Kuznetsov A V and Myasnikov V V 2016 A copy-move detection algorithm based on
binary gradient contours Computer Optics 40(2) 284-293 DOI:
10.18287/2412-6179-2016-40-2-284-293
[7] Kuznetsov A V and Myasnikov V V 2014 A fast plain copy-move detection algorithm based
on structural pattern and 2D rabin-karp rolling hash Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics) 8814 461-468
[8] Hussain M, Chen D, Cheng A, Wei H, Stanley D 2013 Change detection from re-motely
sensed images: from pixel-based to object-based approaches ISPRS Journal of Photogrammetry and
Remote Sensing 80 91-106
Acknowledgments
The reported study was funded by RFBR according to the research project 17-29-03190, research
project 18-01-00748 and by the Federal Agency of scientific organization (Agreement 007-
Γ3/43363/26).
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 192