=Paper=
{{Paper
|id=Vol-2419/paper25
|storemode=property
|title=On the Susceptibility of Deep Neural Networks to Natural Perturbations
|pdfUrl=https://ceur-ws.org/Vol-2419/paper_25.pdf
|volume=Vol-2419
|authors=Mesut Ozdag,Sunny Raj,Steven L. Fernandes,Alvaro Velasquez,Laura Pullum,Sumit Kumar Jha
|dblpUrl=https://dblp.org/rec/conf/ijcai/OzdagRFVPJ19
}}
==On the Susceptibility of Deep Neural Networks to Natural Perturbations==
https://ceur-ws.org/Vol-2419/paper_25.pdf
On the Susceptibility of Deep Neural Networks to Natural Perturbations
Mesut Ozdag1 , Sunny Raj1 , Steven Fernandes1 , Alvaro Velasquez2 , Laura L.
Pullum3 and Sumit Kumar Jha1
1
University of Central Florida, Orlando, FL
2
Air Force Research Laboratory, Rome, NY
3
Oak Ridge National Laboratory, Knoxville, TN
{ozdag*, sraj, steven, jha}@cs.ucf.edu , alvaro.velasquez.1@us.af.mil , pullumll@ornl.gov
Abstract
Deep learning systems are increasingly being
adopted for safety critical tasks such as autonomous
driving. These systems can be exposed to adverse
weather conditions such as fog, rain and snow. Vul-
nerability of deep learning systems to synthetic ad-
versarial attacks has been extensively studied and
demonstrated, but the impact of natural weather
conditions on these systems has not been studied
in detail. In this paper, we study the effects of fog (a) Original image is correctly classified as a minivan by
on classification accuracy of the popular Inception the Inception model.
deep learning model. We use stereo images from
the Cityscapes dataset and computer graphics tech-
niques to mimic realistic naturally occurring fog.
We show that the Inception deep learning model is
vulnerable to the addition of fog in images.
1 Introduction
Deep learning models demonstrate great success in vari-
ous pattern recognition applications and image classification
problems. With recent advancements in high-performance
graphical processing units and the availability of a large num- (b) Image with fog is incorrectly classified as a fountain by
ber of labelled images, deep learning networks have become the Inception model.
even better at image recognition tasks than an average person.
Despite these outstanding success stories, it has been re- Figure 1: The addition of fog to an image causes the Inception
peatedly shown that deep learning networks produce incor- model to incorrectly classify images that would be correctly
classified by a human user.
rect responses when the input is perturbed by small but in-
telligently crafted “adversarial” changes. For example, such
adversarial images can easily cause even state-of-the-art deep
learning networks to erroneously classify the images [1–4]. the dataset are not assigned to a specific class; instead, the
In many cases, the modifications to the input images are so output of the deep neural network is arbitrarily wrong.
small that the original images are nearly indistinguishable Adversarial attacks can also be classified based on the
from the adversarial images to an average human eye. Ad- number of times an input is analyzed during the crafting of
versarial inputs pose a real challenge to the successful adop- the adversarial input. One-time attacks utilize only a single
tion of deep learning in safety-critical applications. Adver- access to the inputs to create the adversarial images. Iterative
sarial attacks on deep learning networks can affect fingerprint attacks require multiple accesses to the input image as they
and face recognition tasks, as well as cause errors in speech create and refine the adversarial images. Perturbations used to
recognition systems, and other applications. generate adversarial images can be broadly classified as dig-
Adversarial images can be used to generate targeted attacks ital and physical. Digital attacks are based on modification
or non-targeted attacks. Targeted attacks misguide the deep of the input image in the memory of a computer that may or
learning networks to produce responses from a specific a pri- may not correspond to an image in the real world, while phys-
ori determined class. In non-targeted attacks, all images in ical attacks are based on images that can be acquired from the
physical world. In this paper, we create non-targeted, itera-
tive, and physical attacks.
Our results show that the addition of synthetically gener-
ated fog to real-world images causes deep learning networks
to incorrectly classify images. Unlike adversarial images, our
inputs are not crafted maliciously by choosing careful ran-
dom perturbations. Instead, our inputs are merely generated
using the synthetic addition of fog; hence, such images can
be expected to occur in the real world. Our results are a small
but essential step towards demonstrating the need to design
more robust machine learning systems for safety-critical ap- (a) Image with fog incorrectly classified as aircraft by
plications. Inception model with tFactor=0.15, atmLight=0.6 and
PSNR=9.44.
2 Related Work
Digital perturbations can be classified as individually-tailored
or universal. Individually-tailored perturbations generate
different perturbations for each of the input images in a
dataset [1–10]. Szegedy et al. [11] was the first to intro-
duce individually-tailored perturbations against deep learn-
ing networks in 2014. The adversarial images were gener-
ated using the L-BFGS method which uses binary search to
obtain an optimal input. The L-BFGS attack was an expen-
sive and time-consuming approach to find an adversarial in-
put. Goodfellow et al. [12] proposed the fast gradient sign
method (FGSM). This method performed only a one-step up- (b) Image with fog incorrectly classified as scooter by
Inception model with tFactor=0.07, atmLight=0.6 and
date of the gradient. Rozsa et al. [2] analyzed FGSM and then PSNR=10.77.
proposed a new approach, called fast gradient value method.
It was obtained by replacing the sign of the gradients with the
raw value of the gradients.
Many recent attacks employ individually-tailored pertur-
bations. However, universal perturbations are easier to de-
ploy. They are image-agnostic as they generate a single per-
turbation for all the images in the dataset [13–17]. Moosavi-
Dezfooli et al. [13] showed that universal perturbations can
be generalized across different image classification models.
This results in image-agnostic and network-agnostic pertur-
bations. The existence of such general perturbations has
been explained by considering the correlation between dif-
ferent image regions of the decision boundary. Mopuri et (c) Image with fog incorrectly classified as submarine by
al. [14] proposed universal perturbations which are quasi- Inception model with tFactor=0.12, atmLight=0.8 and
imperceptible to humans but capable of attacking convolu- PSNR=6.74.
tional neural networks. This approach is able to attack multi-
ple images from the same target dataset across multiple deep
learning networks.
Physical perturbations are generated using real-world ob-
jects such as eye glasses or printed stickers that cause an in-
correct classification in deep learning models [18–20]. Ku-
rakin et al. [18] attacked neural networks by applying adver-
sarial images to the physical world by extending FGSM. They
made small changes for multiple iterations and for each iter-
ation, the pixel values were clipped to avoid a large change
on each pixel. Sharif et al. [19] presented the method of gen-
erating eyeglass frames, which when worn and printed can (d) Image with fog incorrectly classified as submarine by
attack a state-of-the-art deep learning system for face recog- Inception model with tFactor=0.12, atmLight=1 and
nition. The perturbations generated are inconspicuous to a PSNR=4.36.
human and can be physically acquired via photography in the
real world. Lu et al. [20] empirically showed that adversar- Figure 2: Images from Cityscapes dataset are incorrectly classified
ial perturbations can cause a deep learning network to incor- upon the synthetic addition of fog.
rectly detect a stop sign using physical perturbations when the
captured image is taken from a specified range. However, the
physical perturbations presented in [18–20] are not naturally-
occurring perturbations, and require the participation of a
malicious agent. In addition, the latest state-of-the-art ap-
proach for fog simulation on real scenes was proposed re-
cently by Dai et al. [21]. They used scene semantic annotation
as an additional input to their dual-reference cross-bilateral
filter on the Cityscapes dataset to obtain Foggy Cityscapes-
DBF (Dual-reference cross-Bilateral Filter). They also used
a CNN-based approach to estimate fog density.
In this paper, we propose natural attacks using visibly (a) Original left image classified as traffic light by Inception
foggy images to generate input that causes incorrect classi- model.
fication by the Inception deep learning model [22]. Apart
from an earlier preliminary work on attacking computer vi-
sion algorithms using fog generated via the Perlin noise on
two-dimensional images [23], this is the first attempt to at-
tack deep learning classifiers using natural perturbations on
stereo images that include depth information and can hence
be used to model realistic naturally-occurring fog. As shown
in Figure 1 and Figure 2, our approach of adding fog to im-
ages can cause deep neural networks to incorrectly classify
input images.
3 Our Approach (b) Original right image that forms a stereo pair along with left
image.
We use images obtained from the Cityscapes [24] dataset and
added fog to attack the Inception deep learning model. The
Cityscapes dataset contains 25,000 stereo images with 30 var-
ied visual theme categories, such as road, sidewalk, person,
rider, car, bus, building, bridge, traffic sign, and traffic light.
Each stereo image is a pair of images captured from two dif-
ferent cameras. These pairs of images are denoted as left and
right images. We use these pairs of images to create a depth
mapping of objects in the image. Then, we use the depth in-
formation of the objects in the images to synthetically add fog
to these images; the presence of depth information allows the
synthetically-generated fog to resemble naturally occurring
fog in the image.
(c) Disparity image showing the distance of objects from the
The typical aim of an adversarial attack test is to add some observer. Objects closer to the observer appear to be brighter
natural perturbation (e.g. fog, sunlight, visual environmental and objects further away from the observer appear to be darker.
changes and aberrations, etc.) over an input image in order
for the deep learning model to misclassify the image. How-
ever, it is still correctly recognized and identified by a reg-
ular human visual-eye observer. To corroborate our claims,
in this paper we proceed to generate a conventional, outside
fog environment as a naturally-occurring, subtle climate per-
turbation, in order to provide this foggy image as a qualified
difficult adversarial attack against the most advanced, novel
deep learning models to date including Inception.
First, we run the Inception model for an autonomous driv-
ing potential application using the clear weather images in
our dataset. Here, we seek to obtain accurate image recog-
nition decision results. Then we apply generated visual fog
conditions onto said baseline images from this dataset, us- (d) Image with fog incorrectly classified as scooter by
ing specific stereo-pair images and disparity mapping tech- Inception model with tFactor=0.07 and atmLight=0.6.
niques. Once this counterintuitive, adversarial image is pro-
Figure 3: Illustration of our approach for synthetically adding fog
duced, Peak Signal-to-Noise Ratio (PSNR) value disparities to stereo images from the Cityscapes dataset.
between our initial clear weather images and their corre-
Data: Left Image L, Right Image R, Thickness factor Original Image tF & atmL Perturbed Class PSNR
tFactor, Atmospheric Light atmLight. berlin000 0.12 & 1.00 park bench 10.30
Result: Image F with synthetically added fog. berlin009 0.10 & 1.00 parking meter 12.26
begin berlin012 0.10 & 1.00 fountain 9.63
F = L /* Initialize to left image */ berlin015 0.10 & 0.80 bubble 15.44
DN = stereoSGBM (R, L) /* Calculate berlin027 0.07 & 0.80 fountain 17.22
noisy disparity image using stereo berlin070 0.07 & 0.80 stage 17.93
semi-global matching and mutual berlin072 0.10 & 0.80 bubble 17.20
information */ berlin090 0.10 & 1.00 washbasin 10.19
D = filter (DN, R, L) /* Filter noisy berlin144 0.10 & 1.00 parking meter 11.44
disparity image to generate smooth berlin151 0.10 & 1.00 parking meter 11.92
disparity image */ berlin154 0.10 & 0.60 spotlight 21.35
for each pixel index i in F do berlin155 0.12 & 0.60 bullet 20.97
−tFactor
t = e D[i] /* Compute transmission berlin160 0.12 & 0.60 spotlight 19.81
intensity */ berlin164 0.15 & 1.00 fountain 8.59
F [i] = tF [i] + (1 − t)atmLight berlin172 0.10 & 1.00 mailbox 9.92
berlin180 0.15 & 1.00 ship 8.73
end
berlin182 0.12 & 0.80 stage 14.51
return F /* Return foggy image */
berlin183 0.15 & 1.00 locomotive 9.12
end berlin352 0.10 & 1.00 spotlight 12.16
Algorithm 1: Algorithm to add fog to a stereo image pair. berlin437 0.15 & 1.00 parking meter 9.31
Table 1: Images from Cityscapes dataset that are classified as car by
sponding foggy images are observed.
the Inception model and their corresponding foggy image we found
3.1 Fog Generation as adversarial. We add fog on the original left image with the param-
eters tF (tFactor) and atmL (atmLight) to obtain an incorrect class
We used a variant of stereo processing by semi-global block using the Inception model.
matching and mutual information (Stereo SGBM) imple-
mented in the popular OpenCV toolkit to calculate the depth
of every pixel in the image. This depth information is called 3.2 Impact on Deep Learning System
the disparity of the image. Additional depth mapping infor- We test the robustness of the Inception deep learning model
mation is available in the Cityscapes dataset [24] but was not on the synthetic images with fog generated by our method.
precise enough to generate smooth natural fog. We use the We use left images from the stereo image pair for classifica-
depth value of each pixel to mimic realistic fog. A higher tion purposes. We run Inception classification on the origi-
depth value indicates that the object is further away from the nal left image and note the classification label. We then gen-
observer and is less visible. An object that has a lower depth erate a foggy image and run Inception classification on the
value is closer to the observer and is not affected adversely by foggy image and note the new classification label. If the origi-
fog. nal classification is different from the classification generated
Besides the depth of a pixel, our synthetically-generated from the foggy image, we have exposed a potential safety er-
fog includes two additional parameters: the fog thickness ror in the deep learning classifier.
(tFactor) and the ambient atmospheric light (atmLight). The An ideal test of the robustness of the deep learning sys-
thickness parameter tFactor determines the intensity of fog; tem will have foggy images that look similar to the original
a thicker fog can occlude objects that are closer to the ob- image. We measure the similarity between the original and
server. The atmospheric light atmLight parameter determines the foggy image using the peak signal to noise ratio (PSNR)
the color and intensity of ambient light; we used white light value. PSNR value can be calculated using Equation 1, where
of varying intensity for our fog. A lower value of atmLight D is the maximum possible pixel value of the image and
leads to fog that is darker in color and a higher value of atm- RMSE is the root mean squared error calculated between the
Light leads to a fog that is brighter. original and the foggy image.
Steps to generate foggy images are presented in Algo-
rithm 1. This algorithm takes as input a stereo image pair: left
image (L) and right image (R), thickness factor (tFactor) and D
atmospheric light (atmLight). Fog density and other parame- PSNR = 20 log10 (1)
RMSE
ters for disparity computation are all combined into a single
parameter (tFactor) referring to the fog thickness. An exam- High PSNR values indicate a greater similarity between the
ple of right, left, disparity and final foggy images is shown original and foggy image. In Figure 2, we show images with
in Figure 3. Disparity images are stored in such a way that varying PSNR values. We observe that images with more vis-
objects closer to the observer are brighter and objects fur- ible fog have a lower PSNR value indicating lower similarity
ther away from the observer are darker. Examples of fog for between the original and foggy images. In general, fog gen-
various values of tFactor and atmLight values are shown in erated with higher tFactor and atmLight values have lower
Figure 2. PSNR values. We generate multiple foggy images by varying
the values for tFactor and atmLight. Then, we run classifi-
Original Image tF & atmL Perturbed Class PSNR cation on these images and select the image with the highest
berlin014 0.10 & 0.80 spotlight 15.41 PSNR value that is able to fool the deep learning system. In
berlin016 0.10 & 0.80 fountain 15.29 Figure 2, Image b has the highest PSNR among all generated
berlin032 0.10 & 1.00 parking meter 19.69 foggy images that is incorrectly classified by the Inception
berlin039 0.10 & 1.00 wing 10.30 deep learning model.
berlin048 0.15 & 1.00 minivan 12.46 In our experiments, we aim to attack a deep learning
berlin063 0.15 & 1.00 spotlight 17.93 model, Inception, by adding fog using our fog generator (Al-
berlin094 0.10 & 1.00 groom 12.58 gorithm 1) on the Cityscapes dataset. Table 1, Table 2, and
berlin123 0.10 & 1.00 wing 9.66 Table 3 demonstrate the PSNR value between an original left
berlin147 0.10 & 1.00 spotlight 11.10 image and its corresponding foggy image that we find as ad-
berlin153 0.10 & 1.00 vault 9.64 versarial. First, we run the Inception model on the Cityscapes
berlin156 0.10 & 1.00 umbrella 11.92 images and we classify them based on their labels (e.g., car,
berlin159 0.10 & 1.00 stage 11.22 traffic light, bike). Second, we find the largest PSNR value
berlin161 0.12 & 0.80 spotlight 14.31 between the original left image and foggy image that has a
berlin162 0.12 & 0.80 aircraft carrier 14.13 different label from the original one classified by Inception.
berlin226 0.10 & 1.00 parking meter 11.80 Lastly, the model returns the label of the adversarial image
berlin268 0.10 & 1.00 locomotive 11.79 with the corresponding tFactor and atmLight values.
berlin298 0.10 & 1.00 wing 10.54 From our experiments, one may conclude that:
berlin327 0.10 & 0.80 stage 16.06
berlin358 0.12 & 1.00 fountain 10.34 • The bounded PSNR value for the car images is found
berlin430 0.12 & 1.00 parking meter 9.42 to be from 8.59 to 21.35. The adversarial foggy images
berlin483 0.10 & 1.00 tow truck 11.01 of cars are observed to be classified as different labels
berlin484 0.10 & 1.00 locomotive 11.51 (e.g., park bench, parking meter, fountain, stage, bubble,
washbasin).
Table 2: Images from Cityscapes dataset that are classified as traffic • The bounded PSNR value is also observed to be from
light by Inception model and their corresponding foggy image we 9.42 to 19.69 for the traffic light images. The generated
found as adversary. We add fog on the original left image with the adversarial foggy images on traffic light have different
parameters tF (tFactor) and atmL (atmLight) to obtain incorrect class
using Inception model.
labels, such as spotlight, wing, fountain, umbrella, loco-
motive and parking meter.
• The bounded PSNR value varies from 8.60 to 20.76 for
Original Image tF & atmL Perturbed Class PSNR the bike images. The adversarial images on bike are la-
beled as lakeshore, scuba diver, aircraft carrier, bubble,
berlin079 0.15 & 1.00 lakeshore 8.60
fountain, maze, spotlight, washbasin, wing, and subma-
berlin100 0.15 & 1.00 scuba diver 9.59
rine.
berlin176 0.10 & 0.80 submarine 15.53
berlin202 0.10 & 1.00 chair 11.12 • Overall, we see that the decision boundary between the
berlin206 0.10 & 0.60 aircraft carrier 18.04 clear weather images and their corresponding foggy ad-
berlin216 0.10 & 1.00 scuba diver 9.94 versarial images to vary from 8.59 to 21.35 PSNR.
berlin300 0.10 & 1.00 bubble 10.26 • It may also be observed that the minimum tFactor and
berlin302 0.15 & 1.00 spark bench 9.80 atmLight values that result in an adversarial foggy image
berlin303 0.15 & 1.00 wing 9.53 are 0.07 and 0.60, respectively.
berlin306 0.10 & 1.00 bubble 11.16
berlin316 0.15 & 1.00 aircraft carrier 9.85 • It may also be seen that the maximum PSNR values that
berlin326 0.10 & 0.80 fountain 15.41 are found are considerably close to each other for the
berlin336 0.12 & 1.00 maze 11.08 same labels of adversarial images. For example, the
berlin359 0.10 & 1.00 parking meter 11.43 maximum PSNR values for almost all the adversarial im-
berlin367 0.10 & 0.80 fountain 16.55 ages that are labeled as parking meter vary from 9.24 to
berlin384 0.12 & 0.80 spotlight 15.59 12.26.
berlin404 0.15 & 1.00 parking meter 9.24 • These perturbed classes crucially affect the decision
berlin408 0.12 & 0.60 aircraft carrier 20.76 mechanism of any system that works with deep learn-
berlin412 0.12 & 0.60 bubble 20.73 ing classifiers.
berlin417 0.15 & 1.00 washbasin 9.85
4 Conclusion and Future Work
Table 3: Images from Cityscapes dataset that are classified as bike
by Inception model and their corresponding foggy image we found We used computer graphics techniques to generate natural
as adversary. We add fog on the original left image with the parame- fog effects in Cityscapes stereo images, and observe that
ters tF (tFactor) and atmL (atmLight) to obtain incorrect class using these images with synthetically-generated fog are able to fool
Inception model. the current state-of-the-art deep learning system, Inception.
Hence, existing deep learning systems are vulnerable not only
to digital and physical adversarial attacks, but they produce [13] S. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and
incorrect answers even when faced with benign naturally oc- P. Frossard, “Universal adversarial perturbations,”
curring perturbations. Several interesting directions for fu- CoRR, vol. abs/1610.08401, 2016.
ture work remain open. First, we want to explore the ef- [14] K. R. Mopuri, U. Garg, and R. V. Babu, “Fast feature
fects of other naturally occurring conditions such as rain, hail fool: A data independent approach to universal adver-
and snow on deep learning image classification systems. Sec- sarial perturbations,” CoRR, vol. abs/1707.05572, 2017.
ond, we will test the robustness of systems designed specif-
ically for outdoor functionality, such as autonomous driving [15] S. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, P. Frossard,
systems. Third, we will explore the design of defense algo- and S. Soatto, “Analysis of universal adversarial pertur-
rithms that can permit deep neural networks to reason cor- bations,” CoRR, vol. abs/1705.09554, 2017.
rectly about images with fog and other natural perturbations. [16] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer,
“Adversarial patch,” CoRR, vol. abs/1712.09665, 2017.
References [17] J. Hendrik Metzen, M. Chaithanya Kumar, T. Brox,
[1] J. Tarel, N. Hautiere, L. Caraffa, A. Cord, H. Halmaoui, and V. Fischer, “Universal Adversarial Perturbations
and D. Gruyer, “Vision enhancement in homogeneous Against Semantic Image Segmentation,” arXiv e-prints,
and heterogeneous fog,” IEEE Intelligent Transporta- p. arXiv:1704.05712, Apr 2017.
tion Systems Magazine, vol. 4, pp. 6–20, Summer 2012. [18] A. Kurakin, I. J. Goodfellow, and S. Bengio, “Ad-
[2] A. Rozsa, E. M. Rudd, and T. E. Boult, “Adver- versarial examples in the physical world,” CoRR,
sarial diversity and hard positive generation,” CoRR, vol. abs/1607.02533, 2016.
vol. abs/1605.01775, 2016. [19] M. Sharif, S. Bhagavatula, L. Bauer, and M. Reiter, “Ac-
[3] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining cessorize to a crime: Real and stealthy attacks on state-
and Harnessing Adversarial Examples,” arXiv e-prints, of-the-art face recognition,” pp. 1528–1540, 10 2016.
p. arXiv:1412.6572, Dec 2014. [20] J. Lu, H. Sibai, E. Fabry, and D. A. Forsyth, “NO need
[4] N. Papernot, P. D. McDaniel, S. Jha, M. Fredrik- to worry about adversarial examples in object detection
son, Z. B. Celik, and A. Swami, “The limita- in autonomous vehicles,” CoRR, vol. abs/1707.03501,
tions of deep learning in adversarial settings,” CoRR, 2017.
vol. abs/1511.07528, 2015. [21] D. Dai, C. Sakaridis, S. Hecker, and L. V. Gool,
[5] P.-Y. Chen, H. Zhang, Y. Sharma, J. Yi, and C.-J. Hsieh, “Curriculum model adaptation with synthetic and real
“ZOO: Zeroth Order Optimization based Black-box At- data for semantic foggy scene understanding,” CoRR,
tacks to Deep Neural Networks without Training Substi- vol. abs/1901.01415, 2019.
tute Models,” arXiv e-prints, p. arXiv:1708.03999, Aug [22] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and
2017. Z. Wojna, “Rethinking the inception architecture for
[6] J. Su, D. V. Vargas, and K. Sakurai, “One pixel computer vision,” CoRR, vol. abs/1512.00567, 2015.
attack for fooling deep neural networks,” CoRR, [23] A. Ramanathan, L. Pullum, Z. Husein, S. Raj, N. Toros-
vol. abs/1710.08864, 2017.
dagli, S. Pattanaik, and S. K. Jha, “Adversarial attacks
[7] Y. Liu, X. Chen, C. Liu, and D. Song, “Delving on computer vision algorithms using natural perturba-
into transferable adversarial examples and black-box at- tions,” in 2017 Tenth International Conference on Con-
tacks,” CoRR, vol. abs/1611.02770, 2016. temporary Computing (IC3), pp. 1–6, Aug 2017.
[8] N. Carlini, G. Katz, C. Barrett, and D. L. Dill, [24] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. En-
“Ground-truth adversarial examples,” CoRR, zweiler, R. Benenson, U. Franke, S. Roth, and
vol. abs/1709.10207, 2017. B. Schiele, “The cityscapes dataset for semantic ur-
[9] A. M. Nguyen, J. Yosinski, and J. Clune, “Deep neural ban scene understanding,” CoRR, vol. abs/1604.01685,
networks are easily fooled: High confidence predictions 2016.
for unrecognizable images,” CoRR, vol. abs/1412.1897,
2014.
[10] S. Sabour, Y. Cao, F. Faghri, and D. J. Fleet, “Ad-
versarial manipulation of deep representations,” CoRR,
vol. abs/1511.05122, 2015.
[11] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Er-
han, I. J. Goodfellow, and R. Fergus, “Intriguing prop-
erties of neural networks,” CoRR, vol. abs/1312.6199,
2013.
[12] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining
and Harnessing Adversarial Examples,” arXiv e-prints,
p. arXiv:1412.6572, Dec 2014.