=Paper=
{{Paper
|id=Vol-2470/p27
|storemode=property
|title=Numerical analysis of SLSSIM similarity on medical X-ray image domain
|pdfUrl=https://ceur-ws.org/Vol-2470/p27.pdf
|volume=Vol-2470
|authors=Jonas Brusokas,Linas Petkevičius
|dblpUrl=https://dblp.org/rec/conf/ivus/BrusokasP19
}}
==Numerical analysis of SLSSIM similarity on medical X-ray image domain==
https://ceur-ws.org/Vol-2470/p27.pdf
Numerical analysis of SLSSIM similarity on
medical X-ray image domain
Jonas Brusokas Linas Petkevičius
Institute of Computer Science Institute of Computer Science
Vilnius University Vilnius University
Vilnius, Lithuania Vilnius, Lithuania
jonas.brusokas@mif.vu.lt linas.petkevicius@mif.vu.lt
Abstract—The X-ray has been adopted and used for various networks for lossless image compression, to make storing
purposes including medical diagnostics. To remove noise created images and using them for calculation more efficient [15].
by new low dose X-ray imaging procedures and reduce medical Furthermore, a significant amount of research has recently
image size, X-ray image reconstruction and lossless compression
using deep neural networks are being researched. To enable this, been conducted on finding ways of improving X-ray imaging
image similarity metrics capable of performing well on X-ray procedures, especially on automated means of disease or
images must be used. In this paper, the requirements for medical abnormality detection [2], [3], [16].
X-ray similarity metrics are defined. A new similarity metric is One of the key factors to the accuracy and effectiveness
proposed taking into account the quality of structures within of a deep neural network is the objective function. In image
different intensity levels. An analysis is given comparing the
proposed and other currently known metrics performance on reconstruction and compression image similarity metrics are
real X-ray images in simulated scenarios. used as objective functions. They compare two images with
Keywords—Image similarity metrics, Low dose X-ray imaging, each other and produce a scalar result denoting their degree
Medical X-ray images of difference [17]. The similarity metric defines what image
properties are being evaluated so the selection of an effective
I. I NTRODUCTION metric is crucial.
The X-ray ever since its inception has been widely adopted In this paper a similarity metric for medical X-ray images
and used for various purposes. One of the key uses of X- domain is proposed and compared with other known metrics.
ray imaging is in medical diagnostics. It allows for a non- In section II essential properties of X-ray images in the
invasive method of diagnosing various bone structure defects, medical domain and requirements for a similarity metric are
infections, arthritis, and most cancers [1]–[3]. defined. An overview of currently used metrics is made in
Despite wide usage and rapid technological advances the section III and the definition of the proposed metric is made
ionizing radiation emitted during X-ray imaging procedures in section IV. The analysis comparing the performance of the
creates real health risks for patients [4]–[6]. Efforts have been metrics is made in section V.
made to reduce the risks associated with radiation exposure by II. R EQUIREMENTS FOR MEDICAL X- RAY IMAGE
creating new procedures for performing X-ray imaging. These SIMILARITY METRICS
procedures (commonly referred to as low-dose) use lower
Medical X-ray images are created and used to enable
voltage or amperage settings to reduce the amount of radiation
trained radiologists to analyse the human body, determine and
emitted thus reducing the risks [7]. Unfortunately, using these
diagnose various irregularities and illnesses. X-ray images can
types of procedures results in images having artifacts and
be done over any part of the human body and as such contain
noises which can reduce diagnostic suitability [8]. There are
various structures including bones, tissue, and organs [1], [2].
cases where it might not be necessary to remove the noises in
Any assessment of X-ray image similarity or quality must take
order to give an accurate diagnosis [9]. In most cases, however,
into account properties of the images which enable them to
it is imperative to remove or reduce the amount of noise on
be used for accurate and reliable diagnosis.
an X-ray image. Conventional approaches that use defined
Through analysis of literature and working with trained
properties of noise distribution do not yield satisfactory results
specialists in the field of radiography several important re-
[10], [11].
quirements for medical X-ray image similarity metrics have
There have been some successful attempts in using deep
been identified:
neural networks for image reconstruction tasks, outperforming
other approaches in removing various noises from images R1 The metric must detect emerging noises and artifacts
[12]–[14]. There have also been attempts in using deep neural from images created with low dose X-ray procedures. As
was previously stated, in most cases to enable low dose
procedures, it is important to detect and remove noises
©2019 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0) from the created images as they hinder diagnosis [8], [13].
91
R2 The metric must detect perceptual geometric distortions. B. Structural similarity index (SSIM)
They can appear on images during imaging, reconstruc- As a response to unsatisfactory perceived visual quality
tion or decompression. From a radiologists perspective results of MSE and similar metrics a new approach was
the distortions may cause difficulties or even render the proposed to construct new similarity metrics. It was based
images completely unusable for diagnostics [18]. on the idea that images are highly structured and rather
R3 The metric must take into account the discrepancies than evaluating pixels individually (like in MSE), groups of
of images within specific ranges of brightness intensity spatially proximate and related pixels should be taken into
levels. Due to the nature of the X-ray, images contain account together in order to achieve more accurate repre-
various human body structures including bones, tissue, sentation of perceivable quality [17]. This is based on the
organs. They are captured as areas of different brightness assumption that the human visual system is adapted to identify
intensities [1]–[3]. In this paper we will refer to them various structures within field of sight and effectively spot any
as sub-levels. Most image quality assessment approaches distortions affecting them [17], [22]. A well-known metric
attempt to evaluate the broader perceptual view of image based on these principles is the Structural Similarity Index
quality and focus on more visible areas. In medical X- (SSIM) [23]. SSIM metric calculates in account three key
ray images, all structures captured in these images have features of the images: luminance changes, contrast changes
significant value and greatly contribute to diagnosis [13]. and structural changes [17]. In this paper it will be defined
Taking into account reconstruction or decompression ac- and used in its standard form:
curacy of not only the whole picture but also structures
(2µX µY + c1 )(2σXY + c2 )
within different intensity ranges is critical. SSIM∗ (X, Y ) = 2 (2)
(µX + µ2Y + c1 )(σX 2 + σ2 + c )
Y 2
The proposed metric (in section IV) and following experi-
ments (in section V) will take these properties into account. SSIM has seen wide usage for many image processing
tasks [23]. It has served as inspiration and basis for many
III. S ELECTED METRICS other similarity metrics supplementing additional features to
the standard metric [24]–[28].
Research into image similarity metrics (in some cases re-
C. Weighted SSIM (wSSIM)
ferred to as image quality assessment) has witnessed attention
and notable progress over the past decades [19]. Image quality Weighted SSIM is a general-purpose modification of the
assessment is essential in fields that use image processing [20]. SSIM metric mentioned in sub-section III-B. It was created to
The majority of metrics used evaluate the similarity of the be used as an objective function in Deep neural networks. It
distorted image use the complete reference image [19]. In this is a composition of SSIM and traditional L1 loss. Intuitively,
paper 3 selected general purpose similarity metrics which are SSIM gives the perceptual image assessment and L1 decreases
potentially usable in the X-ray image domain will be discussed the metric value for more distorted images and increases for
and compared. It is important to note, that for the rest of the less distorted images [29]. The authors claimed that if used
paper metric definitions will be used, where metric value 1 for training, it would put more emphasis on pairs which are
means the compared images are equal, and value 0, means performing worse and increase training speed and accuracy.
they are completely dissimilar. In this paper L1 will be defined in its normalized form and
wSSIM will be defined in the following form:
A. Mean squared error (MSE) L∗1 (X, Y ) = 1 − |X − Y |L1 / (M · N ) (3)
Mean squared error is considered to be one of the most wSSIM ∗ (X, Y ) = 1 − SSIM ∗ · (1 − L∗1 ) (4)
simple and straight-forward similarity metrics. It is computed
by averaging the intensity differences of the two compared Here X and Y are compared images, N - number of pixels
images [17]. MSE is known for being quite mathematically in the images, M - maximal pixel intensity.
convenient for optimization purposes. Although the metric
does not correlate well with perceived visual quality [21]. IV. P ROPOSED SLSSIM METRIC
Despite its flaws it still remains widely used for many image Substantial research regarding using Deep neural networks
processing tasks. For the purposes of this paper, this metrics’ as a method for medical image reconstruction and processing
performance will be compared to others during analysis. The is very recent [30], [31]. Although a significant amount of
following normalized form of the metric will be used: metrics have been developed to suit different domains, there
is no definitive similarity metric for comparing medical X-ray
n
1X images.
M SE ∗ (X, Y ) = 1 − (Xi − Yi )2 / M 2 (1)
n i=1 As mentioned in section II a metric for medical X-ray
image similarity assessment must take into account specific
Here X and Y are compared images, M - maximal pixel properties of the images. Also, it has become clear from image
intensity, Xi and Yi - a concrete pixel in the image, n - total quality measurement research that handcrafting and tailoring
number of pixels in image. a similarity metric by combining several known metrics can
92
Fig. 1. Results with Poisson Gauss noise augmentation (left), additive Gauss noise augmentation (right)
yield significantly better performance than using any single using real X-ray images from openly available medical X-ray
known metric [32], [34]. image data sets. The data sets used for analysis were:
In this paper a similarity metric for assessing medical X- 1) RSNA pediatric image set created by the Radiological
ray image similarity is proposed. It is called the sub-level society of North America, which contains hand X-ray
structural similarity index (or SLSSIM). As its name implies images [35];
it is based on the widely used structural similarity index (or 2) NIH image set created by the National Institute of Health
SSIM). Its main differentiating feature is the consideration of (of the United States of America) which contains chest
human body structures appearing within different sub-levels X-ray images [36];
of the image (as mentioned in requirement R3). It is achieved 3) MURA musculoskeletal image set created by Stanford
by calculating SSIM values on discrete sub-levels and not the University which contains elbow, forearm, hand and
entire image. This should enable more strict comparison of shoulder X-ray images [37];
the structures appearing within the sub-levels. In this paper
For the analysis 3000 images were selected from the data sets
sub-levels are mathematically defined as:
(1000 from each) in a random order. All medical images within
h iM 0 (i + 1)M 0 the data sets were gray-scale, with pixel value ranging within
X (i) , Y (i) ∈ , (5)
k k the interval [0; 255].
Image augmentations have been used to simulate noises
Here X and Y are images, X (i) , Y (i) are sub-levels of the
and distortions in the X-ray images to simulate real-life
images, N - number of pixels in image, M 0 - amount of
medical X-ray imaging scenarios. To analyse performance
different values that a pixel can have, k - is the number of
values of metrics were calculated on image pairs that contained
compared discrete sub-levels. The mathematical definition of
reference image – unmodified image from a data set, and
SLSSIM is:
v augmented image – created from reference image by applying
u
u Yk an augmentation. Similarity between the two images was
SLSSIM = t(1 − L∗ ) ·
k+1
1 (SSIM ∗ (X (i) , Y (i) )) (6) evaluated using the selected and proposed similarity metrics
i=0 from sections III and IV. The results were aggregated by
SLSSIM is a root of a product where the members are taking the mean metric value from all the images for each
normalized L1 calculated from the entire image and SSIM augmentation level.
calculated from defined sub-levels. Here, L1 is used the same
A. Noise detection results
way as in subsection III-C, to decrease the metric value further
if the image is more distorted and. Also, as all of the product As defined by requirement R1 in section II a medical X-
members have values in interval [0; 1] the root is used to ray image similarity metric must detect emerging noises. Two
prevent a steep downward gradient of the metric. We will be image augmentations simulating real life scenarios have been
using 8 sub-levels (k = 8) for metrics performance analysis. selected to generate augmented images for testing:
• Poisson-Gauss noise augmentation PG(a, b) – represents
V. M ETRICS ANALYSIS artifacts and noises emerging in images obtained via
Analysis has been carried out to evaluate performance of the computed tomography X-ray imaging when using X-ray
selected metrics in real-life scenarios. Tests were conducted sensors in low dosage configuration [8]. The augmented
93
Fig. 2. Results with rotation augmentation (left), Y-axis translation augmentation (right)
image was generated using parameters: a = 31 and B. Geometric distortion detection results
b = 0.05k, k ∈ [0; 20], k ∈ Z. Higher b values result As defined by requirement R2 in section II a medical X-ray
in more noise in the image. There are confirmed results image similarity metric must also detect perceptual geometric
stating that noises generated with parameter b ≈ 0.1 can distortions. Two widely used image augmentations have been
occur in low dose imaging [8]. selected to simulate geometric transformations and generate
• Additive Gauss noise augmentation N (µ, σ) – repre-
augmented images for testing:
sents general noises and artifacts that can appear in X-
• Translation augmentation – used to evaluate general met-
ray images. Some research points to its applicability in
ric robustness against translation (shifting of the image).
simulating noises emerging from low dose imaging as
Theoretically, translation can occur during reconstruction
well [38]. The augmented image was generated using
or decompression. The augmented image was generated
parameters: µ = 0, σ ∈ [0; 20], σ ∈ Z. Higher σ values
by translating the reference image on the Y axis by y
result in more noise in the image. Visually, the noise
pixels, where values y = {−30, −28, ..., 26, 28, 30}
becomes noticeable at σ ≈ 5.
• Rotation augmentation – also used to evaluate general
metric robustness. The augmented image was generated
1) Poisson Gauss noise detection results: On the left side of by rotating the reference image by r degrees, where
figure 1 similarity metrics ability to detect emerging Poisson- values r = {−30, −28, ..., 26, 28, 30}
Gauss noise is observed. Metrics L∗1 , M SE ∗ , wSSIM ∗ do
1) Rotation augmentation detection results: On the left
not detect the noise even when the noise parameter b is high
side of figure 2 similarity metrics ability to detect rotations
(images are highly disrupted). Whereas, SSIM ∗ metric and
is observed. As an emerging pattern, metrics L∗1 , M SE ∗ ,
the proposed metric SLSSIM perform well. When noise
wSSIM ∗ do not detect the distortions even when the degree
parameter b = 0.1, metric values SLSSIM ≈ 0.78 and
of rotation is very high. SSIM ∗ and SLSSIM still perform
SSIM ∗ ≈ 0.66. These results are acceptable, as visually the
well, with SSIM ∗ once again being slightly more sensitive
noise should still allow for accurate diagnosis.
to the level of distortion.
2) Additive Gauss noise detection results: On the right side 2) Translation augmentation detection results: On the right
of figure 1 similarity metrics performance against emerging side of figure 1 similarity metrics performance in detecting
additive Gauss noise is witnessed. Similarly to the previous translations is observed. Once again, metrics L∗1 , M SE ∗ ,
results, metrics L∗1 , M SE ∗ , wSSIM ∗ do not detect the noise wSSIM ∗ do not detect distortions well and metrics SSIM ∗ ,
well, the resulting values do not drop below 0.94. However, SLSSIM ∗ do so.
in this case the difference between the proposed SLSSIM
metric and SSIM ∗ is higher. When noise parameter σ = 5, C. Analysis results
metric values SLSSIM ≈ 0.84 and SSIM ∗ ≈ 0.76. As observed in the tests L∗1 , M SE ∗ , wSSIM ∗ metrics
Although these values are still acceptable, the rate at which are not suitable for accurately detecting noises or geometric
SLSSIM value decreases as the noise level rises is not as transformations in medical X-ray images. While tests show
representative of the added disruptions as the SSIM ∗ . It can that SSIM ∗ is more sensitive to additive Gauss noise, but
be stated that SSIM is slightly more accurate in detecting SLSSIM has an advantage in being the only metric that
additive Gauss noise. specifically measures discrepancies in sub-levels of the image.
94
Fig. 3. Sample chest X-ray image from NIH data set (left), same image distorted with Poisson Gauss noise (right)
Fig. 4. Sample wrist X-ray image from MURA data set (left), image distorted with Poisson Gauss noise (center), visualisation of sub-level range [64; 127]
of distorted image (right)
Both SSIM ∗ and SLSSIM metrics are potentially suitable seen on the right. It simulates the effect that low dose imaging
for use with medical X-ray images. would have on the image [8]. The metric values calculated
from reference and augmented images are: M SE ∗ = 0.99,
VI. S AMPLES OF METRICS BEHAVIOUR ON X-R AY IMAGES L∗1 = 0.93, SSIM ∗ = 0.10, wSSIM ∗ = 0.93 and
Sample images from the data sets have been displayed to SLSSIM = 0.36.
better visualise the impact that augmentations have on the Similarly, in figure 4 a reference X-Ray image from the
images. Also, metric values are given for each image pair. MURA musculoskeletal X-ray image data set can be seen on
In figure 3 a reference X-Ray image from NIH chest X-ray the left. An augmented image distorted by Poisson-Gaussian
image data set can be seen on the left. An augmented image noise, with same parameters as before can be seen in the
distorted by Poisson-Gaussian noise, a = 31, b = 0.08 can be center. The image on the right represents a visualisation of
95
the augmented image sub-level of intensity range [64; 127]. In [12] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol,
this particular case, this sub-level represents the bone structure “Stacked Denoising Autoencoders: Learning Useful Representations in
a Deep Network with a Local Denoising Criterion,” J. Mach. Learn.
from the image, with tissue and other matter being barely Res., vol. 11, pp. 3371–3408, 2010.
represented. The metric values calculated from reference and [13] D. Wu, K. Kim, G. El Fakhri, and Q. Li, “A Cascaded Convolutional
augmented images are: M SE ∗ = 0.98, L∗1 = 0.88, SSIM ∗ = Neural Network for X-ray Low-dose CT Image Denoising,” May 2017.
[14] J. Xie, L. Xu, and E. Chen, “Image Denoising and Inpainting with
0.08, wSSIM ∗ = 0.89 and SLSSIM = 0.27. It is important Deep Neural Networks,” pp. 341–349, 2012.
to note that the metrics values taken from images seen in [15] G. Toderici et al., “Full Resolution Image Compression with Recurrent
figures 3 and 4 correspond with the results of the analysis Neural Networks,” Aug. 2016.
in section V. [16] Q. Ke et al., “A neuro-heuristic approach for recognition of lung
diseases from X-ray images,” Expert Syst. Appl., vol. 126, pp. 218–232,
Jul. 2019.
VII. C ONCLUSIONS [17] Z. Wang, A. C. Bovik, H. R. Sheikh, S. Member, E. P. Simoncelli,
and S. Member, “Image Quality Assessment: From Error Visibility to
Although there are many general purpose image similarity Structural Similarity,” vol. 13, no. 4, pp. 1–14, 2004.
metrics used in various domains, only a few of them are [18] T. Černius, “Image reconstruction of x-ray images using deep learning,”
Vilnius University, 2018, (In Lithuanian).
suitable for use in comparing medical X-ray images. Based [19] K. Xu, X. Liu, H. Cai, and Z. Gao, “Full-reference image quality
on the analysis performed on multiple medical X-ray image assessment-based B-mode ultrasound image similarity measure,” Jan.
data sets and using different augmentations, L∗1 , M SE ∗ , 2017.
[20] D. M. Chandler, “Seven Challenges in Image Quality Assessment: Past,
wSSIM ∗ metrics are ineffective in this domain as metric Present, and Future Research,” ISRN Signal Process., vol. 2013, pp.
values never drop below 0.8 even when significant distortions 1–53, 2013.
are applied. SSIM ∗ and the proposed SLSSIM can be used [21] A. B. Watson and Bernd, Digital images and human vision. MIT Press,
to effectively compare X-ray images as both metrics’ value 1993.
[22] Z. Wang, A. C. Bovik, and L. Lu, “Why is image quality assessment
is much lower when comparing heavily distorted images, in so difficult?,” in IEEE International Conference on Acoustics Speech
some cases going below 0.3. Both metrics perform similarly and Signal Processing, 2002, p. IV-3313-IV-3316.
in tested cases. Unlike SSIM ∗ , SLSSIM takes into account [23] W. Lin and C.-C. Jay Kuo, “Perceptual visual quality metrics: A
survey,” J. Vis. Commun. Image Represent., vol. 22, no. 4, pp. 297–312,
differences between human body structures in the images, May 2011.
which should allow diagnosticians to make more accurate [24] F. Gao, Y. Wang, P. Li, M. Tan, J. Yu, and Y. Zhu, “DeepSim: Deep
diagnoses. The proposed SLSSIM can be used as a basis of similarity for image quality assessment,” Neurocomputing, vol. 257,
pp. 104–114, Sep. 2017.
an objective function in medical X-ray image reconstruction
[25] R. Reisenhofer, S. Bosse, G. Kutyniok, and T. Wiegand, “A Haar
or compression tasks. Wavelet-Based Perceptual Similarity Index for Image Quality Assess-
ment,” Jul. 2016.
R EFERENCES [26] L. Zhang and H. Li, “SR-SIM: A fast and high performance IQA index
based on spectral residual,” in 2012 19th IEEE International Conference
on Image Processing, 2012, pp. 1473–1476.
[1] U. Eglė, “Diagnostic X-ray hardware,” 2011, (In Lithuanian).
[27] W. Xue, L. Zhang, X. Mou, and A. C. Bovik, “Gradient Magnitude
[2] M. Woźniak and D. Połap, “Bio-inspired methods modeled for respi-
Similarity Deviation: A Highly Efficient Perceptual Image Quality
ratory disease detection from medical images,” Swarm Evol. Comput.,
Index,” IEEE Trans. Image Process., vol. 23, no. 2, pp. 684–695, Feb.
vol. 41, pp. 69–96, Aug. 2018.
2014.
[3] Wozniak, M., Polap, D., Kosmider, L., Napoli, C., and Tramontana,
E., “A novel approach toward X-ray images classifier”. In 2015 IEEE [28] Lin Zhang, Lei Zhang, Xuanqin Mou, and D. Zhang, “FSIM: A Feature
Symposium Series on Computational Intelligence, pp. 1635-1641, Similarity Index for Image Quality Assessment,” IEEE Trans. Image
IEEE, 2015. Process., vol. 20, no. 8, pp. 2378–2386, Aug. 2011.
[4] B. V. Daga, V. R. Shah, and S. V. Daga, Radiodiagnosis, nuclear [29] N. Johnston et al., “Improved Lossy Image Compression with Priming
medicine, radiotherapy and radiation oncology. Jaypee Brothers Med- and Spatially Adaptive Bit Rates for Recurrent Networks,” Mar. 2017.
ical Publishers, 2013. [30] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Net-
[5] A. Berrington de González et al., “Projected Cancer Risks From works for Biomedical Image Segmentation,” May 2015.
Computed Tomographic Scans Performed in the United States in 2007,” [31] G. Wang, J. C. Ye, K. Mueller, and J. A. Fessler, “Image Reconstruction
Arch. Intern. Med., vol. 169, no. 22, p. 2071, Dec. 2009. is a New Frontier of Machine Learning,” IEEE Trans. Med. Imaging,
[6] Y. Y. Kim, H. J. Shin, M. J. Kim, and M.-J. Lee, “Comparison of vol. 37, no. 6, pp. 1289–1296, Jun. 2018.
effective radiation doses from X-ray, CT, and PET/CT in pediatric [32] S. Bosse, D. Maniry, K.-R. Muller, T. Wiegand, and W. Samek, “Deep
patients with neuroblastoma using a dose monitoring program.,” Diagn. Neural Networks for No-Reference and Full-Reference Image Quality
Interv. Radiol., vol. 22, no. 4, pp. 390–4, 2016. Assessment,” IEEE Trans. Image Process., vol. 27, no. 1, pp. 206–219,
[7] S. P. Raman, M. Mahesh, R. V. Blasko, and E. K. Fishman, “CT scan Jan. 2018.
parameters and radiation dose: Practical advice for radiologists,” J. Am. [33] Wozniak, M., Polap, D., Nowicki, R. K., Napoli, C., Pappalardo,
Coll. Radiol., vol. 10, no. 11, pp. 840–846, 2013. G., and Tramontana, E., “Novel approach toward medical signals
[8] S. Lee, M. S. Lee, and M. G. Kang, “Poisson–Gaussian Noise Analysis classifier”. In 2015 International Joint Conference on Neural Networks
and Estimation for Low-Dose X-ray Images in the NSCT Domain,” (IJCNN), pp. 1-7, IEEE, 2015.
Sensors, vol. 18, no. 4, p. 1019, Mar. 2018. [34] V. V. Lukin, N. N. Ponomarenko, O. I. Ieremeiev, K. O. Egiazarian,
[9] A. Neverauskiene et al., “Image based simulation of the low dose com- and J. Astola, “Combining full-reference image visual quality metrics
puted tomography images suggests 13 mAs 120 kV suitability for non- by neural network,” 2015, p. 93940K.
syndromic craniosynostosis diagnosis without iterative reconstruction [35] “RSNA Bone Age data set from Kaggle.” [Online]. Available:
algorithms,” Eur. J. Radiol., vol. 105, pp. 168–174, Aug. 2018. https://www.kaggle.com/kmader/rsna-bone-age. [Accessed: 01-Mar-
[10] Jiang Hsieh, Computed Tomography: Principles, Design, Artifacts, and 2019].
Recent Advances, Second Edition, 2009. [36] “NIH Chest X-rays data set from Kaggle.” [Online]. Avail-
[11] L. L. Geyer et al., “State of the Art: Iterative CT Reconstruction able: https://www.kaggle.com/nih-chest-xrays/data. [Accessed: 01-
Techniques,” Radiology, vol. 276, no. 2, pp. 339–357, Aug. 2015. Mar-2019].
96
[37] “Stanford University MURA Dataset: Towards Radiologist-Level Ab-
normality Detection in Musculoskeletal Radiographs.” [Online]. Avail-
able: https://stanfordmlgroup.github.io/competitions/mura/. [Accessed:
01-Mar-2019].
[38] Hanying Li, T. Toth, S. McOlash, Jiang Hsieh, and N. Bromberg,
“Simulating low dose CT scans by noise addition,” in 2002 IEEE
Nuclear Science Symposium Conference Record, 2002, vol. 3, pp.
1832–1834.
97