Comparison of Some Image Quality Approaches Boris B. Parfenenkov, Maksim A. Panachev Ural Federal University {idlerboris,tiopox}@gmail.com Abstract. This paper is devoted to image quality problem. We analyze ad- vantages and disadvantages of existing methods. Classification of quality met- rics into some groups has been done. Based on this classification, we formed proposition about prospects of using this methods in solving image quality problem. Keywords: image processing, image fidelity, image quality, MSE, SSIM, VDP, anisotropic. Introduction The problem of quality assessment arises in many different subjects. From com- puter graphics, where rendering of complex scenes may had a lot of time. To bioin- formatics and computer security, where quality and accuracy of images may safe human lives. Although, count of image editors, which must define image quality and may improve this, significantly increases. Certainly, our eyes is good classifier of image fidelity, but there are a lot of dif- ferent software systems, which must define quality of digital images. Consequently, using of human resource is not acceptable for this problem. In this paper we are describe of existing methods of quality image assessment. Classification of quality metrics into some groups has been done. Although analyze of advantages and disadvantages of the most promising methods was performed. We tested these metrics on the collected set of images and selected metric, which we rec- ommend to use for solving similar tasks. Image quality assessment Let there be two digital images: X – original, Y – test (distorted image – with possible defects). The challenge is to build algorithm, which have these two images and define quality assessment of test image. Digital image may be represent with brightness matrix 𝐼 = (𝑝𝑖,𝑗 )𝐻×𝑊 , where 𝑝𝑖,𝑗 ∈ [𝑎, 𝑏] ⋂ ℤ, 𝐻 𝑎𝑛𝑑 𝑊 – height and width of image, respective- ly. Although, in some cases, we will consider image as one-dimensional signal 𝑋 = (𝑥𝑖∙𝑊+𝑗 )𝐻∙𝑊 . 48 Basic metrics At the beginning, we consider classic metrics that came to computer vision from mathematical statistics. Mean squared error of images 𝑋 and 𝑌 presented as 1 𝑀𝑆𝐸(𝑋, 𝑌) = ∑𝑁 2 𝑖=1(𝑥𝑖 − 𝑦𝑖 ) . Sometimes researchers considered root mean squared 𝑁 error 𝑅𝑀𝑆𝐸(𝑋, 𝑌) = √𝑀𝑆𝐸(𝑋, 𝑌), which may be generalize on 𝑙𝑝 : 𝑑𝑝 (𝑋, 𝑌) = 1 (∑𝑁 𝑝 𝑝 𝑖=1|𝑥𝑖 − 𝑦𝑖 | ) . Peak signal-to-noise ratio is calculated based on MSE and often apply for measure 𝐿2 of distortion when image was compressed. 𝑃𝑆𝑁𝑅 = 10 log10 , where 𝐿 = (𝑏 − 𝑀𝑆𝐸 𝑎) − is the dynamic range of allowable image pixel intensities (e.g., for image that have allocations of 8 b/pixel of gray-scale, 𝐿 = 255). However, these metrics are not best instruments for quality assessment of images [3], because they ignore features of human image perception. Structural similarity metrics In paper [7] was discussed reasons of creating metrics based on structural similari- ty. The main idea is that human able to extract some structure from image and per- ceive it, but not separately pixels. Therefore, metric, which can be measure amount and kind of structural information from image, can significantly increases image qual- ity assessment. The first result in this approach was metric SSIM (Structural SIMilarity), which 2𝜇 𝜇 +𝐶 computing as composition of: illumination (𝑙(𝑋, 𝑌) = 2 𝑥 𝑦2 1 ), contrast (𝑐(𝑋, 𝑌) = 𝜇𝑥 +𝜇𝑦+𝐶1 2𝜎𝑥 𝜎𝑦 +𝐶2 𝜎𝑥𝑦 +𝐶3 ) and structural comparison (𝑠(𝑋, 𝑌) = ); where 𝜇𝑥 – expected value 𝜎𝑥2 +𝜎𝑦2 +𝐶2 𝜎𝑥 𝜎𝑦 +𝐶3 of brightness, 𝜎𝑥 – standard deviation, 𝜎𝑥𝑦 – covariance of 𝑥 and 𝑦, and 𝐶1 , 𝐶2 , 𝐶3 – some constants, that obtained experimentally. (2𝜇𝑥 𝜇𝑦 + 𝐶1 )(2𝜎𝑥𝑦 + С2 ) 𝑆𝑆𝐼𝑀(𝑋, 𝑌) = 𝑙(𝑋, 𝑌) ∗ 𝑐(𝑋, 𝑌) ∗ 𝑠(𝑋, 𝑌) = 2 . (𝜇𝑥 + 𝜇𝑦2 + 𝐶1 )(𝜎𝑥2 + 𝜎𝑦2 + 𝐶2 ) For improve this metric we can compute weighted mean value of SSIM on local features (local feature is small part of image, which focuses people’s attention; it is known, that using local features allow to discard noise and improve quality of metric [6]). Weight indicates the significance of this local feature. MS-SSIM (Multi-scale SSIM) [8] allows to improve the image quality assessment. This metric used setp by step computation 𝑐(𝑋, 𝑌) and 𝑠(𝑋, 𝑌) for different resolu- tions. By using computation MS-SSIM for each local features we can get more im- pressive results [2]. SCSSIM [1] allows to evaluate the image quality by using correction for the struc- tural features of original and test images. Visible Differences Predictor First this metric was developed by Scott Daly in his paper [10]. He analyzed, how to construct Human Visual System model for improving existing methods of image quality assessment. VDP receives 2 input images and generates output differences map (each pixel has value, which describes how different the pixels of corresponding input images). Schema of work VDP presents on Fig. 1. One of the advantages of this model is pos- 49 sibility to get prediction of local differences between images (on the pixel level), while methods described previously provided a single value for the entire image. Alt- hough, the basis of this metric are components that are already recognized in the computer vision: CSF [11], Cortex transform [4, 11], psychometric Weibull function [9]. One of the disadvantages of VDP is non-use information about color, and work only with brightness. Fig. 1. Schema of Visual Differences Predictor Anisotropy Metrics, which described above, based on original image, and quality of test image is defined in relation to it. However, there are a variety of tasks, in which we don’t have original image. Such tasks are appear in different research areas related to image quality assessment in real time. For example quality assessment of rendering complex scene in which we don’t have template of image, or quality assessment of photograph that made by medical device for analyze reliability of the data, and etc. Quality and entropy are related subjects, however, noise and information of image cannot be separated from each other (noise also has some information). For example, human with good eyesight can easily distinguish a clear object even when the image is noisy. However, analytically, entropy increases with sharpness but, in general, there is not a fair correlation when images are noisy. Hence entropy by itself is not a good indicator of image quality. And in paper [5] metric based on anisotropy was proposed, which can be represent by following: 𝐿𝑀𝑄(𝑋) = 𝐵 ∗ 𝑙𝑜𝑔10 𝐴(𝑋), where 𝐴(𝑋) – anisotropy of image and 𝐵 is a constant that must be determined to fix the range of operative values (experimentally good results were obtained for 𝐵 = 20). Analyze image fidelity metrics For analyze image fidelity metrics we collect set of images, which has been obtained by various deformations of original image. Then these images were sorted by de- crease quality in terms of human perception (this sort was done using quality image assessment by three independent people). Result set of images in the sort order pre- sent on Fig. 2 (presented reduced copies, real images have a size 256х256 pixels). Names of the images correspond to deformation types: 1) shift(𝒏) – shift brightness of all pixels by 𝒏 (i.e. image becomes lighter); 2) noise(𝒏) – add Gaussian noise (the 50 more 𝒏 – the more noise); 3) jpeg(𝒏) – image after compression by JPEG for different sizes of block (the less 𝒏 – the more defects); 4) blur(𝒏) – Gaussian blur (the more 𝒏, the more blur). Fig. 2. Images for testing image fidelity metrics Then testing of 4 metrics (PSNR, LMQ, variation of SSIM, VDP) was conducted. Each metric returns evaluation image deformation – i.e. result that inverse to quality and takes values from 0 (quality image) to 1 (deformed image). The resulting graph of the image deformation (vertical axis) of the image (horizontal axis) presented on Fig. 3. Highline that main criteria for correctness of metric is not absolute deformation value but result graph has been directed upwards (i.e. for any pair of images, defor- mation value of left image must be greater than deformation value of right image). 0,80 0,60 PSNR 0,40 LMQ SCSSIM 0,20 VDP 0,00 Fig. 3. Graph for image quality assessment by metrics: PSNR, LMQ, SCSSIM, VDP. As can be seen from the graph, the most correct results were obtained by metrics VDP and SCSSIM. Metric LMQ also gave a good result, but poorly handled with images, which are compressed using JPEG. 51 Conclusions In this paper we considered principal image fidelity metrics, from basic that came to computer vision from mathematical statistics, to hybrid models that use modern knowledge of computer vision and information quality assessment. Described metrics split into the following groups: statistics (MSE, 𝒍𝒑 norm, PSNR); structural similarity (SSIM, MSSIM, MS-SSIM, IW-SSIM, SCSSIM); human perception (VDP); anisot- ropy (LMQ). Advantages and disadvantages of these methods were considered. In the result of this work we determine that the presence of the original image should be use VDP metric. If we don’t have original image for quality assessment, then should be use LMQ, which is very costly from a computational point of view. References 1. K. Gu, G. Zhai, X. Yang, and W. Zhang “An improved fullreference image quality metric based on structure compensation”, APSIPA ASC, pp. 1 - 6, 2012 2. Z. Wang and Qiang Li “Information content weighting for perceptual image quality as- sessment”, IEEE Transaction on Image Processing, vol. 20, no. 5, pp. 1185-1198, 2011 3. Z. Wang and A. C. Bovik “Mean squared error: Love it or leave it?-A new look at signal fidelity measures,” IEEE Signal Processing Mag., vol. 26, no. 1, pp. 98-117, 2009 4. A. S. Lukin. “Improved Visible Differences Predictor Using a Complex Cortex Trans- form”, 19-th International Conference on Computer Graphics GraphiCon'2009, pp. 145−150, 2009 5. Gabarda, Salvador, and Gabriel Cristóbal. “Image quality assessment through a logarith- mic anisotropic measure”, Photonics Europe. International Society for Optics and Photon- ics, pp. 70000J-70000J, 2008 6. Datta, Ritendra, Jia Li, and James Z. Wang. “Content-based image retrieval: approaches and trends of the new age”, Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, pp. 253-262, 2005. 7. Wang, Zhou, et al. “Image quality assessment: from error visibility to structural similari- ty”, Image Processing, IEEE Transactions on 13.4, pp. 600-612, 2004 8. Wang, Zhou, Eero P. Simoncelli, and Alan C. Bovik. “Multiscale structural similarity for image quality assessment”, Signals, Systems and Computers, 2004. Conference Record of the Thirty-Seventh Asilomar Conference on. Vol. 2, pp. 1398-1402, 2003 9. Wichmann, Felix A., and N. Jeremy Hill. “The psychometric function: I. Fitting, sampling, and goodness of fit”, Perception & psychophysics 63.8, pp. 1293-1313, 2001 10. Daly, Scott J. “Visible differences predictor: an algorithm for the assessment of image fi- delity”, SPIE/IS&T Symposium on Electronic Imaging: Science and Technology. Interna- tional Society for Optics and Photonics, pp. 2-15, 1993 11. Mannos, James, and David J. Sakrison. “The effects of a visual fidelity criterion of the en- coding of images”, Information Theory, IEEE Transactions on 20.4, pp. 525-535, 1974 52 Сравнение методов оценки качества изображений Борис В. Парфененков, Максим А. Паначев Уральский Федеральный Университет {idlerboris,tiopox}@gmail.com Аннотация. Рассматривается задача оценки качества цифрового изоб- ражения. Проанализированы преимущества и недостатки существующих методов. Проведена классификация метрик качества в некоторые группы. На основе анализа и тестирования методов на выбранных изображениях, сформировано утверждение о перспективности использования выбранных методов в решении поставленной задачи. Ключевые слова. анализ изображений, метрики качества изображе- ний, image fidelity, MSE, SSIM, VDP, анизотропия. 53