Development of a novel method of adaptive image interpolation for image resizing using artificial intelligence

Abstract

In this paper, we develop an artificial neural network (ANN)-based adaptive image interpolation method for image resizing. A local image dataset is also created, consisting of images with names such as Amir Temur, Muhammad al-Khwarizmi, TUIT and the Tashkent TV Tower. The proposed adaptive image interpolation method based on artificial neural networks is compared with non-adaptive image interpolation methods such as cubic, area, nearest neighbor, lanczos4 and linear using a local image data set. The comparison is based on assessment methods such as Mean Squared Error (MSE), Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The comparison clearly shows that the proposed method outperforms its counterparts considered in this work.

Keywords

• 1 Image interpolation
• ANN
• image resizing
• local dataset

1. Introduction

In today’s age of digital technology, digital images have become an integral part of our lives. Image interpolation methods are widely used in digital image processing. Image interpolation methods fall into two main types: adaptive and non-adaptive. A number of important studies on image interpolation methods have been carried out in the last few decades. In [ 1 ], it is presented an adaptive image resizing algorithm based on the Newton interpolation function. Experimental results show that the visual effect of their procedure surpasses that of bicubic interpolation when resizing images, and the PSNR values of the resized image by their proposed algorithm are larger than those of other classical interpolation algorithms. The proposed algorithm implements image interpolation with high efficiency and is particularly well suited for real-time image resizing. Various image interpolation techniques for image enhancement are discussed in [ 2 ]. An overview of different interpolation techniques such as Nearest Neighbor, Bilinear, Bicubic, New Edge-Directed Interpolation (NEDI), DataDependent Triangulation (DDT) and Iterative Curvature-Based Interpolation (ICBI) is given. Sunil et al. [ 3 ] propose a computationally simple interpolation algorithm. In their algorithm, the unknown pixels are categorized into different bins depending on the property of the neighboring pixels (activity level) and for each bin fixed prediction parameters are used for prediction. A different set of fixed predictors is presented for both smooth and edgy/angular images. A modified algorithm is also proposed in which the selection of the prediction parameter is done on a block-by-block basis rather than on a frame-byframe basis. Their proposed algorithm gives much better qualitative and quantitative performance compared to other computationally simple interpolation algorithms. Non-adaptive image interpolation algorithms based on quantitative measures are examined in [ 4 ].

The survey analyzes the properties of various non-adaptive interpolation techniques based on their PSNR of the interpolated image and their computational complexity. The applicability of these techniques in real-time applications is also examined. Based on the evaluation, it can be suggested that first-order polynomial convolutional interpolation (FOPCI) is suitable for real-time applications due to its better PSNR and low computational cost, and the performance of FOPCI can be improved by using appropriate filters. A new technique for segmenting document images is presented in [ 5 ]. In [ 6 ] an adaptive technique for image interpolation using the bilinear, the bicubic and the cubic spline method is proposed by adaptively weighting the pixels involved in the interpolation process. The adaptive technique is compared to the conventional interpolation technique and the distorted/warped distance interpolation technique.

Another interesting study can be found in [ 7 ]. An adaptive interpolation technique based on the Newtonian forward difference is developed. The forward difference provides a measure of the goodness of grouping pixels around the target pixel for interpolation. In [ 8 ], an image interpolation model based on a probabilistic neural network (PNN) is proposed. The method automatically sets and maintains alignment settings for various smooth image areas, considering the properties of a plane (flat area) and accuracy (edge area) model.

In [ 9 ], a novel adaptive interpolation algorithm based on Newton's polynomial is developed to improve the limitation of the traditional image resizing algorithm. The efficiency of the proposed method is compared to that of the traditional Matlab image resizing toolbox. In [ 10 ], it is realised an image contrast enhancement by using nonlinear oscillatory theory. In the study, it is studied two different uncoupled networks based on nonlinear oscillators. According to the research, results show a possible effective area of application of nonlinear oscillators for image processing tasks. In [ 11 ] develop an adaptive image interpolation technique based on a cubic trigonometric B-spline representation. Image quality metrics such as SSIM, MS-SSIM and FSIM along with the classic PSNR are used to examine the quality of interpolated digital images. In [ 12 ], it is considered the metric objective quality assessment of compressed TV images based on the prediction error values of sums of pixels of the original and decoded images. In [ 13 ], a comparative study of different resampling techniques like Cubic Splines, Nearest Neighbor, Cubic Convolution and Linear Interpolation is given, which can be used as detectors for a altered image containing resampled parts/portions. In [ 14-15 ], an overview of different adaptive and non-adaptive image interpolation techniques is given and a comparison based on their performance parameter (i.e. H. PSNR) is performed. In [ 16 ], it is conducted a systematic discussion of both pros and cons of CNN based and coupled nonlinear oscillators' based approaches for image contrast enhancement. In [ 17 ], it is presented an efficiency estimation of digital image resizing using various image interpolation methods, such as Bicubic, B-Spline, Mitchell, Lanczos. It is also shown the experimental results of quality changing after image reduction and restoration. In [ 18 ], a machine learning based approach for lossy image compression is presented that outperforms all existing codecs while running in real time. According to the proposed algorithm, files are produced that are 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP and 1.7 times smaller than BPG on datasets of generic images across all quality levels. In [ 19 ], an adaptive image scaling algorithm based on continuous fractional interpolation and hierarchical processing with multiple resolutions is proposed. The algorithm achieves a smooth, high-order transition between pixels in the same feature region, and can also modify the pixels of the image adaptively. Finally, in [ 20 ] the adaptive image resizing using edge contrasting concept is presented. The concept is tested with more than 100 frames and found to have far superior performance in terms of PSNR and MSE scores.

Overall, the overview of the previous contribution on image interpolation and resizing witnesses the tremendous attention that has been devoted to the development of various methods and algorithms over the last few decades. However, little attention has been paid to techniques based on neural networks. This paper contributes to the enrichment of the literature by developing a novel, robust, and efficient ANNbased adaptive image interpolation method for image resizing. The advantage of the developed method lies in the possibility of efficiently maintaining the image quality. Furthermore, the developed method has concrete potential applications such as the efficient transmission of high-quality images at high speed.

The rest of the paper is organized as follows. Section 2 is dedicated to both modeling and design of the novel concept. Section 3 focuses on the implementation of the concept and discussion of the results achieved. Concluding remarks are formulated in Section 4.

2. The proposed ANN-based model

This section presents the development process of the proposed ANN-based image resizing model. A synoptic representation of the proposed process is shown in Figure 1. The proposed ANNbased model for image compression consists of the following steps: First, the camera captures the original image [ 21-22 ]. Then the image is resized using the interpolation method. After that, the JPEG compression process takes place. The compressed image is transmitted to the receiver via a radio module. On the receiving side, the image received via the radio module is subjected to JPEG decompression. Then the next steps are to choose an appropriate neural network model for image resizing. There are different types of neural networks in data processing. These include: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Artificial Neural Network (ANN), just to name a few. Amongst the aforementioned types of neural networks, the ANN type is selected to perform the image resizing process and insuring an efficient image recovery. principle of 2 x 2 to 3 x 3. The model in figure 2 encompasses two hidden layers, 12 inputs and 27 outputs. The backpropagation model was used to develop the proposed method. Backpropagation is an algorithm that is widely used for training feedforward neural networks. The main purpose of the backpropagation model is to correct output errors.

( ) = (1+1− ) (1)

The sigmoid activation is shown in Fig. 5. It takes a real value and "squeezes" in the range from 0 to 1. In particular, large negative numbers are equal to 0 and large positive numbers are equal to 1.

3. Performance validation

For the validation of the proposed ANN-based image resizing method, experimental results are evaluated using Mean Squared Error (MSE), Root Mean Square Error (RMSE), Peak Signal-toNoise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) estimation methods.

Mean Square Error (MSE) is a commonly used metric for the evaluation of the image quality. The better image quality is obtained for MSE values closed to zero. The variance of the estimator corresponds to the second moment of error. The standard deviation is deduced from the variance and is used to evaluate the uncertainty. The MSE corresponds to the variance of the predictor in the objective estimator. It has units of measurement equal to the square of the magnitude calculated as the variance.

Mean Squared Error (MSE) between two images, say g (x,y) and ĝ (x,y) is defined in equation (2) (see also Ref. [ 23 ]) to assess the absolute error. = 1 ∑=0

∑=1 ̂[(, − (, ) possible signal power and the power of the distorting noise that affects the quality of its representation. This ratio between two images is computed in decibel form. The Peak signal-tonoise ratio is the most commonly used quality assessment technique to measure the quality of reconstruction of lossy image compression codecs. The signal is treated as the original data and the noise is the error caused by the compression or distortion. The representation of the absolute error (in dB) is ex-pressed by equation (4).

PSNR = 10 log10 peakval2

MSE (4)

Where peakval denotes the peak value and corresponds to the maximal in the image data. If it is an 8-bit unsigned integer data type, the peakval is 255 [ 25 ].

Structural similarity index measure (SSIM). The structural similarity index method is a model based on this perception. The term structural data refers to interconnected pixels or spatially closed pixels. This interconnected resolution points to a number of important information about objects in the field of images. Lighting masking is a term where the distorted part of the image is less visible at the edges of the image. Contrasting masking, on the other hand, is a term that these distortions are less visible in the image structure. The SSIM expressed in equation (5) is used to predict the perceived quality of images and videos. It measures the similarity between the two images: the original and the restored.

SSIM(x, y) = (2μxμy+c1)(2σxy+c2) (μ2x+μ2y+c1)(σ2x+σ2y+c2) x 100 (5) 2 the variance of y;

Where μx is the average of x and μy the average of y; 2 stands for the variance of x and denotes the covariance of x and y ; 1 = ( 1) 2, are two key parameters used to stabilize the division with weak denominator; L is the dynamic range of the pixelvalues (typically this is 2#bits per pixel − 1), k1 = 0.01 and k2 = 0.03 by default.

As mentioned above, a local image dataset was also created in this study. The local data set was used for the comparison. Since this image data set was created only recently and has not yet been used by other scientists, interpolation methods for resizing images were used for comparison.

Interpolation methods such as Nearest, Linear Area, Cubic, Lanczos4 were used for comparison. The comparison to MSE is shown in Figure 4. experimental results of local images

Figure 4 shows the comparison (based on the MSE) of selected estimation methods. The worst result is obtained with the closest interpolation method. The best result is obtained by the method proposed in this work. Among the interpolation methods, the cubic interpolation is the best in terms of quality. For this reason, the method that came closest to the proposed method was the cubic interpolation method.

The comparison of selected methods to RMSE is depicted in Figure 5. We use four selected local images with five alternative interpolation methods. for the cubic method, 30.046 for the Lanczos4 method and 31.797 for the proposed method. This comparison witnesses the fact that based on the PSNR metric the proposed method is better than the counterparts methods used for the benchmark.

Based on the SSIM metric, the proposed method and its counterparts are applied to the local images and the obtained results are compared and presented in Figure 7.

As shown in Figure 8 and Table 1, the average values of the metrics, namely PSNR, RMSE, MSE and SSIM, are each evaluated using different methods. The result of the evaluation has led to the following values: 25.421, 14.091, 211.79 and 86.276 using the Nearest method; 28.93, 9.452, 96.17 and 91 using the Linear method; 28.47, 9.94, 105.8 and 91.44 using the Area method; 30.667, 7.899, 69.270, 93.807 using Cubic method; 30,046, 8.564, 82.341 and 92.700 using the Lancsoz4 method; 31.797, 6.864, 51.550, 94.366 using the proposed method. These results clearly show that for each of the four metrics used for comparison, the proposed method outperforms each of its five other counterparts used for the benchmark.

a) b) c) Figure 10: Example of obtained comparison on local image

As can be seen from Figure 10, a comparison of the image named Amir Temur, which is part of the local image data set, is performed. Here, (a) represents the original image, (b) the result obtained based on the cubic method, which is said to be the best among the interpolation methods, and (c) the result obtained based on the proposed method. When comparing the images, the image (c) obtained by the proposed method shows a result close to the original image (a). This clearly shows that the proposed method is better than the other five counterparts.

4. Conclusions and future work

In this paper, we developed an ANN-based adaptive image interpolation concept for image resizing and a local image dataset consisting of images such as Amir Temur, Muhammad alKhwarizmi, TUIT, and Tashkent TV Tower. Based on selected metrics, namely MSE, RMSE, PSNR and SSIM, the developed method was compared to non-adaptive image interpolation methods like Cubic, Area, Nearest Neighbor, Lanczos4 and Linear. The comparison clearly showed that the proposed method outperforms each of its counterparts.

As an outlook, the following points of necessary importance are currently under investigation: Enrichment of the local data set with new images, as this could contribute to better results (e.g. improvement in the robustness of the method proposed in this work and in the image quality); Demonstration of the potential application of the proposed method in different fields of engineering.

5. References

References

[1] J. Xiao , X. Zou , Z. Liu , and X. Guo “Adaptive Interpolation Algorithm for Real-time Image Resizing,” First International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06) , 2006 , pp. 221 - 224 .

[2] V. Patel and K. Mistree , “ A review on different image interpolation techniques for image enhancement , ” Ijetae , vol. 3 , no. 12 , pp. 129 - 133 , 2013 .

[3] S. Prasad Jaiswal , V. Jakhetiya , A. Kumar , and A. Kumar Tiwari , “ A low complex context adaptive image interpolation algorithm for real-time applications ,” 2012 IEEE I2MTC - Int. Instrum. Meas. Technol. Conf. Proc., no. May , pp. 969 - 972 , 2012 .

[4] C. J. Moses and D. Selvathi , “ A Survey on Adaptive Image Interpolation based on Quantitative Measures,” Asian J. Res. Soc. Sci. Humanit ., vol. 6 , no. 4 , p. 341 , 2016 .

[5] M. N. Ahmed , B. E. Cooper , and S. T. Love, “ Adaptive image interpolation using a multilayer neural network , ” Appl. Artif. Neural Networks Image Process. VI , vol. 4305 , pp. 30 - 38 , 2001 .

[6] M. M. Hadhoud , M. I. Dessouky , F. E. Abd El-Samie , and S. E. El-Khamy , “ Adaptive image interpolation based on local activity levels,” Natl . Radio Sci. Conf. NRSC, Proc. , vol. 2003-Janua, no. May 2014 , pp. C41 - C48 , 2003 .

[7] A. K. Murmu , “ Adpative image interpolation Adaptive image interpolation ,” 2014 .

[8] S. H. Hsieh and C. H. Chen , “ Adaptive image interpolation using probabilistic neural network,” Expert Syst . Appl. , vol. 36 , no. 3 PART 2 , pp. 6025 - 6029 , 2009 .

[9] J. Xiao , X. Zou , Z. Liu , and X. Guo , “ A novel adaptive interpolation algorithm for image resizing , ” Int. J. Innov. Comput. Inf. Control , vol. 3 , no. 6 A , pp. 1335 - 1345 , 2007 .

[10] M. A. Latif , J. C. Chedjou and K. Kyamakya , "The paradigm of Non-linear oscillators in image processing," VXV International Symposium on Theoretical Engineering , 2009 , pp. 1 - 5 .

[11] S. Abbas , M. Irshad , and M. Z. Hussain , “ Adaptive image interpolation technique based on cubic trigonometric Bspline representation,” IET Image Process ., vol. 12 , no. 5 , pp. 769 - 777 , 2018 .

[12] I. A. Gavrilov and K. K. Nosirov , “ Quality Evaluation of Compressed Images Based on Prediction Errors of Pixel Values,” 2018 XIV International Scientific-Technical Conference on Actual Problems of Electronics Instrument Engineering (APEIE) , 2018 , pp. 33 - 36 .

[13] M. S Malini and M. Patil , “ Interpolation Techniques in Image Resampling,” Int. J. Eng . Technol., vol. 7 , no. 3 .34, p. 567 , 2018 .

[14] S. H. Mahajan and V. K. Harpale , “ Adaptive and nonadaptive image interpolation techniques , ” Proc. - 1st Int. Conf. Comput. Commun. Control Autom. ICCUBEA 2015 , pp. 772 - 775 , 2015 .

[15] K. Nosirov , S. Begmatov and M. Arabboev , "Design Of A Model For Multi-Parametric Health Monitoring System," 2020 International Conference on Information Science and Communications Technologies (ICISCT) , 2020 , pp. 1 - 5

[16] K. Kyamakya , J. C. Chedjou , M. A. Latif and U. A. Khan , “ A novel image processing approach combining a 'coupled nonlinear oscillators'-based paradigm with cellular neural networks for dynamic robust contrast enhancement , ” 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010 ), 2010 , pp. 1 - 7 .

[17] A. Puziy , I. Gavrilov, K. Nosirov and A. Akhmedova , “ Efficiency Estimation Of Image Resizing Based On Interpolating Transformations,” 2019 International Conference on Information Science and Communications Technologies (ICISCT) , 2019 , pp. 1 - 5 .

[18] O. Rippel and L. Bourdev , “ Real-time adaptive image compression , ” 34th Int. Conf. Mach. Learn. ICML 2017 , vol. 6 , pp. 4457 - 4473 , 2017 .

[19] J. Jin , “ An adaptive image scaling algorithm based on continuous fraction interpolation and multi-resolution hierarchy processing , ” Fractals , vol. 28 , no. 8 , pp. 1 - 13 , 2020 .

[20] M. Sadiq and A. Aziz , “ Adaptive Image Resizing using Edge Contrasting,” Int. J. Trend Sci. Res . Dev., vol. 4 , no. 6 , pp. 1711 - 1716 , 2020 .

[21] M. Arabboev , S. Begmatov , K. Nosirov , A. Shakhobiddinov , J. C. Chedjou and K. Kyamakya , "Development of a Prototype of a Search and Rescue Robot Equipped with Multiple Cameras," 2021 International Conference on Information Science and Communications Technologies (ICISCT) , 2021 , pp. 1 - 5

[22] K. Nosirov , S. Begmatov and M. Arabboev , "Analog Sensing and Leap Motion Integrated Remote Controller for Search and Rescue Robot System," 2020 International Conference on Information Science and Communications Technologies (ICISCT) , 2020 , pp. 1 - 5

[23] J. Søgaard , L. Krasula , M. Shahid , D. Temel , K. Brunnström , and M. Razaak , “ Applicability of existing objective metrics of perceptual quality for adaptive video streaming ,” IS T Int . Symp. Electron. Imaging Sci. Technol ., pp. 1 - 7 , 2016 .

[24] U. Sara , M. Akter , and M. S. Uddin , “ Image Quality Assessment through FSIM, SSIM, MSE and PSNR-A Comparative Study,” J. Comput . Commun., vol. 07 , no. 03 , pp. 8 - 18 , 2019 .

[25] R. G. Deshpande , L. L. Ragha , and S. K. Sharma , “ Video quality assessment through PSNR estimation for different compression standards,” Indones. J. Electr . Eng. Comput. Sci. , vol. 11 , no. 3 , pp. 918 - 924 , 2018 .