Development of a novel method of adaptive image interpolation for image resizing using artificial intelligence Mukhriddin Arabboev1, Shohruh Begmatov1, Khabibullo Nosirov1, Jean Chamberlain Chedjou2 and Kyandoghere Kyamakya2 1 Tashkent University of Information Technologies named after Muhammad al-Khwarizmi, 108 Amir Temur Av., Tashkent, 100084, Uzbekistan 2 University of Klagenfurt, Universitätsstraße 65-67, 9020 Klagenfurt am Wörthersee, Austria Abstract In this paper, we develop an artificial neural network (ANN)-based adaptive image interpolation method for image resizing. A local image dataset is also created, consisting of images with names such as Amir Temur, Muhammad al-Khwarizmi, TUIT and the Tashkent TV Tower. The proposed adaptive image interpolation method based on artificial neural networks is compared with non-adaptive image interpolation methods such as cubic, area, nearest neighbor, lanczos4 and linear using a local image data set. The comparison is based on assessment methods such as Mean Squared Error (MSE), Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The comparison clearly shows that the proposed method outperforms its counterparts considered in this work. Keywords 1 Image interpolation, ANN, image resizing, local dataset 1. Introduction larger than those of other classical interpolation algorithms. The proposed algorithm implements image interpolation with high efficiency and is In today’s age of digital technology, digital particularly well suited for real-time image images have become an integral part of our lives. resizing. Various image interpolation techniques Image interpolation methods are widely used in for image enhancement are discussed in [2]. An digital image processing. Image interpolation overview of different interpolation techniques methods fall into two main types: adaptive and such as Nearest Neighbor, Bilinear, Bicubic, New non-adaptive. A number of important studies on Edge-Directed Interpolation (NEDI), Data- image interpolation methods have been carried Dependent Triangulation (DDT) and Iterative out in the last few decades. In [1], it is presented Curvature-Based Interpolation (ICBI) is given. an adaptive image resizing algorithm based on the Sunil et al. [3] propose a computationally simple Newton interpolation function. Experimental interpolation algorithm. In their algorithm, the results show that the visual effect of their unknown pixels are categorized into different bins procedure surpasses that of bicubic interpolation depending on the property of the neighboring when resizing images, and the PSNR values of the pixels (activity level) and for each bin fixed resized image by their proposed algorithm are prediction parameters are used for prediction. A IVUS 2022: 27th International Conference on Information Technology, May 12, 2022, Kaunas, Lithuania EMAIL: mukhriddin.9207@gmail.com (M. Arabboev); bek.shohruh@gmail.com (S. Begmatov); n.khabibullo1990@gmail.com (K. Nosirov); jean.chedjou@aau.at (J. C. Chedjou); kyandoghere.kyamakya@aau.at (K. Kyamakya) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings different set of fixed predictors is presented for processing tasks. In [11] develop an adaptive both smooth and edgy/angular images. A image interpolation technique based on a cubic modified algorithm is also proposed in which the trigonometric B-spline representation. Image selection of the prediction parameter is done on a quality metrics such as SSIM, MS-SSIM and block-by-block basis rather than on a frame-by- FSIM along with the classic PSNR are used to frame basis. Their proposed algorithm gives much examine the quality of interpolated digital images. better qualitative and quantitative performance In [12], it is considered the metric objective compared to other computationally simple quality assessment of compressed TV images interpolation algorithms. Non-adaptive image based on the prediction error values of sums of interpolation algorithms based on quantitative pixels of the original and decoded images. In [13], measures are examined in [4]. a comparative study of different resampling The survey analyzes the properties of various techniques like Cubic Splines, Nearest Neighbor, non-adaptive interpolation techniques based on Cubic Convolution and Linear Interpolation is their PSNR of the interpolated image and their given, which can be used as detectors for a altered computational complexity. The applicability of image containing resampled parts/portions. In these techniques in real-time applications is also [14-15], an overview of different adaptive and examined. Based on the evaluation, it can be non-adaptive image interpolation techniques is suggested that first-order polynomial given and a comparison based on their convolutional interpolation (FOPCI) is suitable performance parameter (i.e. H. PSNR) is for real-time applications due to its better PSNR performed. In [16], it is conducted a systematic and low computational cost, and the performance discussion of both pros and cons of CNN based of FOPCI can be improved by using appropriate and coupled nonlinear oscillators' based filters. A new technique for segmenting document approaches for image contrast enhancement. In images is presented in [5]. In [6] an adaptive [17], it is presented an efficiency estimation of technique for image interpolation using the digital image resizing using various image bilinear, the bicubic and the cubic spline method interpolation methods, such as Bicubic, B-Spline, is proposed by adaptively weighting the pixels Mitchell, Lanczos. It is also shown the involved in the interpolation process. The experimental results of quality changing after adaptive technique is compared to the image reduction and restoration. In [18], a conventional interpolation technique and the machine learning based approach for lossy image distorted/warped distance interpolation technique. compression is presented that outperforms all Another interesting study can be found in [7]. existing codecs while running in real time. An adaptive interpolation technique based on the According to the proposed algorithm, files are Newtonian forward difference is developed. The produced that are 2.5 times smaller than JPEG and forward difference provides a measure of the JPEG 2000, 2 times smaller than WebP and 1.7 goodness of grouping pixels around the target times smaller than BPG on datasets of generic pixel for interpolation. In [8], an image images across all quality levels. In [19], an interpolation model based on a probabilistic adaptive image scaling algorithm based on neural network (PNN) is proposed. The method continuous fractional interpolation and automatically sets and maintains alignment hierarchical processing with multiple resolutions settings for various smooth image areas, is proposed. The algorithm achieves a smooth, considering the properties of a plane (flat area) high-order transition between pixels in the same and accuracy (edge area) model. feature region, and can also modify the pixels of In [9], a novel adaptive interpolation algorithm the image adaptively. Finally, in [20] the adaptive based on Newton's polynomial is developed to image resizing using edge contrasting concept is improve the limitation of the traditional image presented. The concept is tested with more than resizing algorithm. The efficiency of the proposed 100 frames and found to have far superior method is compared to that of the traditional performance in terms of PSNR and MSE scores. Matlab image resizing toolbox. In [10], it is Overall, the overview of the previous realised an image contrast enhancement by using contribution on image interpolation and resizing nonlinear oscillatory theory. In the study, it is witnesses the tremendous attention that has been studied two different uncoupled networks based devoted to the development of various methods on nonlinear oscillators. According to the and algorithms over the last few decades. research, results show a possible effective area of However, little attention has been paid to application of nonlinear oscillators for image techniques based on neural networks. This paper contributes to the enrichment of the literature by principle of 2 x 2 to 3 x 3. The model in figure 2 developing a novel, robust, and efficient ANN- encompasses two hidden layers, 12 inputs and 27 based adaptive image interpolation method for outputs. The backpropagation model was used to image resizing. The advantage of the developed develop the proposed method. Backpropagation is method lies in the possibility of efficiently an algorithm that is widely used for training maintaining the image quality. Furthermore, the feedforward neural networks. The main purpose developed method has concrete potential of the backpropagation model is to correct output applications such as the efficient transmission of errors. high-quality images at high speed. The rest of the paper is organized as follows. Section 2 is dedicated to both modeling and design of the novel concept. Section 3 focuses on the implementation of the concept and discussion of the results achieved. Concluding remarks are formulated in Section 4. 2. The proposed ANN-based model This section presents the development process Figure 2: Backpropagation model of the of the proposed ANN-based image resizing proposed ANN based method model. A synoptic representation of the proposed In a neural network, the activation function has process is shown in Figure 1. The proposed ANN- the responsibility for transforming the summed based model for image compression consists of weighted input from the node into the activation the following steps: First, the camera captures the of the node or output for that input. In a neural original image [21-22]. Then the image is resized network, several types of activation functions are using the interpolation method. After that, the used. The proposed ANN based image resizing JPEG compression process takes place. The method uses sigmoid function. Sigmoid function compressed image is transmitted to the receiver is one type of mathematical function that has a via a radio module. On the receiving side, the characteristic "S"-shaped curve or sigmoid curve. image received via the radio module is subjected The sigmoid activation function has a to JPEG decompression. Then the next steps are mathematical form to choose an appropriate neural network model for 1 𝜎(𝑥) = (1+𝑒 −𝑥 ) (1) image resizing. There are different types of neural networks in data processing. These include: The sigmoid activation is shown in Fig. 5. It Convolutional Neural Network (CNN), Recurrent takes a real value and "squeezes" in the range from Neural Network (RNN), Artificial Neural 0 to 1. In particular, large negative numbers are Network (ANN), just to name a few. Amongst the equal to 0 and large positive numbers are equal to aforementioned types of neural networks, the 1. ANN type is selected to perform the image resizing process and insuring an efficient image recovery. Figure 3: Sigmoid activation function The main reason for using the sigmoid function is that it exists between (0 to 1). Figure 1: Example figure Synoptic representation Therefore, it is primarily used for models where it of the ANN-based model for image resizing has to assume probability as an output. As can be seen from Figure 2, the proposed method based-ANN works according to the 3. Performance validation the absolute error (in dB) is ex-pressed by equation (4). peakval2 For the validation of the proposed ANN-based PSNR = 10 log10 MSE (4) image resizing method, experimental results are Where peakval denotes the peak value and evaluated using Mean Squared Error (MSE), Root corresponds to the maximal in the image data. If Mean Square Error (RMSE), Peak Signal-to- it is an 8-bit unsigned integer data type, the Noise Ratio (PSNR) and Structural Similarity peakval is 255 [25]. Index Measure (SSIM) estimation methods. Structural similarity index measure (SSIM). Mean Square Error (MSE) is a commonly used The structural similarity index method is a model metric for the evaluation of the image quality. The based on this perception. The term structural data better image quality is obtained for MSE values refers to interconnected pixels or spatially closed closed to zero. The variance of the estimator pixels. This interconnected resolution points to a corresponds to the second moment of error. The number of important information about objects in standard deviation is deduced from the variance the field of images. Lighting masking is a term and is used to evaluate the uncertainty. The MSE where the distorted part of the image is less visible corresponds to the variance of the predictor in the at the edges of the image. Contrasting masking, on objective estimator. It has units of measurement the other hand, is a term that these distortions are equal to the square of the magnitude calculated as less visible in the image structure. The SSIM the variance. expressed in equation (5) is used to predict the Mean Squared Error (MSE) between two perceived quality of images and videos. It images, say g (x,y) and ĝ (x,y) is defined in measures the similarity between the two images: equation (2) (see also Ref. [23]) to assess the the original and the restored. absolute error. (2μ μ +c )(2σ x y 1 xy+c ) 2 1 SSIM(x, y) = (μ2 +μ 2 +c )(σ2 +σ2 +c ) x 100 (5) 𝑀𝑆𝐸 = 𝑀𝑁 ∑𝑀 𝑁 𝑛=0 ∑𝑚=1[𝑔 ̂(𝑛, 𝑚 − 𝑔(𝑛, 𝑚)]2 (2) x y 1 x y 2 Root-mean-square error (RMSE).The root- Where μx is the average of x and μy the mean-square deviation (RMSD) or root-mean- average of y; 𝜎𝑥2 stands for the variance of x and square error (RMSE) is used to measure the 𝜎𝑦2 the variance of y; 𝜎𝑥𝑦 denotes the covariance differences between values (e.g., sample of x and y ; 𝑐1 = (𝑘1 𝐿)2 , are two key parameters values/data) predicted by our model and the used to stabilize the division with weak values observed. This leads to the measurement of denominator; L is the dynamic range of the pixel- the accuracy used to attribute the differences in values (typically this is 2#bits per pixel − 1), k1 = the prediction errors of different predictors to the 0.01 and k 2 = 0.03 by default. exact variable [24]. As mentioned above, a local image dataset was If it is assumed that the estimated parameter also created in this study. The local data set was given in θ can be a predictor with respect to θ, then used for the comparison. Since this image data set the mean square error is actually the square root was created only recently and has not yet been of the mean square error. used by other scientists, interpolation methods for The determination of RMSE is expressed by resizing images were used for comparison. the following equation: Interpolation methods such as Nearest, Linear Area, Cubic, Lanczos4 were used for comparison. MSE(θ̂) = √MSE(θ̂) (3) The comparison to MSE is shown in Figure 4. Peak signal-to-noise ratio (PSNR). PSNR is used to calculate the ratio between the maximum possible signal power and the power of the distorting noise that affects the quality of its representation. This ratio between two images is computed in decibel form. The Peak signal-to- noise ratio is the most commonly used quality assessment technique to measure the quality of reconstruction of lossy image compression codecs. The signal is treated as the original data Figure 4: Comparison (based on MSE) of and the noise is the error caused by the experimental results of local images compression or distortion. The representation of Figure 4 shows the comparison (based on the MSE) of selected estimation methods. The worst result is obtained with the closest interpolation for the cubic method, 30.046 for the Lanczos4 method. The best result is obtained by the method method and 31.797 for the proposed method. This proposed in this work. Among the interpolation comparison witnesses the fact that based on the methods, the cubic interpolation is the best in PSNR metric the proposed method is better than terms of quality. For this reason, the method that the counterparts methods used for the benchmark. came closest to the proposed method was the Based on the SSIM metric, the proposed cubic interpolation method. method and its counterparts are applied to the The comparison of selected methods to RMSE local images and the obtained results are is depicted in Figure 5. We use four selected local compared and presented in Figure 7. images with five alternative interpolation methods. Figure 7: Comparison using SSIM of experimental results of local images Figure 5: Comparison using RMSE of The results of comparison of local images experimental results of local images between the proposed method and their The results of the local image comparison counterparts based on MSE, RMSE, PSNR and between the proposed method and other PSNR SSIM are presented in Figure 8. The quantitative methods are shown in Figure 6. When comparing representations of four selected local images with PSNR, a higher value is a better result and a lower five alternative interpolations methods (Nearest, value is a worse result. Linear, Area, Cubic, and Lanczos4) obtained based on MSE, RMSE, PSNR, SSIM are presented in Table 1. Figure 6: Comparison using PSNR of experimental results of local images Figure 8: Comparison of experimental results of As shown in Figure 6, the average PSNR value local images based on PSNR, RMSE, MSE and is 25.421 for the nearest method, 28.93 for the SSIM linear method, 28.47 for the area method, 30.667 Table 1 Comparison based on MSE, RMSE, PSNR and SSIM using a local image dataset Parameters Nearest Linear Area Cubic Lanczos4 Proposed PSNR 25.421 28.93 28.47 30.667 30.046 31.797 RMSE 14.091 9.452 9.94 7.899 8.564 6.864 MSE 211.79 96.17 105.8 69.27 82.341 51.550 SSIM 86.276 91 91.44 93.80 92.700 94.366 As shown in Figure 8 and Table 1, the average different methods. The result of the evaluation has values of the metrics, namely PSNR, RMSE, led to the following values: 25.421, 14.091, MSE and SSIM, are each evaluated using 211.79 and 86.276 using the Nearest method; 28.93, 9.452, 96.17 and 91 using the Linear Khwarizmi, TUIT, and Tashkent TV Tower. method; 28.47, 9.94, 105.8 and 91.44 using the Based on selected metrics, namely MSE, RMSE, Area method; 30.667, 7.899, 69.270, 93.807 using PSNR and SSIM, the developed method was Cubic method; 30,046, 8.564, 82.341 and 92.700 compared to non-adaptive image interpolation using the Lancsoz4 method; 31.797, 6.864, methods like Cubic, Area, Nearest Neighbor, 51.550, 94.366 using the proposed method. These Lanczos4 and Linear. The comparison clearly results clearly show that for each of the four showed that the proposed method outperforms metrics used for comparison, the proposed each of its counterparts. method outperforms each of its five other As an outlook, the following points of counterparts used for the benchmark. necessary importance are currently under investigation: Enrichment of the local data set with new images, as this could contribute to better results (e.g. improvement in the robustness of the method proposed in this work and in the image quality); Demonstration of the potential application of the proposed method in different fields of engineering. 5. References Figure 9: Local dataset: (a) Amir Temur; (b) [1] J.Xiao, X.Zou, Z.Liu, and X.Guo “Adaptive Muhammad al-Khwarizmi; (c) TUIT; (d) Tashkent Interpolation Algorithm for Real-time Image TV Tower Resizing,” First International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06), 2006, pp. 221–224. [2] V. Patel and K. Mistree, “A review on different image interpolation techniques for image enhancement,” Ijetae, vol. 3, no. 12, pp. 129–133, 2013. [3] S. Prasad Jaiswal, V. Jakhetiya, A. Kumar, and A.Kumar Tiwari, “A low complex a) b) c) context adaptive image interpolation Figure 10: Example of obtained comparison on algorithm for real-time applications,” 2012 local image IEEE I2MTC - Int. Instrum. Meas. Technol. As can be seen from Figure 10, a comparison Conf. Proc., no. May, pp. 969–972, 2012. of the image named Amir Temur, which is part of [4] C. J. Moses and D. Selvathi, “A Survey on the local image data set, is performed. Here, (a) Adaptive Image Interpolation based on represents the original image, (b) the result Quantitative Measures,” Asian J. Res. Soc. obtained based on the cubic method, which is said Sci. Humanit., vol. 6, no. 4, p. 341, 2016. to be the best among the interpolation methods, [5] M. N. Ahmed, B. E. Cooper, and S. T. Love, and (c) the result obtained based on the proposed “Adaptive image interpolation using a method. When comparing the images, the image multilayer neural network,” Appl. Artif. (c) obtained by the proposed method shows a result close to the original image (a). This clearly Neural Networks Image Process. VI, vol. shows that the proposed method is better than the 4305, pp. 30–38, 2001. other five counterparts. [6] M. M. Hadhoud, M. I. Dessouky, F. E. Abd El-Samie, and S. E. El-Khamy, “Adaptive image interpolation based on local activity 4. Conclusions and future work levels,” Natl. Radio Sci. Conf. NRSC, Proc., vol. 2003-Janua, no. May 2014, pp. C41– In this paper, we developed an ANN-based C48, 2003. adaptive image interpolation concept for image [7] A. K. Murmu, “Adpative image interpolation resizing and a local image dataset consisting of Adaptive image interpolation,” 2014. images such as Amir Temur, Muhammad al- [8] S. H. Hsieh and C. H. Chen, “Adaptive image [18] O. Rippel and L. Bourdev, “Real-time interpolation using probabilistic neural adaptive image compression,” 34th Int. network,” Expert Syst. Appl., vol. 36, no. 3 Conf. Mach. Learn. ICML 2017, vol. 6, pp. PART 2, pp. 6025–6029, 2009. 4457–4473, 2017. [9] J. Xiao, X. Zou, Z. Liu, and X. Guo, “A novel [19] J. Jin, “An adaptive image scaling algorithm adaptive interpolation algorithm for image based on continuous fraction interpolation resizing,” Int. J. Innov. Comput. Inf. Control, and multi-resolution hierarchy processing,” vol. 3, no. 6 A, pp. 1335–1345, 2007. Fractals, vol. 28, no. 8, pp. 1–13, 2020. [10] M. A. Latif, J. C. Chedjou and K. Kyamakya, [20] M. Sadiq and A. Aziz, “Adaptive Image "The paradigm of Non-linear oscillators in Resizing using Edge Contrasting,” Int. J. image processing," VXV International Trend Sci. Res. Dev., vol. 4, no. 6, pp. 1711– Symposium on Theoretical Engineering, 1716, 2020. 2009, pp. 1-5. [21] M. Arabboev, S. Begmatov, K. Nosirov, A. [11] S. Abbas, M. Irshad, and M. Z. Hussain, Shakhobiddinov, J. C. Chedjou and K. “Adaptive image interpolation technique Kyamakya, "Development of a Prototype of based on cubic trigonometric Bspline a Search and Rescue Robot Equipped with representation,” IET Image Process., vol. 12, Multiple Cameras," 2021 International no. 5, pp. 769–777, 2018. Conference on Information Science and [12] I. A. Gavrilov and K. K. Nosirov, “Quality Communications Technologies (ICISCT), Evaluation of Compressed Images Based on 2021, pp. 1-5 Prediction Errors of Pixel Values,” 2018 XIV [22] K. Nosirov, S. Begmatov and M. Arabboev, International Scientific-Technical "Analog Sensing and Leap Motion Integrated Conference on Actual Problems of Remote Controller for Search and Rescue Electronics Instrument Engineering Robot System," 2020 International (APEIE), 2018, pp. 33-36. Conference on Information Science and [13] M. S Malini and M. Patil, “Interpolation Communications Technologies (ICISCT), Techniques in Image Resampling,” Int. J. 2020, pp. 1-5 Eng. Technol., vol. 7, no. 3.34, p. 567, 2018. [23] J. Søgaard, L. Krasula, M. Shahid, D. Temel, [14] S. H. Mahajan and V. K. Harpale, “Adaptive K. Brunnström, and M. Razaak, and nonadaptive image interpolation “Applicability of existing objective metrics techniques,” Proc. - 1st Int. Conf. Comput. of perceptual quality for adaptive video Commun. Control Autom. ICCUBEA 2015, streaming,” IS T Int. Symp. Electron. pp. 772–775, 2015. Imaging Sci. Technol., pp. 1–7, 2016. [15] K. Nosirov, S. Begmatov and M. Arabboev, [24] U. Sara, M. Akter, and M. S. Uddin, “Image "Design Of A Model For Multi-Parametric Quality Assessment through FSIM, SSIM, Health Monitoring System," 2020 MSE and PSNR—A Comparative Study,” J. International Conference on Information Comput. Commun., vol. 07, no. 03, pp. 8–18, Science and Communications Technologies 2019. (ICISCT), 2020, pp. 1-5 [25] R. G. Deshpande, L. L. Ragha, and S. K. [16] K. Kyamakya, J. C. Chedjou, M. A. Latif and Sharma, “Video quality assessment through U. A. Khan, “A novel image processing PSNR estimation for different compression approach combining a 'coupled nonlinear standards,” Indones. J. Electr. Eng. Comput. oscillators'-based paradigm with cellular Sci., vol. 11, no. 3, pp. 918–924, 2018. neural networks for dynamic robust contrast enhancement,” 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010), 2010, pp. 1-7. [17] A. Puziy, I. Gavrilov, K. Nosirov and A. Akhmedova, “Efficiency Estimation Of Image Resizing Based On Interpolating Transformations,” 2019 International Conference on Information Science and Communications Technologies (ICISCT), 2019, pp. 1-5.