=Paper=
{{Paper
|id=Vol-3611/paper6
|storemode=property
|title=Development of a novel method of adaptive image interpolation for image resizing using artificial intelligence
|pdfUrl=https://ceur-ws.org/Vol-3611/paper6.pdf
|volume=Vol-3611
|authors=Mukhriddin Arabboev,Shohruh Begmatov,Khabibullo Nosirov,Jean Chamberlain Chedjou,Kyandoghere Kyamakya
|dblpUrl=https://dblp.org/rec/conf/ivus/ArabboevBNCK22
}}
==Development of a novel method of adaptive image interpolation for image resizing using artificial intelligence==
https://ceur-ws.org/Vol-3611/paper6.pdf
Development of a novel method of adaptive image
interpolation for image resizing using artificial intelligence
Mukhriddin Arabboev1, Shohruh Begmatov1, Khabibullo Nosirov1, Jean Chamberlain
Chedjou2 and Kyandoghere Kyamakya2
1
Tashkent University of Information Technologies named after Muhammad al-Khwarizmi, 108 Amir Temur Av.,
Tashkent, 100084, Uzbekistan
2
University of Klagenfurt, Universitätsstraße 65-67, 9020 Klagenfurt am Wörthersee, Austria
Abstract
In this paper, we develop an artificial neural network (ANN)-based adaptive image
interpolation method for image resizing. A local image dataset is also created, consisting of
images with names such as Amir Temur, Muhammad al-Khwarizmi, TUIT and the Tashkent
TV Tower. The proposed adaptive image interpolation method based on artificial neural
networks is compared with non-adaptive image interpolation methods such as cubic, area,
nearest neighbor, lanczos4 and linear using a local image data set. The comparison is based on
assessment methods such as Mean Squared Error (MSE), Root Mean Square Error (RMSE),
Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The
comparison clearly shows that the proposed method outperforms its counterparts considered in
this work.
Keywords 1
Image interpolation, ANN, image resizing, local dataset
1. Introduction larger than those of other classical interpolation
algorithms. The proposed algorithm implements
image interpolation with high efficiency and is
In today’s age of digital technology, digital
particularly well suited for real-time image
images have become an integral part of our lives.
resizing. Various image interpolation techniques
Image interpolation methods are widely used in
for image enhancement are discussed in [2]. An
digital image processing. Image interpolation
overview of different interpolation techniques
methods fall into two main types: adaptive and
such as Nearest Neighbor, Bilinear, Bicubic, New
non-adaptive. A number of important studies on
Edge-Directed Interpolation (NEDI), Data-
image interpolation methods have been carried
Dependent Triangulation (DDT) and Iterative
out in the last few decades. In [1], it is presented
Curvature-Based Interpolation (ICBI) is given.
an adaptive image resizing algorithm based on the
Sunil et al. [3] propose a computationally simple
Newton interpolation function. Experimental
interpolation algorithm. In their algorithm, the
results show that the visual effect of their
unknown pixels are categorized into different bins
procedure surpasses that of bicubic interpolation
depending on the property of the neighboring
when resizing images, and the PSNR values of the
pixels (activity level) and for each bin fixed
resized image by their proposed algorithm are
prediction parameters are used for prediction. A
IVUS 2022: 27th International Conference on Information
Technology, May 12, 2022, Kaunas, Lithuania
EMAIL: mukhriddin.9207@gmail.com (M. Arabboev);
bek.shohruh@gmail.com (S. Begmatov);
n.khabibullo1990@gmail.com (K. Nosirov); jean.chedjou@aau.at
(J. C. Chedjou); kyandoghere.kyamakya@aau.at (K. Kyamakya)
© 2022 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
different set of fixed predictors is presented for processing tasks. In [11] develop an adaptive
both smooth and edgy/angular images. A image interpolation technique based on a cubic
modified algorithm is also proposed in which the trigonometric B-spline representation. Image
selection of the prediction parameter is done on a quality metrics such as SSIM, MS-SSIM and
block-by-block basis rather than on a frame-by- FSIM along with the classic PSNR are used to
frame basis. Their proposed algorithm gives much examine the quality of interpolated digital images.
better qualitative and quantitative performance In [12], it is considered the metric objective
compared to other computationally simple quality assessment of compressed TV images
interpolation algorithms. Non-adaptive image based on the prediction error values of sums of
interpolation algorithms based on quantitative pixels of the original and decoded images. In [13],
measures are examined in [4]. a comparative study of different resampling
The survey analyzes the properties of various techniques like Cubic Splines, Nearest Neighbor,
non-adaptive interpolation techniques based on Cubic Convolution and Linear Interpolation is
their PSNR of the interpolated image and their given, which can be used as detectors for a altered
computational complexity. The applicability of image containing resampled parts/portions. In
these techniques in real-time applications is also [14-15], an overview of different adaptive and
examined. Based on the evaluation, it can be non-adaptive image interpolation techniques is
suggested that first-order polynomial given and a comparison based on their
convolutional interpolation (FOPCI) is suitable performance parameter (i.e. H. PSNR) is
for real-time applications due to its better PSNR performed. In [16], it is conducted a systematic
and low computational cost, and the performance discussion of both pros and cons of CNN based
of FOPCI can be improved by using appropriate and coupled nonlinear oscillators' based
filters. A new technique for segmenting document approaches for image contrast enhancement. In
images is presented in [5]. In [6] an adaptive [17], it is presented an efficiency estimation of
technique for image interpolation using the digital image resizing using various image
bilinear, the bicubic and the cubic spline method interpolation methods, such as Bicubic, B-Spline,
is proposed by adaptively weighting the pixels Mitchell, Lanczos. It is also shown the
involved in the interpolation process. The experimental results of quality changing after
adaptive technique is compared to the image reduction and restoration. In [18], a
conventional interpolation technique and the machine learning based approach for lossy image
distorted/warped distance interpolation technique. compression is presented that outperforms all
Another interesting study can be found in [7]. existing codecs while running in real time.
An adaptive interpolation technique based on the According to the proposed algorithm, files are
Newtonian forward difference is developed. The produced that are 2.5 times smaller than JPEG and
forward difference provides a measure of the JPEG 2000, 2 times smaller than WebP and 1.7
goodness of grouping pixels around the target times smaller than BPG on datasets of generic
pixel for interpolation. In [8], an image images across all quality levels. In [19], an
interpolation model based on a probabilistic adaptive image scaling algorithm based on
neural network (PNN) is proposed. The method continuous fractional interpolation and
automatically sets and maintains alignment hierarchical processing with multiple resolutions
settings for various smooth image areas, is proposed. The algorithm achieves a smooth,
considering the properties of a plane (flat area) high-order transition between pixels in the same
and accuracy (edge area) model. feature region, and can also modify the pixels of
In [9], a novel adaptive interpolation algorithm the image adaptively. Finally, in [20] the adaptive
based on Newton's polynomial is developed to image resizing using edge contrasting concept is
improve the limitation of the traditional image presented. The concept is tested with more than
resizing algorithm. The efficiency of the proposed 100 frames and found to have far superior
method is compared to that of the traditional performance in terms of PSNR and MSE scores.
Matlab image resizing toolbox. In [10], it is Overall, the overview of the previous
realised an image contrast enhancement by using contribution on image interpolation and resizing
nonlinear oscillatory theory. In the study, it is witnesses the tremendous attention that has been
studied two different uncoupled networks based devoted to the development of various methods
on nonlinear oscillators. According to the and algorithms over the last few decades.
research, results show a possible effective area of However, little attention has been paid to
application of nonlinear oscillators for image techniques based on neural networks. This paper
contributes to the enrichment of the literature by principle of 2 x 2 to 3 x 3. The model in figure 2
developing a novel, robust, and efficient ANN- encompasses two hidden layers, 12 inputs and 27
based adaptive image interpolation method for outputs. The backpropagation model was used to
image resizing. The advantage of the developed develop the proposed method. Backpropagation is
method lies in the possibility of efficiently an algorithm that is widely used for training
maintaining the image quality. Furthermore, the feedforward neural networks. The main purpose
developed method has concrete potential of the backpropagation model is to correct output
applications such as the efficient transmission of errors.
high-quality images at high speed.
The rest of the paper is organized as follows.
Section 2 is dedicated to both modeling and
design of the novel concept. Section 3 focuses on
the implementation of the concept and discussion
of the results achieved. Concluding remarks are
formulated in Section 4.
2. The proposed ANN-based model
This section presents the development process Figure 2: Backpropagation model of the
of the proposed ANN-based image resizing proposed ANN based method
model. A synoptic representation of the proposed In a neural network, the activation function has
process is shown in Figure 1. The proposed ANN- the responsibility for transforming the summed
based model for image compression consists of weighted input from the node into the activation
the following steps: First, the camera captures the of the node or output for that input. In a neural
original image [21-22]. Then the image is resized network, several types of activation functions are
using the interpolation method. After that, the used. The proposed ANN based image resizing
JPEG compression process takes place. The method uses sigmoid function. Sigmoid function
compressed image is transmitted to the receiver is one type of mathematical function that has a
via a radio module. On the receiving side, the characteristic "S"-shaped curve or sigmoid curve.
image received via the radio module is subjected The sigmoid activation function has a
to JPEG decompression. Then the next steps are mathematical form
to choose an appropriate neural network model for 1
𝜎(𝑥) = (1+𝑒 −𝑥 ) (1)
image resizing. There are different types of neural
networks in data processing. These include: The sigmoid activation is shown in Fig. 5. It
Convolutional Neural Network (CNN), Recurrent takes a real value and "squeezes" in the range from
Neural Network (RNN), Artificial Neural 0 to 1. In particular, large negative numbers are
Network (ANN), just to name a few. Amongst the equal to 0 and large positive numbers are equal to
aforementioned types of neural networks, the 1.
ANN type is selected to perform the image
resizing process and insuring an efficient image
recovery.
Figure 3: Sigmoid activation function
The main reason for using the sigmoid
function is that it exists between (0 to 1).
Figure 1: Example figure Synoptic representation Therefore, it is primarily used for models where it
of the ANN-based model for image resizing has to assume probability as an output.
As can be seen from Figure 2, the proposed
method based-ANN works according to the
3. Performance validation the absolute error (in dB) is ex-pressed by
equation (4).
peakval2
For the validation of the proposed ANN-based PSNR = 10 log10 MSE (4)
image resizing method, experimental results are Where peakval denotes the peak value and
evaluated using Mean Squared Error (MSE), Root corresponds to the maximal in the image data. If
Mean Square Error (RMSE), Peak Signal-to- it is an 8-bit unsigned integer data type, the
Noise Ratio (PSNR) and Structural Similarity peakval is 255 [25].
Index Measure (SSIM) estimation methods. Structural similarity index measure (SSIM).
Mean Square Error (MSE) is a commonly used The structural similarity index method is a model
metric for the evaluation of the image quality. The based on this perception. The term structural data
better image quality is obtained for MSE values refers to interconnected pixels or spatially closed
closed to zero. The variance of the estimator pixels. This interconnected resolution points to a
corresponds to the second moment of error. The number of important information about objects in
standard deviation is deduced from the variance the field of images. Lighting masking is a term
and is used to evaluate the uncertainty. The MSE where the distorted part of the image is less visible
corresponds to the variance of the predictor in the at the edges of the image. Contrasting masking, on
objective estimator. It has units of measurement the other hand, is a term that these distortions are
equal to the square of the magnitude calculated as less visible in the image structure. The SSIM
the variance. expressed in equation (5) is used to predict the
Mean Squared Error (MSE) between two perceived quality of images and videos. It
images, say g (x,y) and ĝ (x,y) is defined in measures the similarity between the two images:
equation (2) (see also Ref. [23]) to assess the the original and the restored.
absolute error. (2μ μ +c )(2σ
x y 1 xy+c )
2
1 SSIM(x, y) = (μ2 +μ 2 +c )(σ2 +σ2 +c ) x 100 (5)
𝑀𝑆𝐸 = 𝑀𝑁 ∑𝑀 𝑁
𝑛=0 ∑𝑚=1[𝑔 ̂(𝑛, 𝑚 − 𝑔(𝑛, 𝑚)]2 (2) x y 1 x y 2
Root-mean-square error (RMSE).The root- Where μx is the average of x and μy the
mean-square deviation (RMSD) or root-mean- average of y; 𝜎𝑥2 stands for the variance of x and
square error (RMSE) is used to measure the 𝜎𝑦2 the variance of y; 𝜎𝑥𝑦 denotes the covariance
differences between values (e.g., sample of x and y ; 𝑐1 = (𝑘1 𝐿)2 , are two key parameters
values/data) predicted by our model and the used to stabilize the division with weak
values observed. This leads to the measurement of denominator; L is the dynamic range of the pixel-
the accuracy used to attribute the differences in values (typically this is 2#bits per pixel − 1), k1 =
the prediction errors of different predictors to the 0.01 and k 2 = 0.03 by default.
exact variable [24]. As mentioned above, a local image dataset was
If it is assumed that the estimated parameter also created in this study. The local data set was
given in θ can be a predictor with respect to θ, then used for the comparison. Since this image data set
the mean square error is actually the square root was created only recently and has not yet been
of the mean square error. used by other scientists, interpolation methods for
The determination of RMSE is expressed by resizing images were used for comparison.
the following equation: Interpolation methods such as Nearest, Linear
Area, Cubic, Lanczos4 were used for comparison.
MSE(θ̂) = √MSE(θ̂) (3) The comparison to MSE is shown in Figure 4.
Peak signal-to-noise ratio (PSNR). PSNR is
used to calculate the ratio between the maximum
possible signal power and the power of the
distorting noise that affects the quality of its
representation. This ratio between two images is
computed in decibel form. The Peak signal-to-
noise ratio is the most commonly used quality
assessment technique to measure the quality of
reconstruction of lossy image compression
codecs. The signal is treated as the original data Figure 4: Comparison (based on MSE) of
and the noise is the error caused by the experimental results of local images
compression or distortion. The representation of Figure 4 shows the comparison (based on the
MSE) of selected estimation methods. The worst
result is obtained with the closest interpolation for the cubic method, 30.046 for the Lanczos4
method. The best result is obtained by the method method and 31.797 for the proposed method. This
proposed in this work. Among the interpolation comparison witnesses the fact that based on the
methods, the cubic interpolation is the best in PSNR metric the proposed method is better than
terms of quality. For this reason, the method that the counterparts methods used for the benchmark.
came closest to the proposed method was the Based on the SSIM metric, the proposed
cubic interpolation method. method and its counterparts are applied to the
The comparison of selected methods to RMSE local images and the obtained results are
is depicted in Figure 5. We use four selected local compared and presented in Figure 7.
images with five alternative interpolation
methods.
Figure 7: Comparison using SSIM of experimental
results of local images
Figure 5: Comparison using RMSE of The results of comparison of local images
experimental results of local images between the proposed method and their
The results of the local image comparison counterparts based on MSE, RMSE, PSNR and
between the proposed method and other PSNR SSIM are presented in Figure 8. The quantitative
methods are shown in Figure 6. When comparing representations of four selected local images with
PSNR, a higher value is a better result and a lower five alternative interpolations methods (Nearest,
value is a worse result. Linear, Area, Cubic, and Lanczos4) obtained
based on MSE, RMSE, PSNR, SSIM are
presented in Table 1.
Figure 6: Comparison using PSNR of experimental
results of local images
Figure 8: Comparison of experimental results of
As shown in Figure 6, the average PSNR value
local images based on PSNR, RMSE, MSE and
is 25.421 for the nearest method, 28.93 for the
SSIM
linear method, 28.47 for the area method, 30.667
Table 1
Comparison based on MSE, RMSE, PSNR and SSIM using a local image dataset
Parameters Nearest Linear Area Cubic Lanczos4 Proposed
PSNR 25.421 28.93 28.47 30.667 30.046 31.797
RMSE 14.091 9.452 9.94 7.899 8.564 6.864
MSE 211.79 96.17 105.8 69.27 82.341 51.550
SSIM 86.276 91 91.44 93.80 92.700 94.366
As shown in Figure 8 and Table 1, the average different methods. The result of the evaluation has
values of the metrics, namely PSNR, RMSE, led to the following values: 25.421, 14.091,
MSE and SSIM, are each evaluated using 211.79 and 86.276 using the Nearest method;
28.93, 9.452, 96.17 and 91 using the Linear Khwarizmi, TUIT, and Tashkent TV Tower.
method; 28.47, 9.94, 105.8 and 91.44 using the Based on selected metrics, namely MSE, RMSE,
Area method; 30.667, 7.899, 69.270, 93.807 using PSNR and SSIM, the developed method was
Cubic method; 30,046, 8.564, 82.341 and 92.700 compared to non-adaptive image interpolation
using the Lancsoz4 method; 31.797, 6.864, methods like Cubic, Area, Nearest Neighbor,
51.550, 94.366 using the proposed method. These Lanczos4 and Linear. The comparison clearly
results clearly show that for each of the four showed that the proposed method outperforms
metrics used for comparison, the proposed each of its counterparts.
method outperforms each of its five other
As an outlook, the following points of
counterparts used for the benchmark.
necessary importance are currently under
investigation: Enrichment of the local data set
with new images, as this could contribute to better
results (e.g. improvement in the robustness of the
method proposed in this work and in the image
quality); Demonstration of the potential
application of the proposed method in different
fields of engineering.
5. References
Figure 9: Local dataset: (a) Amir Temur; (b) [1] J.Xiao, X.Zou, Z.Liu, and X.Guo “Adaptive
Muhammad al-Khwarizmi; (c) TUIT; (d) Tashkent Interpolation Algorithm for Real-time Image
TV Tower Resizing,” First International Conference on
Innovative Computing, Information and
Control - Volume I (ICICIC'06), 2006, pp.
221–224.
[2] V. Patel and K. Mistree, “A review on
different image interpolation techniques for
image enhancement,” Ijetae, vol. 3, no. 12,
pp. 129–133, 2013.
[3] S. Prasad Jaiswal, V. Jakhetiya, A. Kumar,
and A.Kumar Tiwari, “A low complex
a) b) c) context adaptive image interpolation
Figure 10: Example of obtained comparison on algorithm for real-time applications,” 2012
local image IEEE I2MTC - Int. Instrum. Meas. Technol.
As can be seen from Figure 10, a comparison Conf. Proc., no. May, pp. 969–972, 2012.
of the image named Amir Temur, which is part of [4] C. J. Moses and D. Selvathi, “A Survey on
the local image data set, is performed. Here, (a) Adaptive Image Interpolation based on
represents the original image, (b) the result Quantitative Measures,” Asian J. Res. Soc.
obtained based on the cubic method, which is said Sci. Humanit., vol. 6, no. 4, p. 341, 2016.
to be the best among the interpolation methods, [5] M. N. Ahmed, B. E. Cooper, and S. T. Love,
and (c) the result obtained based on the proposed
“Adaptive image interpolation using a
method. When comparing the images, the image
multilayer neural network,” Appl. Artif.
(c) obtained by the proposed method shows a
result close to the original image (a). This clearly Neural Networks Image Process. VI, vol.
shows that the proposed method is better than the 4305, pp. 30–38, 2001.
other five counterparts. [6] M. M. Hadhoud, M. I. Dessouky, F. E. Abd
El-Samie, and S. E. El-Khamy, “Adaptive
image interpolation based on local activity
4. Conclusions and future work levels,” Natl. Radio Sci. Conf. NRSC, Proc.,
vol. 2003-Janua, no. May 2014, pp. C41–
In this paper, we developed an ANN-based C48, 2003.
adaptive image interpolation concept for image [7] A. K. Murmu, “Adpative image interpolation
resizing and a local image dataset consisting of Adaptive image interpolation,” 2014.
images such as Amir Temur, Muhammad al-
[8] S. H. Hsieh and C. H. Chen, “Adaptive image [18] O. Rippel and L. Bourdev, “Real-time
interpolation using probabilistic neural adaptive image compression,” 34th Int.
network,” Expert Syst. Appl., vol. 36, no. 3 Conf. Mach. Learn. ICML 2017, vol. 6, pp.
PART 2, pp. 6025–6029, 2009. 4457–4473, 2017.
[9] J. Xiao, X. Zou, Z. Liu, and X. Guo, “A novel [19] J. Jin, “An adaptive image scaling algorithm
adaptive interpolation algorithm for image based on continuous fraction interpolation
resizing,” Int. J. Innov. Comput. Inf. Control, and multi-resolution hierarchy processing,”
vol. 3, no. 6 A, pp. 1335–1345, 2007. Fractals, vol. 28, no. 8, pp. 1–13, 2020.
[10] M. A. Latif, J. C. Chedjou and K. Kyamakya, [20] M. Sadiq and A. Aziz, “Adaptive Image
"The paradigm of Non-linear oscillators in Resizing using Edge Contrasting,” Int. J.
image processing," VXV International Trend Sci. Res. Dev., vol. 4, no. 6, pp. 1711–
Symposium on Theoretical Engineering, 1716, 2020.
2009, pp. 1-5. [21] M. Arabboev, S. Begmatov, K. Nosirov, A.
[11] S. Abbas, M. Irshad, and M. Z. Hussain, Shakhobiddinov, J. C. Chedjou and K.
“Adaptive image interpolation technique Kyamakya, "Development of a Prototype of
based on cubic trigonometric Bspline a Search and Rescue Robot Equipped with
representation,” IET Image Process., vol. 12, Multiple Cameras," 2021 International
no. 5, pp. 769–777, 2018. Conference on Information Science and
[12] I. A. Gavrilov and K. K. Nosirov, “Quality Communications Technologies (ICISCT),
Evaluation of Compressed Images Based on 2021, pp. 1-5
Prediction Errors of Pixel Values,” 2018 XIV [22] K. Nosirov, S. Begmatov and M. Arabboev,
International Scientific-Technical "Analog Sensing and Leap Motion Integrated
Conference on Actual Problems of Remote Controller for Search and Rescue
Electronics Instrument Engineering Robot System," 2020 International
(APEIE), 2018, pp. 33-36. Conference on Information Science and
[13] M. S Malini and M. Patil, “Interpolation Communications Technologies (ICISCT),
Techniques in Image Resampling,” Int. J. 2020, pp. 1-5
Eng. Technol., vol. 7, no. 3.34, p. 567, 2018. [23] J. Søgaard, L. Krasula, M. Shahid, D. Temel,
[14] S. H. Mahajan and V. K. Harpale, “Adaptive K. Brunnström, and M. Razaak,
and nonadaptive image interpolation “Applicability of existing objective metrics
techniques,” Proc. - 1st Int. Conf. Comput. of perceptual quality for adaptive video
Commun. Control Autom. ICCUBEA 2015, streaming,” IS T Int. Symp. Electron.
pp. 772–775, 2015. Imaging Sci. Technol., pp. 1–7, 2016.
[15] K. Nosirov, S. Begmatov and M. Arabboev, [24] U. Sara, M. Akter, and M. S. Uddin, “Image
"Design Of A Model For Multi-Parametric Quality Assessment through FSIM, SSIM,
Health Monitoring System," 2020 MSE and PSNR—A Comparative Study,” J.
International Conference on Information Comput. Commun., vol. 07, no. 03, pp. 8–18,
Science and Communications Technologies 2019.
(ICISCT), 2020, pp. 1-5 [25] R. G. Deshpande, L. L. Ragha, and S. K.
[16] K. Kyamakya, J. C. Chedjou, M. A. Latif and Sharma, “Video quality assessment through
U. A. Khan, “A novel image processing PSNR estimation for different compression
approach combining a 'coupled nonlinear standards,” Indones. J. Electr. Eng. Comput.
oscillators'-based paradigm with cellular Sci., vol. 11, no. 3, pp. 918–924, 2018.
neural networks for dynamic robust contrast
enhancement,” 2010 12th International
Workshop on Cellular Nanoscale Networks
and their Applications (CNNA 2010), 2010,
pp. 1-7.
[17] A. Puziy, I. Gavrilov, K. Nosirov and A.
Akhmedova, “Efficiency Estimation Of
Image Resizing Based On Interpolating
Transformations,” 2019 International
Conference on Information Science and
Communications Technologies (ICISCT),
2019, pp. 1-5.