Dependence research of the communication channels characteristics influence on the videoconferencing quality* Dmitry Tyuhtyaev1[0000-0003-4594-5959], Igor Sharkevich2[0000-0002-3472-4838] and Nadezhda Ermakova1[0000-0001-6909-5536] 1 Volgograd State University, 100, Prospect Universitetsky, Volgograd, 400062, Russian Federation 2 Plekhanov Russian University of Economics, Volgograd Branch, 11, Volgodonskaya Street, Volgograd, 400066 , Russian Federation tyuhtyaev.dmitriy@volsu.Ru Abstract. The purpose of the study was to determine the dependence of the quality of rendering videoconferencing services on the characteristics of communication channels. Under the characteristics of communication channels, we considered such characteristics as: delay, jitter, percentage of lost packets. The article discusses subjective and objective methods for assessing video. The peak signal to noise ratio metric and the MSU Video Quality Measurement Tool software, created by the computer graphics laboratory of the Moscow State University, was used as an objective video assessment method. For the subjective method, the Double Stimulus Continuous Quality Scale method was used. The article presents the results of expert assessment and the results of an objective assessment using the peak signal to noise ratio metric. As a result of the work, it was revealed that if the characteristics of communication channels do not meet the minimum ITU requirements, objective assessments were at a very low level. It was also noted that the results of objective assessments are not always correlated with the result of subjective assessments using the PSNR metric. Keywords: ВКС, PSNR, communication channel characteristics, delay, jitter. 1 Introduction 1.1 Videoconferencing Videoconferencing systems, hereinafter videoconferencing, have been actively used in the last few years in the field of business, education and medicine. In early 2020, there was an outbreak of the SARS-CoV-2 virus. As a result, many areas of our lives have changed dramatically. Currently, a large number of employees work remotely * Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). using video conferencing and webinar systems. It is also worth noting that almost the entire field of education has switched to a remote mode of work. The dynamics of the market for means of collective work is presented in Figure 1. [1] Statistics on the popularity of video conferencing in each of the sectors, according to the analytical company OSP Data, is presented in Figure 2 [2]. Fig. 1. Dynamics of the market for means of collective work (including videoconferencing). Fig. 2. Distribution of video conferencing by industry (source: OSP Data). In connection with the rapid development of videoconferencing systems, a large load of communication channels during a pandemic, the question of the quality provision of videoconferencing services and the influence of the characteristics of communication channels on the quality of videoconferencing is very acute. Reducing the parameters of communication channels leads to degradation of the video and audio track, as well as various negative effects. These effects include the lag of the audio track from the video sequence, as well as its stuttering, a decrease in the quality of the video sequence or its complete loss for a certain time. 2 Materials and methods The quality of video transmission depends on various factors, such as packet loss due to buffer overflows on processing network devices, network latency, jitter, and others. Today, there are subjective and objective methods for assessing the quality of video transmission. The subjective testing method is the use of live experts to determine the quality of the video. The results of the assessments are obtained after averaging all the values of the experts; this assessment is called MOS (Mean Opinion Score). Within the framework of Recommendation ITU-R BT.500-12 [3], additional methods are introduced to improve the accuracy of the assessment of video transmission quality by a group of experts. During the work, the Double Stimulus Continuous Quality Scale (DSCQS) method was used. This method consists in demonstrating to an expert a pair of video sequences received from one source. One video is original, unprocessed. While the second video is modified after passing through the communication channels. After that, the expert is asked to rate the quality of both video sequences on a 5-point scale. At the next stage, the average value for the group of experts is calculated [4]. The objective testing method is the use of special algorithms for video analysis. Objective methods, unlike subjective ones, do not give a complete picture of what a person sees on the screen. At the current time, there is no single recommendation for the use of any single metric. In the course of this work, the PSNR (peak signal to noise ratio) metric was used for an objective study. The choice of this metric is due to its widespread use in the scientific community and its use as a basic metric for assessing the loss of quality. Like any metric, this metric has its advantages and disadvantages. PSNR does not measure all video-specific parameters, as the fidelity of the image is constantly changing depending on the visual complexity of the image, the available bit rate and even the compression method. Thus, PSNR cannot determine how noticeable these distortions will be for the user [5]. The peak signal-to-noise ratio is easiest to determine through the root-mean-square error (RMSE or MSE, Mean Square Error), which for two monochrome images R and Q of size i × j, one of which is considered a noisy approximation of the other, is calculated as follows: MSE    (R  Q ) i j ij ij 2 (1) ij A2 PSNR  10 log( ) (2) MSE Where, A is the maximum value accepted by the image pixel. When the pixels are 8 bits wide, A = 255. To study the influence of the characteristics of communication channels on the quality of video conferencing, the stand shown in Figure 3 was used. Experimental stand included a server with installed server software for video conferencing, a PC of a video conferencing participant, and PC running WANem software. WANem is a WAN emulator that runs under the Linux operating system. On a PC server, the server and client parts of the video conferencing system were installed. The client part of the video conferencing was installed on the client's computer [6]. Fig. 3. Experimental stand. The TrueConf system was used as a video conferencing system. The choice of this system is due to its wide distribution in the Russian market, as well as the availability of a free license for a limited number of customers. The Logitech HD Pro Webcam C910 and HD Webcam C270 cameras on the server and client sides, respectively, were used as web cameras for making video calls. The experiment consisted of creating a common room on a server and connecting both clients to it. In the WANem software, the characteristics of the communication channel were artificially changed: throughput, delay, and jitter and packet loss [6]. The WANem architecture is described in detail in [7] by the creators of the emulator. For objective modeling of communication channels used in the work, it is necessary to know the characteristics of communication channels of real networks. The answer to the question about the average bandwidth of the Internet channel in the Russian Federation is given by the research of the company Yandex "Development of the Internet in the regions of Russia-2016" [8]. The data are presented in Figure 4. Fig. 4. The state of fixed broadband access in Russia. Let us describe the technique used to obtain the characteristics of the communication channels, such as: delay, jitter, and the percentage of lost packets. At the first stage, to measure these parameters, ICMP packets were sent within 10 minutes to various IP addresses located in different geographic locations [9]. To determine the geographic location of a place by IP address, the Internet service was used [10]. At the second stage, the values of the required channel parameters were calculated. For this, the formulas presented below were used. The average delay was calculated using the formula: 1 N   i N i 1 (3) Where N the number of delivered packets is,  i is the delay for the i th packet. Jitter was defined as:  ( )   max   min (4) Where: 1 N  max    N  (   ) 2 i 1 i (5) 1 N  max    N  (   ) 2 i 1 i (6) The packet loss ratio was calculated as the ratio of the number of undelivered packets to the total number of transmitted packets: N loss K loss  (7) N sum As a result, we got the following data (Table 1). Table 1. Characteristics of communication channels obtained during the experiment. Packet loss A source Appointment Delay, ms Jitter, ms rate,% Brazil, 358.19 134.03 0.33 200.19.159.34 Spain, 102.32 35.86 0 138.100.12.149 Canada, 285.33 148.37 1.67 142.103.2.1 United States, 77.17 26.56 0 Russia, 104.108.183.93 Volgograd Germany, 103.82 16.03 0 85.13.131.134 Italy, 109.31 25.84 0 195.110.124.134 Moscow, 42.10 66. 76 0 77.88.55.55 Vladivostok, 193.02 27.17 0 62.76.6.35 The results will be used to simulate communication links in the WANem environment. If we refer to the ITU-T recommendations and the Order of the Ministry of Information Technologies and Communications of the Russian Federation No. 113 dated September 27, 2007, we can see that the values of Table 1, such as jitter and packet loss rate to destination points Brazil and Canada, do not meet the minimum requirements. The rest of the destinations are fully compliant with VoIP and video conferencing standards. First, a reference video was recorded at a bandwidth of 10 Mb / s without delay, jitter and packet loss. Then the videos were recorded, in which delay and packet loss were introduced, based on the ITU-T QoS recommendations. To carry out the PSNR metric, the special software MSU Video Quality Measurement Tool was used, created by the computer graphics laboratory of the VMiK Moscow State University. MSU Video Quality Measurement Tool is a program that allows you to use several metrics: the more common ones - SSIM, PSNR, VQM, Delta, and specially developed metrics for measuring blockiness and sharpness. The program is shareware; there is a PRO version for commercial use [11]. To conduct a subjective assessment, the experts were shown test videos, after which they gave marks. 3 Results To evaluate the video from our experiment, an expert survey was conducted among teachers and university students. The average score is shown in Table 2. Table 2. Average video rating by experts. Average subjective score Track Logitech Pro HD Logitech HD Webcam Webcam C910 C270 Brazil 1.93 2.8 Germany 3.6 4.4 Far East 2.46 4.26 Spain 3.73 4 Italy 3 3.53 Canada 2 1.93 Moscow 3.53 4 USA 3.6 4.06 Threshold 3.46 2.53 Beyond the threshold 1.53 1.4 We present the results of an objective assessment using the PSNR metric (Tables 3 and 4). Table 3. PSNR Logitech HD Pro Webcam C910. Metrics PSNR RGB Track Subjective assessment RRGB GRGB BRGB Brazil 13.87577 13.92380 14.07943 1.93 Germany 33.86976 34.23087 33.55716 3.6 Far East 15.06542 15.84164 15.95278 2.46 Spain 31.19690 31.27460 30.97815 3.73 Italy 26.14936 26.30347 26.11293 3 Canada 19.09947 19.16009 19.05701 2 Moscow 25.92070 26.20447 26.19976 3.53 USA 34.46897 34.77288 33.98076 3.6 Threshold 28.28313 28.43001 28.12028 3.46 Beyond the threshold 13.87320 13.88226 14.19560 1.53 Table 4. PSNR Logitech HD Webcam C270. Metrics PSNR RGB Track Subjective assessment RRGB GRGB BRGB Brazil 15.38073 15.60127 16.48878 2.8 Germany 23.74818 23.81598 23.97613 4.4 Far East 25.26531 25.37411 25.46511 4.26 Spain 27.47735 27.67116 27.71880 4 Italy 23.74838 23.93667 23.00662 3.53 Canada 19.79473 19.88613 20.26584 1.93 Moscow 25.48009 25.66989 25.76536 4 USA 24.23102 24.33977 24.54167 4.06 Threshold 12.84001 13.01919 14.03862 2.53 Beyond the threshold 10.52619 10.66404 11.68362 1.4 Based on the recommendations of ITU-T and Order 113 of the Ministry of Information Technologies and Communications of the Russian Federation, below are examples of frames of the original video and video received at the other end and processed with the PSNR RGB metric. Used footage from a video conference with Spain. Fig. 5. A still from the original video and PSNR-processed video for Logitech HD Pro Webcam C910. Fig. 6. A still from the original video and video processed by PSNR for Logitech HD Webcam C270. As seen in Figures 5 and 6, green and blue colors indicate strong video delays and changes in image quality relative to the first. Minor changes are shown in red and no changes are indicated in black. 4 Discussion As can be seen from Table 2, with the characteristics of communication channels that do not meet the standards of the ITU and the Ministry of Communications, one cannot count on a positive assessment of the quality of videoconferencing by users. At the same time, there is a tendency that the video taken with the simpler camera model C270 with 720p resolution was rated by users higher than the video taken with the C910 with 1080p resolution. Also, three of the experts spoke negatively about autofocus in the Logitech Pro HD Webcam C910. In their opinion, autofocus degrades the image quality in video. Thus, the camera model with the best technical characteristics is not always able to provide a higher subjective assessment of the user, however, for the final conclusion; a study with a large number of camera models is required. From Table 3 and Table 4, it can be seen that despite the best PSNR metrics for the C910, the subjective rating for it is worse than for the C270. This fact can be explained by the presence of autofocus, although only three of the interviewed experts expressed their direct dissatisfaction with its work. 5 Conclusion From the experiments carried out, it can be concluded that the characteristics of communication channels have a direct impact on the quality of video conferencing services. From this, the following recommendations can be made that can be used in the implementation and operation of videoconferencing in corporate networks: ─ Before organizing a video communication session, measure the characteristics of the communication channel; ─ If the characteristics of the communication channel do not meet the standards, it is not necessary to count on a positive subjective assessment; ─ If the channel characteristics do not meet the quality of service standards for this type of traffic, lease dedicated channels with guaranteed quality of service from telecom operators; ─ The use of video cameras or video terminals that provide a higher frame rate and higher resolution may not always have a positive effect on the subjective assessment; ─ During the operation of the videoconferencing system, it would be useful to accumulate a statistical subjective assessment of users to assess the changes made to the videoconferencing system. This can be done by developing an automated electronic questionnaire, in which the user could evaluate the past videoconferencing session; ─ If you cannot achieve the required quality when using cloud video conferencing, you should use your own corporate video conferencing system. References 1. Video conferencing market 2020, https://www.cnews.ru/reviews/rynok_videokonferentsisvyazi_2020/articles/videokonferen tssvyazdo_i_posle_pandemii, last accessed 05.12.2020. 2. The state of videoconferencing in Russia, https://www.osp.ru/netcat_files/18/10/OSP_ DATA_VX_2015_OTChET.pdf, last accessed 07.12.2020. 3. Recommendations ITU-R. BT.500-13 (2012-01). Methodology for the subjective assessment of the quality of television pictures. ITU, 2012 МСЭ-R BT.500-13(01/2012). Methodology for the subjective assessment of the quality of television pictures, http://www.itu.int/pub/R-REC, last accessed 12.12.2020. 4. Dynaev P.A.: Development and research of methods for assessing image quality in digital television. PhD dissertation. Siberian State University of Telecommunications and informatics, Novosibirsk (2018). 5. Makolkina M.A.: Development and research of video transmission quality assessment models in IP networks. PhD dissertation. The Bonch-Bruevich Saint-Petersburg State University of Telecommunications (2014). 6. Tyukhtyaev D., Zhuykova S., Kirianova N.: Influence of characteristics of communication channels on the quality of rendered video conference services. Problems of information transmission in infocommunication systems Collection of reports and abstracts of the X All-Russian Scientific and Practical Conference, 86-93 (2019). 7. Kalitay H.K., Nambiarz M.K.: Designing WANem: A Wide Area Network emulator tool. International Conference on Communication Systems and Networks (COMSNETS), 1-4 (2011). 8. Development of the Internet in the regions of Russia, https://yandex.ru/company/researches/2016/ya_internet_regions_2016, last accessed 05.12.2020. 9. Galich S., Tkachenko N.: Virtual laboratory bench for emulating the characteristics of communication channels of ip-networks. Problems of information transmission in infocommunication systems Collection of re-ports and abstracts of the VIII X All-Russian Scientific and Practical Conference, 31-36 (2017). 10. Determine location by IP address, https://2ip.ru/geoip/, last accessed 15.12.2020. 11. MSU Video Quality Measurement tools, https://www.compression.ru/video/quality_measure/index.html, last accessed 15.12.2020.