The purpose of the study was to determine the dependence of the quality of rendering videoconferencing services on the characteristics of communication channels. Under the characteristics of communication channels, we considered such characteristics as: delay, jitter, percentage of lost packets. The article discusses subjective and objective methods for assessing video. The peak signal to noise ratio metric and the MSU Video Quality Measurement Tool software, created by the computer graphics laboratory of the Moscow State University, was used as an objective video assessment method. For the subjective method, the Double Stimulus Continuous Quality Scale method was used. The article presents the results of expert assessment and the results of an objective assessment using the peak signal to noise ratio metric. As a result of the work, it was revealed that if the characteristics of communication channels do not meet the minimum ITU requirements, objective assessments were at a very low level. It was also noted that the results of objective assessments are not always correlated with the result of subjective assessments using the PSNR metric.
1.1
Videoconferencing
Videoconferencing systems, hereinafter videoconferencing, have been actively used
in the last few years in the field of business, education and medicine. In early 2020,
there was an outbreak of the SARS-CoV-2 virus. As a result, many areas of our lives
have changed dramatically. Currently, a large number of employees work remotely
* Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License
using video conferencing and webinar systems. It is also worth noting that almost the
entire field of education has switched to a remote mode of work. The dynamics of the
market for means of collective work is presented in Figure 1. [
In connection with the rapid development of videoconferencing systems, a large load of communication channels during a pandemic, the question of the quality provision of videoconferencing services and the influence of the characteristics of communication channels on the quality of videoconferencing is very acute. Reducing the parameters of communication channels leads to degradation of the video and audio track, as well as various negative effects. These effects include the lag of the audio track from the video sequence, as well as its stuttering, a decrease in the quality of the video sequence or its complete loss for a certain time. 2
The quality of video transmission depends on various factors, such as packet loss due to buffer overflows on processing network devices, network latency, jitter, and others. Today, there are subjective and objective methods for assessing the quality of video transmission.
The subjective testing method is the use of live experts to determine the quality of
the video. The results of the assessments are obtained after averaging all the values of
the experts; this assessment is called MOS (Mean Opinion Score). Within the
framework of Recommendation ITU-R BT.500-12 [
The objective testing method is the use of special algorithms for video analysis. Objective methods, unlike subjective ones, do not give a complete picture of what a person sees on the screen. At the current time, there is no single recommendation for the use of any single metric. In the course of this work, the PSNR (peak signal to noise ratio) metric was used for an objective study. The choice of this metric is due to its widespread use in the scientific community and its use as a basic metric for assessing the loss of quality.
Like any metric, this metric has its advantages and disadvantages. PSNR does not
measure all video-specific parameters, as the fidelity of the image is constantly
changing depending on the visual complexity of the image, the available bit rate and
even the compression method. Thus, PSNR cannot determine how noticeable these
distortions will be for the user [
The peak signal-to-noise ratio is easiest to determine through the root-mean-square error (RMSE or MSE, Mean Square Error), which for two monochrome images R and Q of size i × j, one of which is considered a noisy approximation of the other, is calculated as follows:
MSE i j (Rij Qij )2
ij PSNR 10 log(
A2 MSE ) (1) (2)
Where, A is the maximum value accepted by the image pixel. When the pixels are 8 bits wide, A = 255.
To study the influence of the characteristics of communication channels on the
quality of video conferencing, the stand shown in Figure 3 was used. Experimental
stand included a server with installed server software for video conferencing, a PC of
a video conferencing participant, and PC running WANem software. WANem is a
WAN emulator that runs under the Linux operating system. On a PC server, the
server and client parts of the video conferencing system were installed. The client part
of the video conferencing was installed on the client's computer [
The TrueConf system was used as a video conferencing system. The choice of this system is due to its wide distribution in the Russian market, as well as the availability of a free license for a limited number of customers. The Logitech HD Pro Webcam C910 and HD Webcam C270 cameras on the server and client sides, respectively, were used as web cameras for making video calls.
The experiment consisted of creating a common room on a server and connecting
both clients to it. In the WANem software, the characteristics of the communication
channel were artificially changed: throughput, delay, and jitter and packet loss [
The answer to the question about the average bandwidth of the Internet channel in
the Russian Federation is given by the research of the company Yandex
"Development of the Internet in the regions of Russia-2016" [
Let us describe the technique used to obtain the characteristics of the
communication channels, such as: delay, jitter, and the percentage of lost packets. At
the first stage, to measure these parameters, ICMP packets were sent within 10
minutes to various IP addresses located in different geographic locations [
At the second stage, the values of the required channel parameters were calculated. For this, the formulas presented below were used.
The average delay was calculated using the formula:
N1 iN1 i Where N the number of delivered packets is, i is the delay for the i th packet. Jitter was defined as:
( ) max min max
The packet loss ratio was calculated as the ratio of the number of undelivered packets to the total number of transmitted packets:
The results will be used to simulate communication links in the WANem environment. If we refer to the ITU-T recommendations and the Order of the Ministry of Information Technologies and Communications of the Russian Federation No. 113 dated September 27, 2007, we can see that the values of Table 1, such as jitter and packet loss rate to destination points Brazil and Canada, do not meet the minimum requirements. The rest of the destinations are fully compliant with VoIP and video conferencing standards.
First, a reference video was recorded at a bandwidth of 10 Mb / s without delay, jitter and packet loss.
Then the videos were recorded, in which delay and packet loss were introduced,
based on the ITU-T QoS recommendations. To carry out the PSNR metric, the special
software MSU Video Quality Measurement Tool was used, created by the computer
graphics laboratory of the VMiK Moscow State University. MSU Video Quality
Measurement Tool is a program that allows you to use several metrics: the more
common ones - SSIM, PSNR, VQM, Delta, and specially developed metrics for
measuring blockiness and sharpness. The program is shareware; there is a PRO
version for commercial use [
To conduct a subjective assessment, the experts were shown test videos, after which they gave marks. 3
To evaluate the video from our experiment, an expert survey was conducted among teachers and university students. The average score is shown in Table 2.
We present the results of an objective assessment using the PSNR metric (Tables 3 and 4).
Based on the recommendations of ITU-T and Order 113 of the Ministry of Information Technologies and Communications of the Russian Federation, below are examples of frames of the original video and video received at the other end and processed with the PSNR RGB metric. Used footage from a video conference with Spain.
As seen in Figures 5 and 6, green and blue colors indicate strong video delays and changes in image quality relative to the first. Minor changes are shown in red and no changes are indicated in black. 4
As can be seen from Table 2, with the characteristics of communication channels that do not meet the standards of the ITU and the Ministry of Communications, one cannot count on a positive assessment of the quality of videoconferencing by users. At the same time, there is a tendency that the video taken with the simpler camera model C270 with 720p resolution was rated by users higher than the video taken with the C910 with 1080p resolution. Also, three of the experts spoke negatively about autofocus in the Logitech Pro HD Webcam C910. In their opinion, autofocus degrades the image quality in video. Thus, the camera model with the best technical characteristics is not always able to provide a higher subjective assessment of the user, however, for the final conclusion; a study with a large number of camera models is required.
From Table 3 and Table 4, it can be seen that despite the best PSNR metrics for the C910, the subjective rating for it is worse than for the C270. This fact can be explained by the presence of autofocus, although only three of the interviewed experts expressed their direct dissatisfaction with its work. 5
From the experiments carried out, it can be concluded that the characteristics of communication channels have a direct impact on the quality of video conferencing services. From this, the following recommendations can be made that can be used in the implementation and operation of videoconferencing in corporate networks: ─ Before organizing a video communication session, measure the characteristics of the communication channel; ─ If the characteristics of the communication channel do not meet the standards, it is not necessary to count on a positive subjective assessment; ─ If the channel characteristics do not meet the quality of service standards for this type of traffic, lease dedicated channels with guaranteed quality of service from telecom operators; ─ The use of video cameras or video terminals that provide a higher frame rate and higher resolution may not always have a positive effect on the subjective assessment; ─ During the operation of the videoconferencing system, it would be useful to accumulate a statistical subjective assessment of users to assess the changes made to the videoconferencing system. This can be done by developing an automated electronic questionnaire, in which the user could evaluate the past videoconferencing session; ─ If you cannot achieve the required quality when using cloud video conferencing, you should use your own corporate video conferencing system.