The Italian Communications Regulatory Authority (AGCOM) has among its institutional tasks ensuring fair competition between operators on the market and protecting consumers rights in sectors under its jurisdiction. AGCOM is a “convergent” authority and, as such, performs regulatory and supervisory functions in the sectors of electronic communications, audiovisual, publishing, postal services, and online platforms. In the field of quality of services AGCOM has developed projects that allow end users to measure the quality parameters for fixed and mobile Internet access services and the user experience for video streaming services. The present paper concerns the definition and application of parameters related to the quality of services for Internet access services and user quality experience (QoE) for video streaming services.
The project involves the implementation of a system for measuring the quality parameters of fixed broadband network connections provided by national operators. This has been granted by probes (some hundreds) installed throughout the country of Italy. Figure 1 shows the high-level architecture of the MisuraInternet project. The results of the measurements are available on the website https: //www.misurainternet.it in a comparative mode and include data transmission and reception speed, packet loss rate, latency, and jitter measured, in such probes, for each Internet service provider (ISP), on a time basis of six months. This provides users with a realistic and transparent picture of the average actual quality of service ofered, by each ISP, in the national territorial areas. As part of the project, AGCOM also made available the software “Ne.Me.Sys”1, which allows users to independently measure the quality of their specific fixed-line Internet connection and generate a report whose results have probative value for exercising the right to complain and withdraw if the quality of service is lower than contractually agreed. Since its launch, the project has counted more than 418,600 registered users with more than 76,800 certificates issued and has evolved to allow the measurement of increasingly fast connections and other technologies such as Fixed Wireless Access (FWA). In addition, the “MisuraInternet Online Speed Test” tool is available online2 to measure the instantaneous quality of Internet access through a web browser to improve the knowledge of the state of broadband in Italy and the quality perceived by end users. Since its release (June 2020), the online speed test has recorded more than 6 million measurements.
The “MisuraInternet Mobile” project provides end users with extensive transparency in the performance and quality of mobile Internet connection services. The results of the various measurement campaigns, which ofer users an overview of the quality level of mobile broadband services provided by operators with their own infrastructure in Italy, are also available on the website https://www.misurainternetmobile.it/ in a comparative mode, on a single measurement pixel, and on an average mode of the MNO involved in the project, at national level. The results cover data transmission and reception speed, web page loading time, transmission delay, video start time, and video freezing interruption rate. The project has evolved, over time, in parallel to the development of mobile radio technologies. In recent years, drive test verification campaigns have been conducted on the quality of Internet access service on 4G and 5G networks, in the best technology mode; at the various "measurement points" the best available connection was detected, providing users with comparative, useful, and updated information on the performance of mobile operator networks.
AGCOM has also engaged in the topic of quality of experience (QoE) with regard to video streaming services provided by Over the Top (OTT) operators. In implementation of Art. 33 of Legislative Decree No. 208 of November 8, 2021, AGCOM adopted the decision n. 74/24/CONS3, which defines, in agreement with the Ministero delle imprese e del made in Italy, the conditions and parameters of the 2https://misurainternet.it/misura-speedtest/ 3https://www.agcom.it/provvedimenti/delibera-74-24-cons regularity of service and image quality that must be ensured by audiovisual media service providers transmitting events of social interest and of great public interest4. OTTs that transmit such events have to measure the quality parameters of live streaming audio-video transmissions and make them available to users in their customer web area so that they can have such tools to check the quality of provided service. While knowing that the perception of quality of user experience is subjective, AGCOM deemed it important to identify objective indicators based on considerations that come from the rate of abandonment of the service during the start-up phase, the total time spent by the user in the play mode, and the probability of the user returning to use the service within a given time. This approach has been useful for addressing user complaints received by AGCOM and tracing them back to specific quality parameters that users easily understand and assess and to allow them to provide evidence in the event of a complaint or dispute with operators. Before adopting the regulations, AGCOM also launched a public consultation with the participation of the main OTTs and stakeholders, who have expressed their considerations on the topic. AGCOM adopted the following definitions of the quality indicators of user experience of live video streaming platforms: • Malfunctions in accessing platform (MAP): the number of consecutive failed access attempts to the live streaming platform by a user who, while meeting the Internet connection and supported device requirements set out in the service terms and conditions, is unable to access the platform or successfully log in, despite having entered correct and valid credentials and not having exceeded the limit on the number of connected devices. Only access attempts with at least 20 seconds between attempts should be considered for calculating this indicator. • Malfunctions in accessing events (MAE): the number of consecutive failed attempts to select one of the available live streaming events, while meeting the Internet connection speed and supported device requirements specified in the service terms and conditions. An attempt to access an event is considered failed if an error message is received from the platform or if the playing of content has not started after a timeout of 60 seconds from the event selection. Only access attempts with at least 20 seconds between attempts should be considered for calculating this indicator. • Starting time (TA): the time elapsed between the selection of a live streaming event, by user correctly connected to the platform and meeting the requirements regarding Internet connection speed and supported device required in the terms and conditions of service provision, and the start of playback of the chosen event. • Total number of freezing events (TF): the total number of interruptions that occurred during the streaming of a live event experienced by a user correctly connected to the platform, in the absence of any action by the user and under conditions of satisfaction of the requirements regarding Internet connection speed and supported device required under the terms of service provision, typically in conjunction with bufering events. • Maximum freezing time (MFT): the maximum duration of freezing interruptions (TFi) that occurred during the streaming of a live event by a user correctly connected to the platform, in the absence of any action by the user and under conditions of satisfaction of the requirements regarding Internet connection speed and supported device required in the terms of service provision, typically in conjunction with bufering event. • Total freezing time (TFT): the sum of the TFi durations of all interruptions that occurred during the streaming of a live event by a user correctly connected to the platform, in the absence of any 4Such events have been identified by the decree of 27 May 2022 of the Ministero delle imprese e del made in Italy, formerly Ministero per lo sviluppo economico, and include, for example, the end-of-year message of the President of the Italian Republic, the Serie A football championship matches, the Champions League and the Europa League matches, the Olympics, the World Cup and the Sanremo Festival.
action by the user and under conditions of satisfaction of the requirements regarding the Internet connection speed and the supported device required in the terms of service provision, typically in conjunction with bufering event. • Rebufering ratio induced by connection (RRIC) : the sum of the durations of all interruptions due to rebufering during the playing of a live streaming event, experienced by a user correctly connected to the platform, in the absence of any action by the user and under conditions of satisfaction of the requirements regarding the Internet connection speed and the supported device required in the terms of service provision, in relation to the total streaming time (TST ). • Minimum resolution (Rmin): the minimum value of video resolutions displayed for at least 10% of the duration of the event, during the use of the contents of a live streaming event by a user correctly connected to the platform and in conditions of satisfaction of the requirements regarding Internet connection speed and the supported device required in the terms of service provision. • Minimum video frame rate (Fmin): the minimum value of the number of frames per second displayed for at least 10% of the duration of the event, experienced by a user correctly connected to the platform and in conditions of satisfaction of the requirements regarding the Internet connection speed and the supported device required in the terms of service provision. • Maximum delay (Dmax): the maximum time diference between the real event and the audio-video content received during the live streaming event by a user correctly connected to the platform and in conditions of satisfaction regarding the Internet connection speed and the supported device required in the conditions of provision of the service. • Number of event disconnections (ND): the number of times a user is disconnected from a live streaming event, requiring the event to be restarted, when the user was correctly connected to the platform and met the Internet connection speed and supported device requirements set out in the terms and conditions of service provision.
AGCOM performed analysis of the complaint reports received from users regarding poor service quality and it reveals that most of them are attributable to problems of accessing the platform, freezing events and poor image resolution. Based on that, AGCOM has defined the following thresholds beyond which users can request compensation from the operator, which must be provided as a bill discount or reimbursement (at least one condition has to be verified): • MAP + MAE > 5; • RRIC > 5% at the end of the event; • Rmin < 540p in case of download connection speed more than 3 Mbps when using a fixed or mobile connection; Rmin < 720p in case of download connection speed more than 6 Mbps when using a fixed or mobile connection; Rmin < 720p or Fmin < 50 in case of download connection speed more than 9 Mbps when using a fixed line connection; Rmin < 1080p or Fmin < 50 in case of download connection speed more than 16 Mbps when using a fixed line connection. For the purposes of user transparency and proof, in case of a compensation claim, the OTT operator must make available to each user, in a dedicated section of the web customer area and within 24 hours by the end of event, the information regarding the values of the MAP,MAE, RRIC, Rmin, Fmin and the average throughput measured during the video streaming event. AGCOM has also established a technical committee involving OTTs and interested associations, with the aim of supporting the implementation of the measures set out by regulation, under the coordination of the AGCOM Consumer Protection Directorate.
AGCOM has also launched a research and development project in collaboration with the National Interuniversity Consortium for Informatics (CINI), concerning the issue of measuring the quality of experience of live streaming audio-video signal transmission services provided by OTTs, taking into account the quality parameters established by the Authority with decision n. 74/24/CONS.
The author(s) have not employed any Generative AI tools.