The article presents the results of testing and improvement of the developed comprehensive methodology for evaluating websites of authorities and local self-government. The study was conducted with the aim of evaluating the openness of the work of authorities and local selfgovernment, the availability of council decisions, content, ease of use of electronic resources. In the process of research, the authors of the article developed a system of evaluation criteria, as well as conducted an in-depth analysis of the technical parameters of websites. The possibility of improving the methodology is considered. One of the ways was to reduce the dimensionality of the task, which consisted in reducing the number of sub-indicators and/or main indicators, according to which the evaluation was carried out. Correlation analysis was used to identify indicators that allow merging or removing from the system of criteria. In order to understand the state and functioning of community websites as a whole, identify problem groups and develop a system of recommendations to improve their work, the data mining technique was applied, namely, cluster analysis. A clustering model has been built that allows you to find hidden structures in the information content of websites of communities (authorities), reveal complex relationships between information resources, and separate groups of communities whose websites have similar characteristics. The results of clustering can be used to make decisions in e-government processes and improve the perception and use of information by citizens. The proposed approach will be of particular value in the case of research scaling. correlational analysis Website, e-government, authorities and local self-government, data mining, clustering, CITI'2023: 1st International Workshop on Computer Information Technologies in Industry 4.0, June 14-16, 2023, Ternopil, Ukraine ORCID: 0000-0001-8549-2910 (A. 1); 0000-0003-3297-0776 (A. 2); 0000-0001-5667-6569 (A. 3); 0000-0001-8844-5490 (A. 4); 0000-
Today, one of the priority directions for the development of e-government in Ukraine is the creation of a digital ecosystem for territorial communities. Such an ecosystem will help improve the quality of life of residents, develop the economy, and ensure more effective management of the territory. It includes the following components:
Electronic government – digital tools and services that allow interaction between government institutions and residents (electronic services, electronic declarations, online consultations and other tools that simplify interaction with government bodies). EMAIL: (A.
1);
2); strutynskairy@gmail.com (A.
3);
2023 Copyright for this paper by its authors. 2. Digital infrastructure – infrastructure necessary to support digital initiatives, such as highspeed Internet, communication networks, cloud services, remote servers, digital data storage and others. 3. Digital services for citizens – digital tools and services that allow residents of the territorial community to be more active participants in governance, including online citizen meetings, petitions, online voting, and others. 4. Digital services for business – digital tools and services that allow business to be more competitive, including electronic procurement system, online business registration, electronic reporting system and others. 5. Digital educational environment – digital tools and services that allow residents of the territorial community to develop their skills and abilities in the learning process.
The site plays a key role in the formation of the community's digital ecosystem, because it allows a wide range of people to find the necessary information about the activities of the community, decisions of authorities and various services provided by the community. This ensures transparency and openness in the work of local self-government bodies and promotes interaction with the community. The site is also a platform for receiving messages and requests from residents, providing answers to questions, publishing news and announcements, organizing online surveys and consultations, etc.
It can be used to provide electronic document circulation, electronic voting, organization of online consultations with the community, etc. The site can help promote the community, promote the development of tourism, culture and other industries that ensure the development of the local economy and increase the standard of living of residents.
That is why the purpose of this study was to understand the state and functioning of community sites as a whole, to improve the system of criteria for evaluating the effectiveness of websites, to identify successful and problematic groups using the data mining techniques for the further development of a system of recommendations so that websites become more effective. 2.
Digital transformation of territorial communities is an important issue that requires attention and
research in the era of information society. Researchers, scientists, and practitioners (local government
bodies) around the world are exploring the issue of building digital societies and forming certain
digital ecosystems of the community. For example, in the study [
In the work of author Bashuk A.I. [
Research conducted by many foreign and domestic scientists, such as Sarmad Ali, Inayatullah
Shah, and Abdul Rauf [
In a study [12] conducted in Ukraine in 2019, it was shown that 80% of local government authorities have their own website, but only 22% of them have full information about their activities and services. This indicates the need to improve the quality and accessibility of information on local government websites.
Regarding the methodology for assessing the websites of local self-government bodies, it is worth noting the works of foreign authors Huseyin Ince, Orhan Dolu, and Mehmet Nafiz Aydin in their research [13], which analyzed the evaluation of local self-government websites from the perspective of citizens. They proposed an evaluation methodology based on five criteria, including information accessibility, website functionality, and effectiveness. In the work [14], authors Jose A. CastilloManzano, Mercedes Castro-Nuño, and Juan A. Parra-Domínguez evaluated the quality of local selfgovernment websites in Spain. The authors proposed an evaluation methodology that uses 18 parameters to determine the level of website effectiveness and quality. In the research of authors Wenjing Pan, Rui Guo, and Yuan Cheng [15], a comparative analysis of the quality of local selfgovernment websites in China was conducted. They proposed an evaluation methodology that uses 9 criteria, including accessibility, interaction with citizens, and ensuring confidentiality.
In the works of domestic scientists, issues of the functioning of websites of state authorities and local self-government bodies are investigated. For example, in the work [16] by author Korzh I.F., the functioning of websites was analyzed regarding compliance with current legislation, which regulates issues related to the requirements for their content and information filling [17, 18]. However, more attention should be paid to the issue of monitoring the websites of territorial communities in Ukraine, as the relevance and importance of web resources are extremely high, and research in the relevant context has not been conducted.
During September-December 2022, the authors of the article monitored the websites of authorities and local governments of 14 territorial communities (hereinafter referred to as TC) of Ternopil Oblast, as well as the Ternopil Oblast Military Administration. The study was conducted with the aim of evaluating the openness of the work of authorities and local self-government, the availability of council decisions, content, to ease the use of electronic resources.
Among the studied communities were Ternopil, Baikivtsi, Velyki Hayi, Pidhorodne, Zalishchyky, Lanivtsi, Vyshnivtsi, Shumska, Skalat, Chortkiv, Zboriv, Pochaiv, Terebovlya, and Mykulyntsi ones. In the process of research, the authors of the article developed separate evaluation criteria, as well as conducted an in-depth analysis of the technical parameters of the sites. In the evaluation process, each site received a score based on individual parameters (Table 1).
{ 1, 2, 3, 4, 5}, = 1,2, … ,15, where 1 – is the overall rating of the k-th site (in points) according to indicator 1 "Availability of important general information on sites"; 2 – overall evaluation of the k-th site (in points) according to indicator 2 "Availability of important documentation on sites"; 3 – overall evaluation of the k-th site (in points) according to indicator 3 "Availability of additional information on sites"; 4 – overall evaluation of the k-th site (in points) according to indicator 4 "Availability of network functions and site efficiency (e-democracy)"; 5 – the overall assessment of the k-th site (in points) according to indicator 5 "Technical requirements for the site".
The final evaluations of the websites were obtained according to the formula:
= ∑5=1 , = 1,2, … ,15. (1)
Each of the five elements (criteria) of the developed evaluation system contained a number of subcriteria. The maximum possible value of the final assessment for each site is 108 points (Table 2).
According to the results of the analysis, Ternopil City Council, Shumsk Territorial Community and Baikivtsi Territorial Community were among the three leaders in terms of the main site assessment indicators. The results of the research were brought to the leaders of all communities to optimize the work of the sites and borrow the experience of progressive communities in the formation of a digital ecosystem.
An important task of this study, in addition to directly evaluating the sites, was to test the very methodology of site research. Improving the methodology will increase the efficiency of further research and may be useful for scaling the research to individual regions or the country as a whole.
A total of 54 sub-indicators were used in the assessment of the state and functioning of the sites in five sections of the assessment based on the main indicators. In order to analyze the effectiveness of the developed system of indicators for the sites’ performance assessment, the columns with ratings, of summary table 2 were used for each of the communities for each of the main indicators. One of the ways to improve the methodology was to reduce the dimensionality of the task, which consisted in reducing the number of sub-indicators and/or main indicators by which the assessment was conducted. We note that reducing the size of the criterion system should not lead to the loss of important information about the operation of websites.
To identify a possible correlation between the 5 main site assessment criteria, pairwise values of Pearson correlation coefficients were calculated for all possible combinations of criteria: (2) where – is the average value (in points) obtained according to the i-th criterion; – is the average value (in points) obtained by the j-th criterion, both for a sample of 15 sites.
Note that the value of the correlation coefficient does not depend on the order of the selected criteria
= , therefore, the correlation matrix is symmetric for all pairs of criteria. All elements of the main diagonal = 1. All obtained values of pairwise correlation coefficients are entered in the correlation matrix (Table 3). The value of pairwise correlation coefficients in the form of a correlation matrix 2. Availability of important documentation 3. Availability of additional information 4. Availability of network functions and site efficiency (e-democracy) 5. Technical requirements for the site Name of the assessment
indicator 1. Availability of important general information 2. Availability of important documentation 3. Availability of additional information 4. Availability of network functions and site effectiveness (edemocracy) 5. Technical requirements for the site 1.
of Availability important
general information 1,0000 0,2047 0,5035 0,5171 0,0050 1,0000 0,2409 -0,0495 0,0404 1,0000 0,3065 0,2563 1,0000 0,2632 1,0000
The values highlighted in the table (greater than 0.5) indicate the presence of a positive linear correlation between: 1) 1st and 3rd criteria (0.5035); 2) 1st and 4th criteria (0.5171).
This indicates that positive changes in the values of one of the indicators lead to positive changes in the average values of the other. A comparative analysis of the sub-indicators included in the specified pairs of criteria will make it possible to make a decision on merging the main criteria that showed a positive correlation, or removing one criterion from the pair that showed a positive correlation from the system of criteria.
A detailed analysis of the content of the sub-indicators of the 1st and 3rd criteria showed that the more a community or authority places important general information on its website, the more additional information is presented on it. A detailed analysis of the content of the sub-indicators of the 1st and 4th criteria showed that in communities where important general information is posted in a sufficient amount, as a rule, there are quite developed network functions and the indicators of the site's effectiveness are significantly higher. So, the research results showed that the developed methodology can be improved by consolidating the criteria system (up to 4 indicators), combining criteria 1 and 3 into the group "Availability of important general and additional information on the sites". 4.2.
( 1, 2, 3, 4, 5) and formula:
In order to understand the state and functioning of community sites as a whole, to identify successful and problematic groups, as well as to develop a system of recommendations so that websites become more effective, the data mining technique was applied, namely the clustering of sites according to the values of 5 indicators, similar to what made by the authors in [20].
To build the clustering model, the Cluster option of the Data Mining add-on in MS Excel 2010 was used. The clustering model was obtained by the Scalable K-means method, which allows the selection of the desired number of K clusters and the possibility of working with large volumes of data [21]. The K-means method is a well-known unsupervised machine learning method that combines elements with the smallest distances into one cluster, and at the same time assigns elements with the largest distances to different clusters. The k-means algorithm calculates the squared Euclidean distances between elements in a cluster and the cluster mean, and converges on a final set of K clusters when that sum reaches its minimum value [22].
The Euclidean distance between the two sites and with sets of estimates = = ( 1, 2, 3, 4, 5) respectively, is calculated by the ( ; ) = √∑5=1( − )2 , where , – sites, the Euclidean distance between which is calculated; – the value of indicators for the site , obtained according to 5 criteria; – the value of indicators for the site , obtained according to 5 criteria.
The result of building a clustering model for 3 clusters, namely the number of elements in each cluster and the main characteristics of the clusters, is shown in Figure 1. The cluster analysis made it possible to distinguish groups of community sites based on their similarity (10 participants, 3 participants, 2 participants).
The 1st cluster included 10 sites that received average scores according to the criterion "Availability of important documentation" (43.66% of sites received scores of 15-19 out of 30 possible) and above average scores according to the indicator "Availability of important general information on sites" (57.1% of sites received 14 points out of 20). 42.86% of the participants of the first cluster received the maximum points according to the indicator "Technical requirements for the site", the same percentage of sites received a high score according to the criterion "Availability of network functions and effectiveness of the site (e-democracy)" (14 points out of 16 possible).
The 2nd cluster included 3 sites that received low points according to the indicator "Availability on sites of important documentation" (81.28% of sites received 5-9 points out of 30 possible) and average points according to the criterion "Availability on sites of important general information" (2 sites received 13 points out of 20 possible, 1 site – 15 points).
The 3rd cluster included 2 sites, the main difference from other evaluation participants is the presence of important documentation (the sites received 20-26 points out of a possible 30). (3)
The built clustering model allowed us to find hidden structures in the information content of community (authority) websites and to separate groups that have similar characteristics. The results of clustering can be used to identify complex relationships between information resources and further develop a system of recommendations to improve the perception and use of information by citizens. In addition, it can be useful in reducing data processing costs, since clustering can help identify the most important characteristics of information resources that will be used to make decisions in egovernment processes.
The proposed approach will be of particular value in the case of research scaling. In this case, it is necessary to justify the choice of the clustering method, its settings, as well as methods of determining the quality of the built clustering model.
The proposed methodology for evaluating community websites when used can have several key effects on the development of the e-government system:
Improving interaction with the community. Evaluating website performance will help to understand how the community is interacting with the site and whether the website is meeting their needs. Based on this information, authorities and local governments can identify shortcomings and improve the functionality of the site to ensure better interaction with the community.
Attracting new users. If a website is not effective in engaging with the community, new users may not find the information they need and may not return to the website. Evaluating website performance will help identify these issues and improve site functionality to attract new users.
Assessment of achievements and effectiveness. Evaluating the effectiveness of the website will help to understand the development of the community and its effectiveness in achieving its goals. For example, if a website is successful in promoting community initiatives and programs, then this may indicate the success of the community.
Reputation improvement. An effective website can help improve the reputation of a community, particularly if the website contains useful information for citizens. Evaluating the performance of a website can help identify weaknesses and improve it to ensure a more positive reputation.
The possibility of improving the methodology was considered. The results of the correlation analysis of the criteria for evaluating the effectiveness of websites showed that the developed methodology can be improved by consolidating the system of criteria (up to 4 indicators), combining criteria 1 and 3 into the group "Availability of important general and additional information on the sites".
For a deeper understanding and possible typification of community approaches to website occupancy and maintenance, a website clustering model was constructed using the K-means method. According to the results of clustering, 3 groups of communities (10, 3 and 2 participants) were singled out, the content and effectiveness of which websites are similar. In the case of scaling up the research, the results of clustering can be used to identify complex relationships between information resources and further develop a system of recommendations for improving the perception and use of information by citizens. In this case, it is necessary to justify the choice of clustering method, its settings, as well as methods of determining the quality of the built clustering model.
The proposed methods of improving the methodology for evaluating the effectiveness of websites (correlation analysis of the main evaluation criteria and cluster analysis of community websites) can be useful for reducing data processing costs, as clustering can help identify the most important characteristics of information resources that will be used for decision-making in e-government processes. In further work, it is interesting to consider the issue of optimization of the methodology for researching the effectiveness of websites using various techniques and methods of intellectual data analysis. [12] Monitoring of the Information Content on the Official Websites of Central and Local Executive Authorities in the First Half of 2019. URL: https://www.kmu.gov.ua/news/derzhkomteleradiooprilyudniv-rezultati-monitoringu-veb-sajtiv-organiv-vikonavchoyi-vladi [13] Huseyin Ince, Orhan Dolu та Mehmet Nafiz Aydin. Assessment of Local Government Websites in the Age of E-Government: An Analysis from the Citizen's Perspective. (2021). [14] Castillo-Manzano, J.A., Castro-Nuño, M., & Parra-Domínguez, J.A. Evaluating the quality of local government websites in Spain. Revista de Administración Pública, (2017). 53-88. [15] Pan, W., Guo, R., & Cheng, Y. Evaluating Local Government Websites: A Comparative Study in China. International Journal of Public Administration, (2020), 43(11), 963-975. DOI: 10.1080/01900692.2019.1684975. [16] Korzh, I.F. Websites of state authorities and local governments: mechanisms for accessing public information. (2018) (9-16). DOI:10.37750/2616-6798.2018.2(25).270655. [17] Order of the State Committee for Information Policy, Television and Radio Broadcasting of Ukraine and the State Committee for Communications and Informatization of Ukraine on the functioning of websites of executive authorities (2002). URL: http://zakon5.rada.gov.ua/laws/show/z1022-02. [18] Order of the State Committee for Information Policy, Television and Radio Broadcasting of Ukraine and the State Committee for Communications and Informatization of Ukraine on the approval of the Procedure for information filling and technical support of the Unified Web Portal of executive authorities and the Procedure for the functioning of websites of executive authorities (2002). URL: http://zakon3.rada.gov.ua/laws/show/z1021-02. [19] O. Sorokivska, I. Strutynska, L. Melnyk. A Comprehensive Report Based on the Sociological Study Results: Challenges and Opportunities of Communicative Interaction Between the Public and the Authorities, TNTU, 2022. [20] Strutynska I., Kozbur, H., Dmitrotsa L., Bodnarchuk I., Hlado O. (2019). Small and Medium Business Structures Clustering Method Based on Their Digital Maturity. 278-282. 10.1109/PICST47496.2019.9061464. [21] Bahmani, B., Moseley, B., Vattani, A., Kumar, R., & Vassilvitskii, S. Scalable k-means++.
Proceedings of the VLDB Endowment, (2012) 5(7), 622-633. Retrieved from https://theory.stanford.edu/~sergei/papers/vldb12-kmpar.pdf. [22] Microsoft Clustering Algorithm Technical Reference (2022). Retrieved from https://learn.microsoft.com/en-us/analysis-services/data-mining/microsoft-clustering-algorithmtechnical-reference?view=asallproducts-allversions.