Digital transformation, a crucial innovation in local government, is anticipated to reshape European public service delivery, administration structures, and overall governance. The recent COVID-19 pandemic underscored the significance of well-prepared digital administration, particularly at the local government level, which plays a pivotal role in digital transformation. However, current comparative research on the digital transformation of state and administration lacks sufficient investigation into local government levels, creating a knowledge gap on implementation and effects across Europe.
DIGILOG 1 is a research project determined to close this gap. It is an international and interdisciplinary project that consists of political and computer scientists from the University of Potsdam (DE), the Vienna University of Economics and Business (AU), and the Zurich University of Applied Science ZHAW (CH). The Researchers of the project in the field of Computer Science are the contributing authors of this paper. The project is financed by the Swiss National Science Foundation (SNSF / Project Nr. 200839) and Deutsche Forschungsgemeinschaft (DFG, German Research Foundation). The start of the project was in spring 2022 and the end will be in summer 2025. The research project seeks to address this above-mentioned gap by examining two key questions:
• What are the dynamics, scale, and pace of digital transformation in European local governments? Is the change radical, revolutionary, incremental, or evolutionary, and are there identifiable regional differences? • What effects does digital transformation have on these organizations, specifically in terms of output (service delivery, organization, processes, and resources), outcomes (performance and accountability), and impact (citizen acceptance, governance, and emerging tensions)?
To address these questions comprehensively, data will be collected in different ways from all municipalities in the 46 member states of the Council of Europe. As shown in Figure 1 we collect data for the different communities in three ways.
In conjunction with the quantitative surveys, comparative case studies are conducted in selected municipalities, which are also part of the extended survey sample. The case studies are carried out in communities with different administrative cultures to capture the country-specific variance of local administrative systems. The case study approach relies on field research methods, semi-structured expert interviews, and focus groups conducted with local CEOs, Chief Information Officers (CIOs), department heads, employee representatives, and staff. The aim is to gain in-depth insights into the internal processes and actor constellations of the respective digital transformation paths, building on the quantitative part's interim results by capturing the municipalities' organizational realities.
In addition to the qualitative case studies, the DIGILOG project is based on two quantitative forms of data collection: a web crawler for analyzing municipal websites and a survey among the leaders of European municipal administrations. The survey has several objectives. The main goal is to collect information on the status of the surveyed municipal administrations' external and internal digital transformation, from which a Europe-wide index will be created. In the external domain, this primarily includes the digital service offerings of the administrations, classified into various maturity levels according to an established social science model. The categorization spans from basic information provision to options for digital interaction with administrative personnel and completely digital and seamless administrative process handling. The internal domain, on the other hand, covers aspects such as the technical equipment of the administration, forms of internal communication, data management, and the automation of processes and routine decisions.
Furthermore, the survey collects data on various other variables related to digitization. These include factors that can help explain the state of digital transformation in municipalities, such as the size and organizational form of the municipality, as well as those that can reflect the consequences of digitization, such as questions about the efficiency of administration or the satisfaction of citizens with administrative work.
Web crawling is a central component of the DIGILOG project. In addition to surveys, automatic crawling and analysis of municipal websites are part of the quantitative analysis. The results of the data analysis described below will be displayed on a dashboard within a monitoring platform. Additionally, the lists of website URLs and email addresses for surveys, if not already provided, are completed through crawling public data sources.
The monitoring platform is based on three main components that interact with each other: web crawling, data storage, and subsequent data analysis. This platform ensures monitoring of the political municipalities' websites during the project duration. To manage the volume of data, several methods enable targeted data collection with minimal information loss. One project goal is to explore and implement the most efficient method for this task.
Data storage is ensured with two different database systems, a relational and a documentoriented system. The relational system stores database keys and normalized information. Complementarily, the website documents and the analysis results are stored in a documentoriented system. For analysis, clues (e.g., mention of selected services or keywords) indicating digital transformation are extracted and evaluated. Various methods from Natural Language Processing (NLP), a subfield of Machine Learning, are applied.
The analysis, in turn, can provide effective feedback to the intelligent crawler, contributing to its continuous improvement. The quality of the analysis is ensured by domain experts who interpret and contextualize the results for management, political science, and public administration.
Several relevant indices on digital service provision exist, offering country rankings and potentially serving as a valuable foundation for an index on local digital service provision within the scope of this project. The European Commission publishes the Digital Economy and Society Index Report on digital public services; however, it lacks specificity for the local government tier [1]. The Digital Adoption Index by the World Bank, a composite index gauging the adoption of digital technologies globally, focuses on the government sector, with sub-indices covering core administrative systems, online public services, and digital identification. The United Nations' E-Government Development Index assesses the effectiveness of public service delivery, identifying patterns in e-government development and regional challenges. Despite its Local Online Service Index focusing on the local level, evaluating the scope and quality of online services, telecommunication infrastructure development, and human capital, it only assesses portals in a selection of 100 cities worldwide, overlooking smaller local governments [2]. The E-Government Monitor, conducted through a representative survey of populations in Germany, Austria, and Switzerland, explores the usage and satisfaction related to e-government services. Results indicate a pronounced use of e-government services in Austria, followed by Switzerland and Germany [3]. Nonetheless, once again, this index lacks specificity for the local government tier. The German Index of Digitalization (Deutschland-Index Digitalisierung) scrutinizes digital infrastructure, the use of digital services, the digital economy, and e-government in individual German states but is confined to Germany [4].
The project described is interdisciplinary. It intersects with the research area of political Science and Information Retrieval in Computer Science. However, we only focus on the related work of Information Retrieval related to this project.
There is already work claiming to measure the level of digital transformation within local governments. Garcia-Sanchez et al. [5] presents an analysis of the development of e-governments of 102 Spanish municipalities where they select features from various papers and frameworks. Pina et al. [6] conducted an empirical study about the effect of e-government on transparency, openness, and hence accountability in 15 countries of the EU and a total of 318 government websites. This task of assessing websites even finds its application in other domains such as health [7].
Since we focus on website content to measure digital transformation, we note the importance of existing work on website processing, classification, and embedding, which is the encoding of data into a lower-dimensional representation in such a way that preserves some relationship in the data. We might focus on a website's visual or textual aspects, or even both, and leverage machine learning for our digitalization measurements. It's not surprising that recent work often uses Large Language Models (LLMs) and Convolutional Neural Networks (CNNs). Other classical machine-learning approaches rely more on feature engineering. However, they do not generalize as well as the state-of-the-art models due to their lack of flexibility regarding structural changes of an HTML page. A large amount of related work exists in the field of text-based embedding and classification of websites, which might help us categorize certain website elements. Kowsari et al. [8] and Minaee et al. [9] provide reviews on past work on text classification in general, while Hashemi [10] gives us a survey on web page classification. While "classification" refers to categorizing websites, before making the final prediction, we need to transform website data into a more manageable form which can involve creating embeddings for the websites. These website embeddings can be compared based on numerical similarity for various use cases. The classification models can be used to detect important digitalization elements on the website while also giving us insight into how to process websites effectively.
Visual-only classifications are, in many cases, applied to the detection of harmful content such as propaganda of terrorism [11], alcohol, adult content, weapons [12,13] or just food, fashion or landscapes [14]. These classes all have distinct visual features. However, in many cases, these approaches can't distinguish between visually similar pages (e.g., municipality homepage vs. tourism page of the same municipality).
In text-based website classification, some approaches rely on classical machine learning [15,16]. However, the majority are based on neural networks [17,18,19,20] and the more recent approaches are transformers architecture [21,22,23,24]. Most notably, [23] proposes MarkupLM for document understanding tasks based on the raw text and markup language, which is also used to code websites.
A mixed approach using both textual and visual features can be seen in [25] and [26]. The ladder encodes multiple parts of a website, such as a screenshot and metadata, and combines them to feed it into a neural network as input. The model is trained to categorize websites into 14 different classes. While previous work gives insight into how websites are processed and represented numerically, we must apply this knowledge to our specific data. How exactly website data is handled is not a solved problem. Kiesel et al. [27], for example, compares different web page segmentation algorithms. Dividing the page into individual segments might provide more concentrated information sources for our future algorithms. Finally, recent AI chatbots such as ChatGPT or open-source variants are capable of understanding a wide range of instructions. Recent developments have made it possible for the models to even react to image input while understanding user instruction, making them large multimodal models. They are foundation models that can be used in a variety of ways, and they can understand website code as well as screenshots. As development continues, it is becoming easier to use these models for automatic extraction, summarization, analysis, and categorization of municipality websites. As these models generate text, natural language analysis is essential.
The field of our work in this project consists of two parts:
• The URL gathering consists of the following questions: Has the municipality a website, and if so, what is the URL? Furthermore, the retrieved URLs must be distinguished from non-municipality URLs to eliminate false positives. • The website must be preprocessed (website segmentation, selection of relevant data, and removal of noisy data) and processed. The municipality website must be assessed based on the criteria defined by political scientists. A classifying model must be capable of detecting certain features if they exist on this website.
Assessing a website requires a semantic understanding of a website by the machine learning model used to process the Websites. Whether it is URL classification (specifically discerning municipality websites from others), topic modeling (classification of services), or e-service detection on web pages, a robust foundation in embedding is essential. In our previous work, we conducted not yet published experiments with general pre-trained webpage embedding models and developed a basic embedding method to effectively differentiate municipality websites from non-municipality ones. All methods yielded very good results, with the more complex ones resulting in slightly better results. However, it's crucial to acknowledge that basic embeddings demonstrated a faster processing speed than more complex models, a significant consideration given the vast number of websites in our study. We additionally evaluated different data sources concerning their completeness of data. The categories evaluated were search engines, encyclopedias, and blind requests with fabricated URLs based on certain patterns. The retrieved URLs partially consisted of wrong URLs that did not belong to the local government or municipality. Although the URL appeared to be correct in many of those cases, containing the municipality name, the content was of another topic such as tourism, airports, other official organizations in this municipality, or even completely unrelated content to the municipality. Thus, an automated distinction and classification by analyzing the website's content was required. Furthermore, as mentioned in Section 1.4, there are many ways of measuring digitization. In a conference paper, we defined three key aspects of our analysis, which consisted of different indices. The categories are Service Maturity (measurement of provision of information, communication possibility, and transactions), Usability (evaluation of accessibility and convenience of use), and Technical Maturity (evaluation of security and privacy). This index was published in a conference paper [28]. We tested the index on a sample of municipality websites and are currently working on implementing and applying it to the whole data set. Looking ahead, our plan encompasses the application of webpage embedding techniques for e-form detection, including webpage segmentation and relevant information extraction. Further, we plan to leverage large Language Models for topic modeling of webpages and webpage content. This approach aims to further automate the process of monitoring the digital transformation of European communities.
This work is supported by Grant No. GR 200839 of the Swiss National Science Foundation (SNF) and German Research Foundation (DFG) for the research project "Digital Transformation at the Local Tier of Government in Europe: Dynamics and Effects from a Cross-Countries and Over-Time Comparative Perspective (DIGILOG)".