DIGILOG is an interdisciplinary research project between Computer and Political Science. The goal of the research project is to monitor and evaluate the digital transformation of the local governments of Europe. The project will generate coherent data for a systematic comparison using methodological triangulation, i.e., quantitative and qualitative methods. It will take the form of a regular and automated quantitative survey of all local authorities in 47 European countries (members of the Council of Europe), based on web crawling and machine learning techniques - this is a novel approach in the context of the social sciences - and qualitative research, namely case studies in selected European countries. Renowned scholars from the University of Potsdam, ZHAW, and the Vienna University of Economics and Business, with extensive experience in local government and comparative research, form the consortium of this project. Key project deliverables will be an openly accessible monitoring platform of digital transformation accessible to researchers and practitioners worldwide and contribute to a better understanding of longterm developments. The duration of the project submitted to the SNSF/DFG is three years; however, by automating the process, the real-time platform will continue to exist and be updated regularly beyond this time frame. The research project will yield policy-relevant knowledge concerning local digitization measures from a European perspective, which can then be utilized to improve policymaking for future public sector modernization.
CEUR ceur-ws.org
Digital transformation, a crucial innovation in local government, is anticipated to reshape European public service delivery, administration structures, and overall governance. The recent COVID-19 pandemic underscored the significance of well-prepared digital administration, particularly at the local government level, which plays a pivotal role in digital transformation. However, current comparative research on the digital transformation of state and administration https://www.zhaw.ch/de/ueber-uns/person/gerj/ (J. Gerber); https://www.zhaw.ch/de/ueber-uns/person/saxr/ lacks suficient investigation into local government levels, creating a knowledge gap on implementation and efects across Europe.
DIGILOG1 is a research project determined to close this gap. It is an international and interdisciplinary project that consists of political and computer scientists from the University of Potsdam (DE), the Vienna University of Economics and Business (AU), and the Zurich University of Applied Science ZHAW (CH). The Researchers of the project in the field of Computer Science are the contributing authors of this paper. The project is ifnanced by the Swiss National Science Foundation (SNSF / Project Nr. 200839) and Deutsche Forschungsgemeinschaft (DFG, German Research Foundation). The start of the project was in spring 2022 and the end will be in summer 2025. The research project seeks to address this above-mentioned gap by examining two key questions: • What are the dynamics, scale, and pace of digital transformation in European local governments? Is the change radical, revolutionary, incremental, or evolutionary, and are there identifiable regional diferences? • What efects does digital transformation have on these organizations, specifically in terms of output (service delivery, organization, processes, and resources), outcomes (performance and accountability), and impact (citizen acceptance, governance, and emerging tensions)? To address these questions comprehensively, data will be collected in diferent ways from all municipalities in the 46 member states of the Council of Europe. As shown in Figure 1 we collect data for the diferent communities in three ways.
In conjunction with the quantitative surveys, comparative case studies are conducted in selected municipalities, which are also part of the extended survey sample. The case studies are carried out in communities with diferent administrative cultures to capture the country-specific variance of local administrative systems. The case study approach relies on field research methods, semi-structured expert interviews, and focus groups conducted with local CEOs, Chief Information Oficers (CIOs), department heads, employee representatives, and staf. The aim is to gain in-depth insights into the internal processes and actor constellations of the respective digital transformation paths, building on the quantitative part’s interim results by capturing the municipalities’ organizational realities.
In addition to the qualitative case studies, the DIGILOG project is based on two quantitative forms of data collection: a web crawler for analyzing municipal websites and a survey among the leaders of European municipal administrations.
The survey has several objectives. The main goal is to collect information on the status of the surveyed municipal administrations’ external and internal digital transformation, from which a 1https://www.digilog-project.org/ Europe-wide index will be created. In the external domain, this primarily includes the digital service oferings of the administrations, classified into various maturity levels according to an established social science model. The categorization spans from basic information provision to options for digital interaction with administrative personnel and completely digital and seamless administrative process handling. The internal domain, on the other hand, covers aspects such as the technical equipment of the administration, forms of internal communication, data management, and the automation of processes and routine decisions.
Furthermore, the survey collects data on various other variables related to digitization. These include factors that can help explain the state of digital transformation in municipalities, such as the size and organizational form of the municipality, as well as those that can reflect the consequences of digitization, such as questions about the eficiency of administration or the satisfaction of citizens with administrative work.
Web crawling is a central component of the DIGILOG project. In addition to surveys, automatic crawling and analysis of municipal websites are part of the quantitative analysis. The results of the data analysis described below will be displayed on a dashboard within a monitoring platform. Additionally, the lists of website URLs and email addresses for surveys, if not already provided, are completed through crawling public data sources.
The monitoring platform is based on three main components that interact with each other: web crawling, data storage, and subsequent data analysis. This platform ensures monitoring of the political municipalities’ websites during the project duration. To manage the volume of data, several methods enable targeted data collection with minimal information loss. One project goal is to explore and implement the most eficient method for this task.
Data storage is ensured with two diferent database systems, a relational and a documentoriented system. The relational system stores database keys and normalized information. Complementarily, the website documents and the analysis results are stored in a documentoriented system. For analysis, clues (e.g., mention of selected services or keywords) indicating digital transformation are extracted and evaluated. Various methods from Natural Language Processing (NLP), a subfield of Machine Learning, are applied.
The analysis, in turn, can provide efective feedback to the intelligent crawler, contributing to its continuous improvement. The quality of the analysis is ensured by domain experts who interpret and contextualize the results for management, political science, and public administration.
Several relevant indices on digital service provision exist, ofering country rankings and
potentially serving as a valuable foundation for an index on local digital service provision within the
scope of this project. The European Commission publishes the Digital Economy and Society
Index Report on digital public services; however, it lacks specificity for the local government
tier [
The project described is interdisciplinary. It intersects with the research area of political Science and Information Retrieval in Computer Science. However, we only focus on the related work of Information Retrieval related to this project.
There is already work claiming to measure the level of digital transformation within local
governments. Garcia-Sanchez et al. [
Since we focus on website content to measure digital transformation, we note the importance
of existing work on website processing, classification, and embedding, which is the encoding of
data into a lower-dimensional representation in such a way that preserves some relationship in
the data. We might focus on a website’s visual or textual aspects, or even both, and leverage
machine learning for our digitalization measurements. It’s not surprising that recent work
often uses Large Language Models (LLMs) and Convolutional Neural Networks (CNNs). Other
classical machine-learning approaches rely more on feature engineering. However, they do
not generalize as well as the state-of-the-art models due to their lack of flexibility regarding
structural changes of an HTML page. A large amount of related work exists in the field of
text-based embedding and classification of websites, which might help us categorize certain
website elements. Kowsari et al. [
Visual-only classifications are, in many cases, applied to the detection of harmful content such
as propaganda of terrorism [
In text-based website classification, some approaches rely on classical machine learning [15, 16]. However, the majority are based on neural networks [17, 18, 19, 20] and the more recent approaches are transformers architecture [21, 22, 23, 24]. Most notably, [23] proposes MarkupLM for document understanding tasks based on the raw text and markup language, which is also used to code websites.
A mixed approach using both textual and visual features can be seen in [25] and [26]. The ladder encodes multiple parts of a website, such as a screenshot and metadata, and combines them to feed it into a neural network as input. The model is trained to categorize websites into 14 diferent classes. While previous work gives insight into how websites are processed and represented numerically, we must apply this knowledge to our specific data. How exactly website data is handled is not a solved problem. Kiesel et al.[27], for example, compares diferent web page segmentation algorithms. Dividing the page into individual segments might provide more concentrated information sources for our future algorithms. Finally, recent AI chatbots such as ChatGPT or open-source variants are capable of understanding a wide range of instructions. Recent developments have made it possible for the models to even react to image input while understanding user instruction, making them large multimodal models. They are foundation models that can be used in a variety of ways, and they can understand website code as well as screenshots. As development continues, it is becoming easier to use these models for automatic extraction, summarization, analysis, and categorization of municipality websites. As these models generate text, natural language analysis is essential.
The field of our work in this project consists of two parts: • The URL gathering consists of the following questions: Has the municipality a website, and if so, what is the URL? Furthermore, the retrieved URLs must be distinguished from non-municipality URLs to eliminate false positives. • The website must be preprocessed (website segmentation, selection of relevant data, and removal of noisy data) and processed. The municipality website must be assessed based on the criteria defined by political scientists. A classifying model must be capable of detecting certain features if they exist on this website.
Assessing a website requires a semantic understanding of a website by the machine learning model used to process the Websites. Whether it is URL classification (specifically discerning municipality websites from others), topic modeling (classification of services), or e-service detection on web pages, a robust foundation in embedding is essential. In our previous work, we conducted not yet published experiments with general pre-trained webpage embedding models and developed a basic embedding method to efectively diferentiate municipality websites from non-municipality ones. All methods yielded very good results, with the more complex ones resulting in slightly better results. However, it’s crucial to acknowledge that basic embeddings demonstrated a faster processing speed than more complex models, a significant consideration given the vast number of websites in our study. We additionally evaluated diferent data sources concerning their completeness of data. The categories evaluated were search engines, encyclopedias, and blind requests with fabricated URLs based on certain patterns. The retrieved URLs partially consisted of wrong URLs that did not belong to the local government or municipality. Although the URL appeared to be correct in many of those cases, containing the municipality name, the content was of another topic such as tourism, airports, other oficial organizations in this municipality, or even completely unrelated content to the municipality. Thus, an automated distinction and classification by analyzing the website’s content was required.
Furthermore, as mentioned in Section 1.4, there are many ways of measuring digitization. In a conference paper, we defined three key aspects of our analysis, which consisted of diferent indices. The categories are Service Maturity (measurement of provision of information, communication possibility, and transactions), Usability (evaluation of accessibility and convenience of use), and Technical Maturity (evaluation of security and privacy). This index was published in a conference paper [28]. We tested the index on a sample of municipality websites and are currently working on implementing and applying it to the whole data set. Looking ahead, our plan encompasses the application of webpage embedding techniques for e-form detection, including webpage segmentation and relevant information extraction. Further, we plan to leverage large Language Models for topic modeling of webpages and webpage content. This approach aims to further automate the process of monitoring the digital transformation of European communities.
This work is supported by Grant No. GR 200839 of the Swiss National Science Foundation (SNF) and German Research Foundation (DFG) for the research project “Digital Transformation at the Local Tier of Government in Europe: Dynamics and Efects from a Cross-Countries and Over-Time Comparative Perspective (DIGILOG)”. gory classification of websites based on image content, IEEE Computational Intelligence Magazine 10 (2015) 30–41. Publisher: IEEE. [13] L. Espinosa-Leal, A. Akusok, A. Lendasse, K.-M. Björk, Website classification from webpage renders, in: Proceedings of ELM2019 9, Springer, 2021, pp. 41–50. [14] D. López-Sánchez, J. M. Corchado, A. G. Arrieta, A CBR system for image-based webpage classification: case representation with convolutional neural networks, in: The Thirtieth International Flairs Conference, 2017. [15] V. K. Bhalla, N. Kumar, An eficient scheme for automatic web pages categorization using the support vector machine, New Review of Hypermedia and Multimedia 22 (2016) 223–242. Publisher: Taylor & Francis. [16] G. Matošević, J. Dobša, D. Mladenić, Using machine learning for web page classification in search engine optimization, Future Internet 13 (2021) 9. [17] E. Buber, B. Diri, Web page classification using RNN, Procedia Computer Science 154 (2019) 62–72. Publisher: Elsevier. [18] B. Y. Lin, Y. Sheng, N. Vo, S. Tata, Freedom: A transferable neural architecture for structured information extraction on web documents, in: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 1092–1102. [19] A. K. Nandanwar, J. Choudhary, Semantic features with contextual knowledge-based web page categorization using the GloVe model and stacked BiLSTM, Symmetry 13 (2021) 1772. [20] Y. Zhou, Y. Sheng, N. Vo, N. Edmonds, S. Tata, Simplified dom trees for transferable attribute extraction from the web, arXiv preprint arXiv:2101.02415 (2021). [21] X. Chen, Z. Zhao, L. Chen, D. Zhang, J. Ji, A. Luo, Y. Xiong, K. Yu, WebSRC: a dataset for web-based structural reading comprehension, arXiv preprint arXiv:2101.09465 (2021). [22] A. Gupta, R. Bhatia, Ensemble approach for web page classification, Multimedia Tools and Applications 80 (2021) 25219–25240. Publisher: Springer. [23] J. Li, Y. Xu, L. Cui, F. Wei, MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding, 2022. URL: http://arxiv.org/abs/2110.08518, arXiv:2110.08518 [cs]. [24] A. K. Nandanwar, J. Choudhary, Contextual Embeddings-Based Web Page Categorization Using the Fine-Tune BERT Model, Symmetry 15 (2023) 395. URL: https://www.mdpi.com/ 2073-8994/15/2/395. doi:10.3390/sym15020395, number: 2 Publisher: Multidisciplinary Digital Publishing Institute. [25] R. Bruni, G. Bianchi, Website categorization: A formal approach and robustness analysis in the case of e-commerce detection, Expert Systems with Applications 142 (2020) 113001.
Publisher: Elsevier. [26] S. Lugeon, T. Piccardi, R. West, Homepage2Vec: Language-Agnostic Website Embedding and Classification, Proceedings of the International AAAI Conference on Web and Social Media 16 (2022) 1285–1291. URL: https://ojs.aaai.org/index.php/ICWSM/article/view/19380. doi:10.1609/icwsm.v16i1.19380. [27] J. Kiesel, L. Meyer, F. Kneist, B. Stein, M. Potthast, An Empirical Comparison of Web Page
Segmentation Algorithms, 2021, pp. 62–74. doi:10.1007/978- 3- 030- 72240- 1_5. [28] J. Marquardt, J. Gerber, J. Machljankin, C. Kaiser, & R. Steiner, Applying web crawling for data collection in the social sciences - Opportunities and limits using the example of digital transformation in European local governments, Zagreb, Croatia, 2023.