In comparative law, standardization and harmonization of law are interrelated concepts that aim to increase legal certainty, eficiency, and accessibility. Standardization focuses on creating uniform rules and procedures within a specific jurisdiction, while harmonization seeks to align legal frameworks across jurisdictions. This research aims to investigate the impact of standardization and harmonization on legal systems through the application of AI methods. Two case studies were analyzed using machine learning, natural language processing, and process mining techniques. The paper also reports a brief discussion of the methods adopted and early results from two legal domain experts.
Comparative law, by examining the similarities and diferences between various legal systems, plays
a crucial role in promoting the harmonization and standardization of legal processes across diferent
jurisdictions [
In recent years, legal decision making has benefited from analyzing large volumes of data using
computational tools, which allow knowledge to be extracted automatically [
CEUR
ceur-ws.org
This article summarizes and discusses some recent research contributions in the practical case of automated analysis of legal harmonization and public procurement law in the European Union (EU). The methods we present in the article and the initial results obtained from ongoing research are encouraging towards further adoption of advanced information technology tools in comparative law. Indeed, knowledge management and process mining (PM) tools make it possible to facilitate comparative analysis to identify similarities, diferences, and possible convergences between diferent legislation. Finally, two legal experts provided valuable support at all stages of the research, playing a key role in analyzing both methods and results.
The article is structured as follows. Section 2 details the background of the harmonization of European legislation and public procurement studies. Section 3 details the methodologies applied in the analysis, ifnally proposing a brief review of the initial results obtained in Section 4. Finally, Section 6 concludes the articles with future work.
Comparative law involves the systematic comparison of diferent legal systems to gain a deeper understanding of the law’s functions by analyzing the similarities and diferences between various legal systems. The discipline plays a crucial role in fostering international legal harmonization, as well as in understanding the diverse legal frameworks governing the acquisition of goods, services, and works by public entities across diferent countries. In the past decades the use of quantitative methods aside the more traditional qualitative methods of analysis have been very much debated in comparative law literature [10]. Legal harmonization refers to the process of aligning laws and regulations across diferent jurisdictions to create a more uniform legal framework. Public procurement law governs how public authorities purchase goods and services. The legal standardization of public procurement processes enhances transparency, eficiency, and compliance by establishing uniform rules and procedures, reducing complexity and disputes, and facilitating cross-border participation and fair competition.
In recent years, the integration of computational tools and applications has opened new avenues for comparative law research, enabling more eficient and comprehensive analyses [ 11, 12]. Information system databases collect valuable data and information that can be analysed with computer techniques and algorithms to extract knowledge. The data used in legal process analysis can be extensive and scattered, particularly for activities like calls for procurement. Online repositories that collect information on European-level procurement ofer a rich dataset for researchers to explore.
PM ofers a data-driven approach to analyze business processes [ 13]. Data with timed events from information systems can be transformed into event logs to capture the sequence of activities within a system. Dataset can be merged to obtain more events and information on a process. This is the case, for instance, of a French dataset on public procurement described in [14], which combines dataset at the EU and national level. Event logs enable to uncover the actual processes executed (process discovery) or the diferent paths (variant analysis), as well as to compare the discovered processes against predefined models to ensure compliance and identify deviations (conformance checking). Such analysis allows the identification of bottlenecks, ineficiencies, and opportunities for automation to enhance overall operational eficiency.
To improve understanding of legal processes, by applying PM techniques, organizations can gain a deeper understanding of their operations, enabling them to make data-driven decisions and improve compliance. Legal issues have so far not been much explored through PM techniques, perhaps because of the dificulty of finding information systems with timed legal events. Nevertheless, some studies investigated legal process discovery to analyze the judicial performance of a Brazilian court [15], the public procurement processes in Croatia [16], the automated analysis of Italian [17] or European public procurement processes [18]. Process oriented approaches enable the identification and mitigation of compliance risks by providing a detailed view of actual process execution and deviations from standard operating procedures [19]. The use of PM for audit and assurance provides auditors and compliance oficers with a robust tool to assess the efectiveness of internal controls and identify areas where process improvements can enhance compliance [20]. PM can help organizations to demonstrate GDPR compliance by providing visibility into data processing activities and identifying potential data breaches [21].
Knowledge management in the legal field has been developed in several research directions. A first area of research focused on the representation and reasoning of legal knowledge in a structured way, such as through deontic logic, rule-based systems, and legal argumentation [22]. Another relevant research area focused on the development of advanced systems for organizing, storing, and retrieving legal documents including the use of semantic search technologies and AI to improve access and accuracy of legal information [23]. A pipeline for exploit the domain expert modus operandi in similarity case matching has been introduced by [24]. Semantic web and legal ontologies help to structure and link legal concepts for allowing machines to understand and reason with legal data [25] and to enhance integration and interoperability of legal systems [26]. This kind of research supports applications like automated legal advice and intelligent legal research [27].
Recent research in the fields focused on AI, ML, and NLP applications in legal practice to automate legal tasks such as document analysis or fraud detection[28], legal advice [27], legal outcome predictions [29, 30] and explain them [31].
This section outlines the methodologies employed in standardization and harmonization research, introducing two case studies and their respective datasets, as illustrated in Figure 1. The following two sections provide an overview of research techniques used in comparative law and international law, supported by two practical applications.
As stated above, public procurement is a key area within legal standardization, as it involves creating and enforcing uniform legal frameworks that govern how public contracts are awarded and managed. In EU, this process facilitates consistent practices across diferent national countries and jurisdictions. This section describes the practical application of PM for examining public procurement processes. Figure 2 summarises the approach, starting with the collection of timed data (1st step) from legal public repositories. In particular, we refer to a dataset at the EU level, i.e. Tender Electronic Daily TED1, merged with data from a National repository on public procurement, i.e. ANAC2. The public procurement dataset from 2016 to 2023 covers 3 million tenders is available online and has been described in [32]. TED data includes five events (publication, participation, award, contract-start, contract-end), from the creation of the procurement up to the end of the contract. Italian dataset adds some events in each procurement process, e.g. the contracting company, the presence of variants during the process, or the state of progress of the procurement. The next step is the construction of the event log (2nd step), with information on each case (case identifier), the name of the event or the activity, and the corresponding timestamp. Eventually, some attributes can be added at the level of case or trace. Next, the enrichment of the event log (3rd step) can occur in several directions: new events with new dates as well as new attributes can be extracted from the procurement text (at the case or trace level). This task can be performed with both traditional techniques, such as regular expressions, as well as modern ones such as large language models [27]. PM techniques (4th step) are applied to the final event log, enabling the analysis of the legal process control-flow with process discovery and variant analysis, as well as the automatic analysis of compliance against an ideal model or the application of AI methods such as forecasting.
While the comparison between reference legislation (e.g., EU directives) and national legislation requiring alignment necessitates substantial legal expertise, this study endeavours to explore the feasibility of incorporating automated techniques into this process. In this work, we describe a pipeline for applying textual similarity metrics to verify the correspondence of content between European and national legislation, which represents a preliminary level of legal harmonization in Europe.
This NLP pipeline has been explored on a dataset of an EU-funded project3 that is developing a platform designed as a legal comparison system. This system aims to quantify the extent to which an EU member state has transposed a certain segment of European legislation into its legislation. The process starts with a pre-processing phase, during which European and national legislative segments are subjected to cleaning and normalization to ensure consistent text formatting. This step is followed by the generation of semantic paragraph and word embeddings, both using the nlpaueb/bert-base-uncasedeurlex model [33]. Finally, a harmonization metric is computed between the European legislative segments and their corresponding national transpositions, quantifying the alignment level within the dataset. The Harmonization Index (HI) is a metric designed to assess the degree of semantic overlap between two legal texts. It serves as a similarity measure, identifying and emphasizing common elements to compare diferent legal texts, considering both symmetrical and asymmetrical relationships. The HI 1https://ted.europa.eu/en 2https://dati.anticorruzione.it/opendata 3The FACILEX project web site: https://site.unibo.it/facilex/en/the-project was introduced to measure the degree of coherence or uniformity between legal texts across diferent jurisdictions or entities. This index is particularly valuable in the legal domain because it provides a quantitative means to assess the alignment between national laws and supranational regulations, such as those adopted at the European level, or between local laws and federal regulations [34]. In the context of legal texts, the HI plays a critical role in eforts to harmonise laws, especially in areas such as European law or international law, where there is a continuous drive to align the legal frameworks of various countries. This is particularly relevant in public procurement and regulatory compliance processes, where harmonisation can ensure smoother cross-border transactions and reduce legal uncertainties [35]. The index also serves as an empirical tool to monitor the progress and efectiveness of harmonisation eforts by providing an indication of the level of legal alignment across jurisdictions [ 36].
This research leverages the HI to gauge the extent to which a European legal act, such as the EU Council Framework Decision on the European Arrest Warrant, has been implemented at the national level. The computation of the HI for a given EU text and its corresponding national text is a complex task as each section of an EU legal act can be transposed into national law through a varying number of legal sources, ranging from zero to potentially numerous sources. The HI is computed for each European legislative segment in two ways: symmetric and asymmetric. The symmetric version, computed at the paragraph level, represents the average cosine similarity4 between the European segment and each of its national transpositions. Conversely, the asymmetric version considers the highest cosine similarity between each word in the European segment and all words within the corresponding national transposition. This version incorporates a weight function applied to each word, reflecting its relative importance in the overall computation. Consequently, the HI is a cumulative measure that accounts for all instances of transposition for each part of the EU text. Following are the symmetric and asymmetric cosine similarity formulas; in the symmetric form, the denominator considers the magnitudes of both vectors, while in the asymmetric form, only the magnitude of vector A is considered.
Cosine Similarity (Symmetric) = Cosine Similarity (Asymmetric) = A ⋅ B ‖A‖2 A ⋅ B ‖A‖‖B‖
This section describes the first results with a brief discussion. A description of the initial dataset opens each presentation. Finally, a brief summary of the challenges collected by the domain experts involved in the analyses closes the section. Note that the initial TED dataset is publicly available5, while the graphs obtained from the event log come from Fluxicon’s DISCO tool6.
TED Dataset. The extraction of data from the TED web repositories7 allows to reach 27,252 records for Italian cases in the years 2016 - 20228. For the same years, 99,915 French and 47,743 Spanish cases were also extracted; according to domain experts, the three member states have similar legal review systems. 4Cosine similarity measures the similarity between two non-zero vectors by calculating the cosine of the angle between them, ranging from -1 (completely dissimilar) to 1 (completely similar). It can be symmetric, where the similarity between vectors A and B is the same as between B and A, or asymmetric, where the order of the vectors afects the result. 5https://github.com/roberto-nai/PLC2024 6https://fluxicon.com/disco 7https://data.europa.eu/data/datasets/ted-csv?locale=en 8The time frame is fairly consistent with the validity of the Italian public procurement code, which came into force in April 2016 and was replaced on 31 March 2023
Event log construction. The event log is obtained by extracting from the TED dataset the relevant data needed to construct the traces of each procurement. Figure 3 contains an extract of the event log obtained; in addition to the basic elements (Case ID, Activity and Timestamp), the traces also contain the attributes Sector (“Works”, “Services”, “Supplies”), Amount, NUTS9 and Country. Event log enrichment. Event log enrichment with respect to dates can use traditional techniques, such as regex, as well as large language models. In our case study, new events captured from the text ifles connected to every procurement is the bid-opening; in TED, it involves the formal process where received bids are unsealed and examined after the submission deadline.
Further enrichment with national dataset, such as ANAC for example, allows for discover more detailed insights into the process beyond the creation, award, start/end of contract phases; through the ANAC dataset, it is possible to add other procurement events, such as the presence of a variant (i.e., a modification of the initial process), the verification of the status of works, any sub-contracts with other economic operators.
Process mining techniques. Figure 4 illustrates an example of the processes uncovered by the data, which made it possible to immediately identify the similarities and diferences in the three process diagrams for each country. The diagrams illustrate the average duration between two activities, indicated by the thickness of the arcs, which also helps identify bottlenecks easily. The average time from the procurement notice to participation is approximately 36 days (minimum 34, maximum 39), while the average time to determine the procurement winner is around 45 days (minimum 36, maximum 54). The most prolonged part of the process is the decision-making phase for the winning procurement, which takes about 19.3 months. In addition to the timing, loops can be observed in the Spanish process, particularly in the PUBLICATION and PARTICIPATION activities. According to domain experts, this occurs because contracting authorities may re-publish a procurement notice, prompting participation to take place again based on the new publication. Another observation is that the AWARD activity does not always occur; according to domain experts, this is due to ’framework agreements’ (multi-year contracts across several procurements that are published but automatically awarded).
Figure 5 presents, on the other hand, the newly identified BID-OPENING event extracted from the procurement notices text. This event demonstrates that the opening of received bids occurs promptly (transition from PARTICIPATION to BID-OPENING). At the same time, most of the time required to award the contract to the economic operator is consumed by the decision-making process of the contracting authorities (transition from BID-OPENING to AWARD).
Finally, Figure 6 illustrates how the TED event log was extended with further activities using the ANAC dataset, which contains a specialisation of the TED data with more activities like VARIANTS, SUBCONTRACTS, etc. 9https://www.europarl.europa.eu/factsheets/en/sheet/99/nomenclatura-comune-delle-unita-territoriali-statistiche-nuts
TT Dataset. The dataset comprises Transposition Tables (TT) aligning EU legal acts, namely Decision 2002/584, Directive 2014/41, and Regulation 2018/1805, and their corresponding national transpositions as annotated by legal experts. The project focused on French, German, Italian, Portuguese, and Spanish national implementations for each of the above-mentioned European documents.
This dataset is structured into documents, articles, and paragraphs, each of which may or may not have a corresponding transposition, which is in turn classified as either explicit or implicit. This classification aligns with diferent levels of national implementation, which include full implementation, partial implementation, and instances where no implementation has occurred at all.
Despite its multilingual nature, the dataset is fully translated into English and available on the project platform, in both tabular and JSON formats, allowing for easier computational analysis. Harmonization Index results. Segment-level scores were computed following both a symmetrical and asymmetrical approach, resulting in two diferent distributions (Figures 7 and 8). The symmetrical version exhibited a mean score of 0.93 and a median of 0.95, while the asymmetrical version yielded a mean of 0.73 and a median of 0.69.
Manual analysis indicates that the symmetrical version of the Harmonization Index aligns more closely with the human judgment of legislative transposition completeness than its asymmetrical counterpart. This suggests the symmetrical version is preferable for evaluating transposition levels. However, this finding prompts further investigation into how the index’s performance compares to alternative approaches operating at diferent textual granularities.
PM ofers a granular view of how legal processes are actually executed. By analyzing event logs, legal experts appreciated the possibility to reveal hidden patterns, ineficiencies, and opportunities for improvement. With respect to legal standardization, the benefits ofered by automated analysis mainly concern the identification of legal process variants and compliance issues. By comparing the actual legal process execution to the ideal process model, PM can easily detect where and why deviations occur. At the same time, inconsistencies can be detected, e.g. areas where diferent jurisdictions apply laws diferently, as the procurement process case study demonstrated.
The methodology enables country-based comparative analyses across diferent time-frames, although data availability varies significantly across countries on platforms like TED. This disparity can be attributed to diferences in resources and priorities in data sharing among nations. The analysis revealed that procurement procedures cannot be reduced to mere legislative frameworks - they require both quantitative and qualitative approaches involving key stakeholders.
In the Italian context, PM has proven particularly valuable for monitoring compliance with recent amendments to the Public Contracts Code. The analysis showed that for services, the mean time between contract award and start is 36.4 days, for works 55.2 days, and for supplies 24.2 days. These timelines can be compared against regulatory requirements, such as the standstill period of 35 days established by Legislative Decree 50/2016, with exceptions for sub-threshold services and supplies.
The process analysis identified several possible bottlenecks, particularly between publication and award phases (approximately 80 days) and between award and contract start (22.5 days). The data revealed that contract duration varies significantly by type - works averaging 31 months, supplies 9.1 months, and services 5.3 months. These diferences reflect the distinct nature and regulatory requirements of each category.
Comparative analysis between countries showed that German procedures tend to be longer than Italian ones, though Germany publishes significantly more procedures on TED. Portugal demonstrated faster processes, albeit with a smaller sample size. With respect to harmonization, such comparisons can be performed via performance metrics, measuring both the similarity of legal frameworks and the duration of expected activities in the legal process flow.
The methodology has proven particularly efective in verifying compliance with newly imposed deadlines, such as those introduced by Law Decree 76/2020 and now incorporated into Legislative Decree 36/2023, which establish specific time-frames for diferent procurement procedures. To address procurement delays, several improvements can be targeted at critical phases. During the initial evaluation phase (80 days), digital pre-evaluation tools and simplified selection methods can streamline the process. For the negotiation period (22.5 days), digitization of procedures and reduced paper-based interactions are recommended. Contract duration eficiency (32.4-36 months) can be improved through early completion incentives, performance penalties, and project management software. These solutions align with legislative objectives while maintaining process quality.
The present work showed the feasibility, with the methodologies and initial results, of automated analysis in comparative law. The two examples presented in the paper, concerning both standardization and harmonization, have identified distinct challenges. An initial challenge lies in data quality, as the accuracy and completeness of legal data are crucial for both methodologies. Furthermore, the integration of domain experts is indispensable, aligning with a human-in-the-loop approach. Despite technological advancements, legal expertise remains vital for providing context, judgment, and oversight. Finally, the interpretability of results presents a significant challenge. While AI and PM can identify patterns, understanding the underlying reasons for these patterns in the legal domain can be complex. As a potential future work, we aim to investigate further the adoption of additional features from openly available datasets to improve the process-oriented analysis.
Finally, it is essential to recognise the significance of the AI Act [ 37], particularly with regard to the transparency obligations imposed on AI systems. These requirements are fundamental to ensure that AI technologies operate in a manner that is accountable and understandable to all stakeholders but also play a key role in fostering public trust in AI innovations. By adhering to these transparency standards, organisations can demonstrate compliance with regulatory frameworks while promoting ethical and responsible AI development.
The research work in this article was partially conducted as part of the following projects: Circular Health European Digital Innovation Hub (CHEDIH) - Grant Agreement n. 101083745; FACILEX - Grant Agreement n. 10108963; GECOPAM and EXTRAI projects - Bando a Cascata PE FAIR Spoke 8. [9] R. Nai, E. Sulis, E. Marengo, M. Vinai, S. Capecchi, Process mining on students’ web learning traces: A case study with an ethnographic analysis, in: O. Viberg, I. Jivet, P. J. Muñoz-Merino, M. A. Perifanou, T. Papathoma (Eds.), Responsive and Sustainable Educational Futures - 18th European Conference on Technology Enhanced Learning, EC-TEL 2023, Aveiro, Portugal, September 4-8, 2023, Proceedings, volume 14200 of Lecture Notes in Computer Science, Springer, 2023, pp. 599–604. doi:10.1007/978-3-031-42682-7\_48. [10] R. Whalen, Quantitative methods in comparative law, Elgar Encyclopedia of Comparative Law (2023). [11] D. Audrito, E. Sulis, L. Humphreys, L. Di Caro, Analogical lightweight ontology of eu criminal procedural rights in judicial cooperation, Artificial Intelligence and Law 31 (2023) 629–652. [12] E. Sulis, L. B. Humphreys, D. Audrito, L. Di Caro, Exploiting textual similarity techniques in harmonization of laws, in: S. Bandini, F. Gasparini, V. Mascardi, M. Palmonari, G. Vizzari (Eds.), AIxIA 2021 – Advances in Artificial Intelligence, Springer International Publishing, Cham, 2022, pp. 185–197. [13] W. M. P. van der Aalst, Process Mining - Data Science in Action, Second Edition, Springer, 2016.
doi:10.1007/978-3-662-49851-4. [14] L. Potin, V. Labatut, P. Morand, C. Largeron, FOPPA: an open database of french public procurement award notices from 2010-2020, Scientific Data 10 - 303 (2023). doi: https://doi.org/10.1038/ s41597-023-02213-z. [15] A. J. Unger, J. F. dos Santos Neto, M. Fantinato, S. M. Peres, J. Trecenti, R. Hirota, Process mining-enabled jurimetrics: analysis of a brazilian court’s judicial performance in the business law processing, in: J. Maranhão, A. Z. Wyner (Eds.), ICAIL ’21: Eighteenth International Conference for Artificial Intelligence and Law, São Paulo Brazil, June 21 - 25, 2021, ACM, 2021, pp. 240–244. doi:10.1145/3462757.3466137. [16] K. Rabuzin, N. Modrušan, S. Križanić, R. Kelemen, Process mining in public procurement in croatia, in: B. Lalic, D. Gracanin, N. Tasic, N. Simeunović (Eds.), Proceedings on 18th International Conference on Industrial Systems – IS’20, Springer International Publishing, Cham, 2022, pp. 473–480. doi:10.1007/978-3-030-97947-8_62. [17] R. Nai, I. Fatima, G. Morina, E. Sulis, L. Genga, R. Meo, P. Pasteris, AI applied to the analysis of the contracts of the italian public administrations, in: Proceedings of the Italia Intelligenza Artificiale - Thematic Workshops co-located with the 3rd CINI National Lab AIIS Conference on Artificial Intelligence (Ital IA 2023), Pisa, Italy, May 29-30, 2023, 2023, pp. 255–260. URL: https://ceur-ws.org/Vol-3486/100.pdf. [18] R. Nai, E. Sulis, L. Genga, Automated analysis with event log enrichment of the european public procurement processes, in: T. P. Sales, J. Araújo, J. Borbinha, G. Guizzardi (Eds.), Advances in Conceptual Modeling - ER 2023 Workshops, Lisbon, Portugal, November 6-9, 2023, Proceedings, volume 14319 of LNCS, Springer, 2023, pp. 178–188. doi:10.1007/978-3-031-47112-4\_17. [19] W. M. P. van der Aalst, Process Mining - Discovery, Conformance and Enhancement of Business
Processes, Springer, 2011. doi:10.1007/978-3-642-19345-3. [20] M. J. Jans, M. Alles, M. A. Vasarhelyi, Process mining of event logs in auditing: Opportunities and challenges, SSRN 1578912 (2010). doi:dx.doi.org/10.2139/ssrn.1578912. [21] R. Zaman, M. Hassani, On enabling GDPR compliance in business processes through data-driven solutions, SN Comput. Sci. 1 (2020) 210. doi:10.1007/S42979-020-00215-X. [22] D. Gabbay, J. Horty, X. Parent, R. Van der Meyden, L. van der Torre, et al., Handbook of deontic logic and normative systems, College Publications, 2021, 2021. [23] C. Sansone, G. Sperlí, Legal information retrieval systems: State-of-the-art and open issues, Inf.
Syst. 106 (2022) 101967. doi:10.1016/J.IS.2021.101967. [24] R. Mignone, I. Spada, C. Bonfanti, M. Colombino, G. Iacobellis, L. J. M. Zaharia, M. Molinari, I. A. Amantea, E. Sulis, L. D. Caro, G. Boella, Augmented reading and similar case matching: from legal domain experts’ modus operandi to a computational pipeline, in: F. Toyoshima, M. Katsumi, G. Righetti, S. D. Giorgis, M. M. Hedblom, O. Kutz, B. Brodaric, M. Grüninger, T. Hahmann, D. D. Simon, M. Lange, H. Küçük-McGinty, A. S. Simon, R. Cameron, B. Gajderowicz,