Today, interoperability between software systems is in full swing. This aspect allows information to be exchanged and reused between diferent data sources, as well as between software. One of the fields of application is state governments that share open public data and facilitate joint management and transparency between organizations and citizens. This paper addresses the importance of open data interoperability in the governmental context. This article presents a review of various open government initiatives in Latin America, in addition, it analyzes a sample of sets of open public data catalogs extracted from government web portals in Latin America (Brazil, Mexico, Colombia, Uruguay, Chile, and Argentina), to study possible shortcomings in aspects of accessibility, interoperability and contextualization of these to mitigate them in the future. For said analysis, a self-developed software tool is used that uses validation criteria based on initiatives of the Argentine Republic. Finally, an analysis of the findings is carried out to propose aspects of improvement through good practices in their treatment and help in the standardization of certain technical and structural aspects and levels of the exchange processes.
In today’s global landscape, the digital era has reshaped how we access, distribute, and use information. Information, in this setting, is not only seen as a precious asset but also as a vital tool for making decisions, generating knowledge, and enhancing government transparency. Interoperability, defined as the capacity of various systems and applications to exchange and process information seamlessly, stands out as a central element in this evolving paradigm.
Latin America, despite its rich cultural, political, and economic diversity, several nations in the region, such as Brazil, Mexico, and Colombia, have shown an increasing dedication to enhancing the interoperability of their information infrastructures. However, these praiseworthy initiatives still encounter inherent challenges related to data standardization, updating, and structuring.
This study sets out to provide a thorough and critical examination of open data catalogs from government websites in selected Latin American nations. Using a purpose-built software tool, we will scrutinize these catalogs to pinpoint areas of improvement, shortcomings, and exemplary practices. Moreover, we will aim to place these discoveries within the broader context of global movements in interoperability and data openness.
Through this research, we hope to add to the international scholarly body on interoperability and open data, delivering valuable perspectives on the current state and future outlook of this topic in Latin America, a region with its own set of unique digital public management challenges and opportunities.
The structure of this work presents a brief introduction to the subject, then the problem is developed in this context, later, a comparative study of some Latin American countries on this situation is shown, and then an analysis of data collected from the countries is explained. government websites of the countries studied, in which some data catalogs are analyzed using the proposed software tool, which makes it possible to identify some shortcomings in the contents of these public files. Finally, it culminates with the analysis and proposal of good practices to facilitate the exchange of data between diferent organizations.
The concept of open government is booming in diferent countries around the world. This approach consists of an open society, that is, a society in which the state of a country and its citizens are the main actors in which they participate in various activities that benefit diferent people in social and political aspects. This approach presents an active participation of citizens in public afairs and has some pillars that are essential to consider, these are: citizen transparency, accountability, citizen participation, and the use of various technologies and innovations that favor this context.
Open government refers to the freedom of information that is available to citizens and
government authorities, this makes governments more transparent and accountable in their
activities. To carry out the latter, it is necessary to develop freedom of information laws to
improve transparency, accountability, and public trust [
One of the most important associations at the international level is the Open Government Partnership (OGP)1, which is based on the idea that an open government is more accessible, more receptive, and accountable to citizens and that it improves the relationship between people and their government has long-term benefits for all. OGP includes members from various countries who work together to cocreate two-year action plans with concrete steps (commitments) on a wide range of issues.
Several points are related to the commitment of governments to improve services, support citizen participation, the administration of public resources, and promote public innovation. This last point is essential to increase the availability of information, both to citizens and to various government agencies. It is important to keep in mind the importance of awareness, the dissemination of activities, as well as having digital platforms that provide support in the publication of data. This leads to the need to use open standards to promote civil society access 1Open Government Partnership (OGP). https://www.opengovpartnership.org/es/about/approach/ to public data, as well as to facilitate the interoperability of government information systems2. Innovative technologies ofer a wonderful opportunity to share information and provide public services with greater transparency.
That information on the activities of public entities is created and is available to the public,
on time and in open data formats with no limits to reuse, allowing an initiative-taking work
methodology by public entities [
The next section discusses the importance of interoperability and its benefits. Interoperability is indispensable for the practical cooperation of digital public services. It is an essential aspect to improve the eficiency and efectiveness of the provision of services, and at the same time, reusing interoperable government technological solutions could allow for reducing costs in technological platforms. It is important to understand that in addition to the public data that can be ofered on public platforms, a strategic analysis must be presented on the digital service infrastructures that are available. It is necessary to analyze the access to digital resources; service infrastructure; platforms that will be provided to the community for the interconnection of data; access to reusable public sector information; and crossborder interoperable online services3.
Although the concept of interoperability provides a collaborative space in which governments expose and consume services constantly, it can be defined that the success of this proposal lies in the constant feedback between the parties. As part of the public digital transformation, it is essential to have the appropriate regulatory framework and technology that allows the secure exchange of documents and data between public entities, so this is one of the problems to be considered in each of the governments. I would like to use this approach. Proper use would allow citizens to access the services provided by the administration, avoiding having to submit the same information repeatedly. In other words, one of the shortcomings to work on in this context is data redundancy, since it should prevent the administration from generating data replicas, keeping the authentic source safe, and making them available to those who require groups of data to fulfill your task-interconnection of data; access to reusable public sector information; and cross-border interoperable online services4.
The investigations refer to the fact that there are several challenges and impediments to open government management, identified in three categories: impediments to data access (data availability and searchability), data use (related to problems with usability impediments, understandability, quality, compatibility and metadata) and data deposition, that is, dificulties 2Open Government Partnership (OGP). https://www.opengovpartnership.org/process/joining-ogp/ open-government-declaration/
3Gobierno de España. https://administracionelectronica.gob.es/pae_Home/pae_Estrategias/pae_lineas_ ccoperacion/pae_Cooperacion_Internacional/pae_Interoperabilidad_Coop_Inter.html
4Gobierno de España. https://administracionelectronica.gob.es/pae_Home/pae_Estrategias/pae_lineas_
ccoperacion/pae_Cooperacion_Internacional/pae_Interoperabilidad_Coop_Inter.html
in the interaction with the data provider and the challenges to release the data sets, as well as
those of communication between the end user and the data of the entity [
One of the approaches proposed by the government regarding the aforementioned aspects is to make datasets openly available to the public on government portals, to disseminate citizen transparency and strengthen the link between government and citizens. One of the main problems encountered with these open data portals is the inconsistency of some data since not all public organizations follow a particular standard for the treatment of dataset content. There are no specific mandatory international standards. This drawback leads to poor data quality, and as a result, various problems can occur when trying to reuse data for review, analysis, or interoperability across data sources. To improve the quality of the open public data provided, it is necessary to introduce a set of standards or aspects that are consistent and sustainable over time, which will give greater possibilities of analysis to the information provided, as well as new knowledge about possible preventive measures. This will allow avoiding flaws in the data analysis process and, in addition, avoiding erroneous or biased results that lead to poor decision-making.
It is important to consider that in open data portals, the availability of these does not always
coincide with their quality, unfortunately, today it continues to be a dificulty and is a great
challenge for public policies [
In Latin America, the interoperability of government systems continues to be a critical issue. Although projects to improve interoperability have been implemented in many countries in the region, significant challenges still exist. One of the main challenges is the integration of systems at the municipal and regional level since in many cases each municipality or region uses its computer systems, which makes communication and information exchange between government entities dificult.
Another major challenge is the lack of data standardization. Many government systems use diferent formats and protocols to store and transmit information, making systems integration dificult. Data standardization is critical to efective interoperability, as it allows computer systems to communicate efectively. The lack of common standards can make it dificult to compare data and generate consolidated reports, which can afect informed decision-making. In addition, information technology procurement and contracting processes are often complex and lengthy, making it dificult to adopt more modern and eficient technologies.
To address these challenges, further work on data standardization and systems integration is needed. Governments in Latin America can promote the adoption of common data standards and communication protocols to improve interoperability between government systems. In addition, it is important to improve the training and education of government personnel so that they can efectively use information systems and work together to improve the eficiency and transparency of public administration. A focus on the education and training of government personnel is needed to ensure that they can efectively use information systems and work together to improve the eficiency and transparency of public administration.
The selection of countries such as Brazil, Mexico, Colombia, Uruguay, Chile, and Argentina for this study is based on rigorous criteria that reflect their prominent position in the Latin American landscape and their commitment to adopting digital policies. Brazil, Mexico, and Colombia, for instance, are leading economies in the region and have proven to be pioneers in the adoption of digital initiatives. Meanwhile, Uruguay, Chile, and Argentina have positioned themselves as models in terms of open data policies, showcasing notable advancements in promoting interoperability. This selection of countries aims to ofer a balanced and representative view of current interoperability trends in Latin America. Geographic representativeness, the magnitude of open data initiatives, and the relevance of their policies on the international stage were prioritized. In this way, the study seeks to provide valuable insights and contribute to the international academic discourse on the subject, considering the particularities and challenges presented by the region.
The following section analyzes aspects of interoperability in the government portals of some Latin American countries. The study surveyed below arises from the analyses carried out on the government websites of these countries, considering the most relevant aspects of data interoperability.
The complexity of public administration in Brazil has characteristics typical of a large territory
with a large population that stands out in Latin America. Currently, it is made up of more
than 5,560 municipalities throughout the entire territory that until 2003 had some autonomy to
purchase software and manage computer networks. It was in that year that the Secretary of
Logistics and Information Technology of the Ministry of Planning, Budget, and Management
decided to focus on interoperability. The decision was immediately supported by the state IT
entities represented by the Brazilian Association of State Entities [
In Brazil, there are various open data catalogs at the national, state, and municipal levels. The Brazilian Open Dice Portal is a government initiative that has been instrumental in promoting transparency and citizen participation in Brazil. This online resource has stood out as a centralized platform for the publication and access to data sets generated by public institutions in the country5. These catalogs typically provide detailed information about each dataset, including its description, format, update date, relevant metadata, and download links. structure. This makes it easier for users to identify and select the data sets of interest, as well as understand their content and structure.
In addition, open data catalogs in Brazil often implement advanced search functions and iflters, allowing users to make specific queries and quickly access the desired information. This
contributes to improving usability and eficiency in accessing data, promoting its reuse and value creation. The data sets available on the portal cover a wide range of areas, including health, education, environment, security, transportation, and economy, among others. This means that users can access a significant amount of information to explore various issues of social and economic importance.
In addition, the Brazilian Open Dice Portal aims to strengthen civic engagement. By providing
citizens with the information, they need to understand and evaluate government activities, the
platform fosters an active and informed citizenry. Citizens can use the data to monitor public
policies, participate in decision-making, and hold the government accountable. The Brazilian
government has implemented a series of policies and programs to promote the interoperability
of its information systems. One of the most important programs is the National Interoperability
System (SIN) [
Janssen, Charalabidis, & Zuiderwijk [
While Brazil has made notable advances in data management and interoperability, there is still work to be done. Collaboration between the government, civil society, and the academic sector will be essential to overcome the remaining challenges and ensure that data management in Brazil is truly inclusive and beneficial for all.
The Mexican government has established a series of policies and programs to improve the interoperability of government systems. The National Digital Strategy, for example, establishes a framework for the digital transformation of government, which includes improving the
instituto-nacional-de-tecnologia-da-informacao da
Informação.
https://www.gov.br/pt-br/orgaos/ interoperability of systems and promoting collaboration between diferent government agencies and entities7.
Within this framework of digital transformation, the National Digital Platform (PDN) stands out, an initiative of the Mexican government that seeks to centralize and standardize government information to improve eficiency and transparency 8. The PDN focuses on the use of web services and APIs for systems integration, the creation of platforms and tools for data and information management, and the adoption of cybersecurity practices.
Among the specific initiatives and projects that have been carried out to improve the interoperability of systems in the Mexican government, the following can be mentioned: a) The creation of the National Interoperability System (SNI), which aims to allow the integration of diferent government information systems and interoperability between them. This system is based on the use of common standards and protocols, such as the XML language and the use of web services9; b) The adoption of the Unique Digital Identity (IDU) as a common standard for the identification and authentication of users in government systems. This allows citizens to access diferent government services more easily and securely, without having to provide their personal information each time they register in a new system.
The National Digital Platform (PDN) is a Mexican government initiative to centralize and make government information accessible to foster transparency, improve the eficiency of public services, and promote data-based decision-making. The National Digital Platform uses state-ofthe-art technologies, such as the blockchain, to guarantee the security and authenticity of the data. In addition, it promotes system interoperability by using open standards and provides APIs for data access. In this way, the PDN is an essential tool for the digital transformation of the Mexican government.
The creation of the Open Data Portal of the Government of Mexico, allows access and reuse of public information by citizens, companies, and civil society organizations. This portal is based on open data standards and promotes the interoperability of systems through the publication of data in common and reusable formats10.
The Mexican government has promoted the creation of an open data catalog at the national
level to promote transparency and access to public information. This catalog, hosted by the
National Transparency Platform (PNT), centralizes defective data sets by various government
institutions 11. Through the catalog, users can explore and download data in open formats,
which encourages the reuse of information for various purposes, such as research, application
development, and knowledge generation. The open data catalog in Mexico not only includes
data sets from the Federal Government but also from state and municipal governments that
have adopted transparency and openness policies [
Other authors, Pérez and Rodríguez [
Furthermore, for digital transformation to be efective, citizens need to interact with the
government simply and securely. Martínez and Sánchez [46] emphasize the importance of the
Unique Digital Identity (UDI) in this process. The UDI allows citizens to uniformly identify and
authenticate themselves in various government services, simplifying access to these services
and enhancing information security. On the other hand, transparency is a key element in
any government modernization process. Rodríguez and Mendoza [
Digital transformation in the Mexican government is a complex process involving multiple facets, from centralizing information to promoting transparency and citizen participation. Although there are challenges along the way, the ongoing initiatives and projects demonstrate a clear commitment to continuous improvement and adaptation to the demands and needs of the digital era. It is essential to maintain this momentum and continue working together to ensure a modern, eficient public administration that serves the citizens.
Interoperability and the open data catalog play a key role in the context of public administration
in Colombia. These aspects are essential to promote transparency, eficiency, and citizen
participation in access to public information [
13Sistema Nacional de Interoperabilidad. Gobierno de México
The open data catalog in Colombia not only includes information from the public sector but also incorporates data generated by civil society, nongovernmental organizations, and academic institutions. This broadens the diversity and scope of the available data sets, fostering the collaboration and participation of diferent actors in the generation and dissemination of relevant information. The Datos.gov.co portal is an initiative of the National Government of Colombia, led by the Ministry of Information and Communication Technologies (MinTIC) and the National Digital Government Agency (ANGD). Through this portal, it seeks to promote the openness and transparency of government data, and allow its access and reuse by citizens. The Ministry of Information and Communication Technologies has led the “Online Government” project, which seeks to integrate the computer systems of government entities to improve the provision of public services. One of the most important programs is the Interoperability Reference Framework (IMR), which establishes the standards and protocols necessary to ensure that government information systems can communicate with each other efectively.
Some of the challenges in the interoperability of Colombian government information systems14 include the lack of standardization that makes it dificult to share data between diferent government entities since the data is in diferent formats; lack of coordination between agencies with diferent priorities and objectives that make collaboration and cooperation dificult; and resistance to change. However, there are opportunities to improve interoperability through the adoption of emerging technologies, training, and strengthening the culture of collaboration among government entities.
However, there are challenges associated with the open data catalog in Colombia that require attention. One of them is to guarantee the quality and constant updating of the data sets, as well as the veracity and integrity of the published information. Likewise, it is necessary to improve the usability and accessibility of catalogs, ensuring that users can perform intuitive searches and easily find the data they need.
Although eforts have been made to promote open data and interoperability in the Colombian public sector, there is still little participation by some public entities in the publication of open data and the implementation of interoperability standards. This limits the scope and availability of open data sets.
Uruguay has developed an interoperability policy for the public sector that seeks to establish the necessary guidelines so that information systems can share data and services eficiently and securely, and thus improve the quality of citizen care. This interoperability policy is framed in the Law on Access to Public Information (Law No. 18.381), which establishes that all information in the framework of public management is a public good that must be available to citizens and dealt with transparency. Established a series of initiatives and policies to promote data interoperability by creating the Agency for Electronic Government and the Information and Knowledge Society (AGESIC) and has led these eforts, developing the Interoperability and Standards Framework (MIyE) as a guide for the integration of data and systems.
14Revisión del Gobierno Digital en Colombia: Hacia un Sector Público Impulsado por el Ciudadano. https: //www.oecd.org/publications/revision-del-gobierno-digital-en-colombia-9789264292147-es.htm
The National System for Productive Transformation and Competitiveness (Transforma Uruguay) is in charge of leading the implementation of the interoperability policy in Uruguay. The program has developed a series of initiatives to promote interoperability between the diferent State information systems, such as a) A catalog of interoperability services that allows information systems to share services and data more eficiently and securely. This catalog includes services such as user authentication, information consultation, and data validation; b) A methodological guide for the design and implementation of interoperability services, which serves as a reference for developers of information systems in the public sector. This guide describes the steps required to design and implement interoperability services, from defining the requirements to commissioning and monitoring the services; c) A business architecture model that makes it possible to align information systems with the strategic objectives of the government and guarantee interoperability between them. This model defines the key components of the enterprise architecture and describes how these components interact with each other to support government business processes.
Regarding interoperability standards and protocols, the Uruguay Digital Government website uses open standards and technologies for the development of its online services, such as HTML, CSS, JavaScript, and JSON. In addition, it is in the process of implementing the SOA (Service Oriented Architecture) interoperability standard for the integration of diferent government systems and platforms. The open data catalog in Uruguay is a key tool to facilitate access and reuse of public information by citizens. The Uruguayan Government has developed the Datos.gub.uy portal, which functions as the main open data repository in the country. The open data catalog in Uruguay, through Datos.gub.uy, has fostered transparency and accountability, allowing citizens to access relevant information and use it to make informed decisions, monitor government performance, and develop innovative projects.
This has allowed a greater integration of the computer systems of diferent government entities. Despite advances in technological infrastructure and regulations, the interoperability of Colombian government information systems remains a challenge. Many government entities operate isolated systems and there is no efective integration between them. 2.5. Chile One of the main programs for the promotion of interoperability in Chile is the Electronic Government program, which aims to improve the quality of public services and government transparency through the use of information and communication technologies. In this sense, the Chilean government has developed an online services platform, which integrates more than 800 procedures and online services ofered by diferent public entities. This platform allows citizens and companies to conduct procedures and requests electronically, avoiding the need to hide from public ofices.
Regarding the interoperability of the Chilean government’s computer systems, work has been done on the definition of standards and protocols for the exchange of information between the diferent government systems and platforms. In this sense, the SOA (Service Oriented Architecture) interoperability standard has been adopted for the integration of diferent government systems and platforms.
In addition, the Chilean government’s open data portal has been created, which allows citizens and companies to access public information from diferent government agencies and areas. This portal is in line with open data and government transparency initiatives worldwide. Another important initiative in terms of interoperability in Chile is the National Territorial Information System (SNIT), which is an integrated geospatial information platform that allows access to information from diferent agencies and government areas related to territory and the environment. This platform uses open standards and technologies for the integration of diferent government systems and platforms. One of the most prominent catalogs in Chile is Datos.gob.cl, managed by the Digital Government Agency (AGD). The catalog houses a wide range of data sets from various government entities in Chile. This includes data produced by ministries, statistical institutes, municipalities, and other agencies. The catalog includes data analysis and visualization tools that allow users to interactively explore and understand information. These tools make it easy to create charts, maps, and visual aids that help identify patterns, trends, and relationships in data. This promotes a deeper understanding of the information and encourages its use in the generation of knowledge and decision-making.
One of the main objectives of government interoperability in Argentina is to simplify and expedite the procedures and services that citizens need to carry out before the State. In Argentina, the government has taken important measures to promote the interoperability of systems and improve the eficiency of the public administration as a whole. One of the main instruments to achieve this is the Law on Access to Public Information No. 27,275, sanctioned in 2016, which establishes a legal framework for the implementation of open data policies and the promotion of interoperability of systems in the public sector. The Distance Procedures Platform (TAD), which allows citizens to carry out procedures online with diferent state agencies, was created in 2012 and currently has more than 1000 procedures available.
In addition, the government has worked on the implementation of common standards and protocols to ensure interoperability between diferent government systems. The Biometric Authentication System for the National Registry of People (Renaper) has also been developed, which allows the identification and authentication of people using biometric technology. One of these standards is the Digital Identity System (SID), which allows citizens to have a unique digital identity to access diferent online services.
Another important initiative is the Electronic Document Management System (GDE), which allows the electronic management of documents and files in the public sector. The system is designed to be interoperable and allows integration with other government systems.
In Argentina, “the data catalogs represent a list of data assets of a state agency with their
corresponding metadata, which describes what they are and how they can be used. This helps
you find and better understand public data on a given topic. The concept of the data sets, better
known as datasets, refers to the fundamental part of the entire data catalog, that is, the group
of one or more data resources. Maintaining quality in datasets and their catalogs, makes correct
accessibility possible for citizens, for example, what these concepts go together” [
In summary, the interoperability analysis of each country fulfills a key aspect in the digital transformation, and as explained above, diferent initiatives and projects have been conducted to improve it. Although there are still challenges and opportunities for improvement in this area, the diferent governments are committed to the continuous improvement of the interoperability of systems to guarantee eficiency and efectiveness in the provision of public services. One of the challenges associated with the open data catalog is ensuring the quality and constant updating of the information. It is necessary to establish clear processes for maintaining and updating data sets, as well as feedback mechanisms and citizen participation to improve the quality and relevance of the data provided.
For this work, a proposal of the relevance of open data catalogs that were extracted from the government websites of the analyzed countries is analyzed.
The objective of this study is to validate the content of these files through the software developed by the authors, which presents a series of validation criteria that were considered for the Argentine Republic. The choice of said country was since a series of criteria explained on its government website for their analysis were identified, which allowed the design of a prototype oriented to these validation aspects. It should be noted that the criteria proposed for this country are considered good international practices, developed on its oficial website 151617.
This research work is an expansion of studies carried out [
There are numerous open data catalogs in various Latin American countries. Various government websites were analyzed, some of these are: a) Brazil1920; b) Mexico2122; c) Colombia2324; d) 15Datos Argentina. https://datos.gob.ar/ 16Ministerio de Salud. Datasets. http://datos.salud.gob.ar/dataset 17Perfil de Aplicación Nacional de Metadatos para Datos Abiertos. paquete-apertura-datos/perfil-metadatos/ 18The world’s leading open source data management system. https://ckan.org/ 19Portal da Transparência da Mobilidade Urbana de Natal. http://dados.natal.br/ 20Portal de Dados Abertos da Cidade do Recife. http://dados.recife.pe.gov.br/ 21La plataforma cívica de datos abiertos de México. https://datamx.io/ 22Gobierno de la Ciudad de México. https://datos.cdmx.gob.mx/ 23Alcaldía de Santiago de Cali. http://datos.cali.gov.co/dataset 24Datos Abiertos Bogotá. https://datosabiertos.bogota.gov.co/ https://datosgobar.github.io/ Uruguay2526; e) Chile2728; Argentina2930, and it was found that these countries have decentralized repositories by commune, municipality, locality, city, and state as appropriate, in addition to a central repository of federal aspect.
Table 1 shows the number of datasets identified by government websites and the number of data types detected. It is observed that the types of open data formats are various, open data formats allow interoperability between software, but there are public open data that are not in a suitable standard format, for example, the open PDF format.
In addition, in Table 1, it can be seen that the type of format most used in the analysis sample is the CSV type, which remains one of the most standard formats in the selection of data available in Latin American government portals.
It is important to mention that, regarding the periodicity of the published datasets, it was identified that they are updated daily, weekly, monthly, quarterly, and annually. In addition, there is a measurable aspect that is "AccrualPeriodicity", which corresponds to the updating of the data, every 10 years, every 4 years, every 2 years, every 1 year, every 4 months, every 2 months, twice a month, three times a month, once a week, among others. For example, in Argentina, approximately 50% of the datasets are updated monthly.
In the next subsection, the validations carried out with the software prototype made are briefly explained.
25Catálogo de Datos Abiertos. https://catalogodatos.gub.uy/ 26Bienvenidos al Portal de Datos Abiertos de Montevideo. https://ckan.montevideo.gub.uy/ 27Datos.gob el repositorio de datos abiertos centralizado del Estado. https://datos.gob.cl// 28Consejo para la transparencia. https://www.consejotransparencia.cl/datosabiertos/ 29Datos Argentina. https://datos.gob.ar/ 30Ministerio de Salud. Datasets. http://datos.salud.gob.ar/dataset
A prototype was developed for the validation of part of the sample that, through a processing script in Python, processes the open data catalogs in JSON, that is, it analyzes the metadata in JSON format, usually called “data.json”. This prototype generates two reports as a result in which the analysis of the structure of open data catalogs is shown. This is done using the methods provided by the “pydatajson” library43. It is validated that: • The URL present in the open data catalog field is available (through the UrlValidator field). • The script validates that the files associated with the distributions are consistent formats (through the ConsistentDistributionFieldsValidator field). • It is validated that there are no duplicate elements (through the Theme Ids Not Repeated
Validator field).
In the new version that is in full development, graphics were integrated regarding the dataset update frequency and its update to date according to its data type. Data from the DCAT-type scheme was incorporated. Finally, improvements in the detection of processing errors, and performance improvements, for example: pydatajson vendorization for changes and subsequent PR, visualization of booleans instead of integers, messages cut of, bugs, and index array, are in the testing phase. out bounds, null pointers, etc.
In summary, the validation of the content of open data catalog files is carried out from two approaches: a) From the use of the Python library, called “pydatajson”: A validation is carried out on certain fields in the catalogs that cannot be covered by the JSON Schema library. For example, the landing pages of each dataset.
The open data opening package indicates that it is mandatory that this link be present, and it must be functional (accessible and with a satisfactory response code), this is what is validated. Another custom validation is done for the access and download URLs of the distributions of the data sets.
Finally, it is verified that there are no taxonomically repeated catalogs in the files, this is done through the topic ID. It is validated that the distributions have a specific format. b) Through a proposed own development: Initial prototype uses a processing script in Python language. This one has like functionality, receiving a list of CKAN – or Andino – portal nodes, or raw metadata in JSON or XLS format, usually called “data.json” or “data.xlxs” correspondingly.
The developed script will generate two reports with a .CSV file extension, as a result of the validation generated on the analyzed file of the open data catalog.
Based on the sample indicated in the previous section, the self-developed prototype was used, to carry out a general study of the structures of the public data files that are available in the government portals of Latin America.
43GitHub. https://github.com/datosgobar/pydatajson
It is worth mentioning that the prototype was developed with technical aspects that are used in Argentine government portals, that is, a guide to good practices for metadata44, oriented to CKAN, so that the analysis of results could focus on aspects of noncompliance for not taking the same criteria of said country. Brazil; Chile; ’superTheme’ is a re- For these countries, no LATAM It should have a taxonomy of those Colombia; quired property. dataset has the “superTheme” prop- specified in45 Mexico; erty.
Uruguay Brazil; Chile; ’accrualPeriodicity’ Colombia; is a required property Mexico; or value is invalid’ Uruguay Brazil; Chile; ‘*’ is too short Colombia; Mexico; Uruguay Brazil; Chile; ‘*’ is too long Colombia; México; Uruguay Uruguay
’*’ is not a uri Brazil ’*’ is not a uri
Some LATAM datasets do not have the “accrualPeriodicity” property or if they do, it does not have a valid value. 730 of the 19921 LATAM datasets have a very short field title.
This field is the periodicity with which the data set should be updated.
Minimum 4 characters. 71 of the 19921 LATAM datasets have a very long field title.
Maximum 60 characters. 79 of the 2408 Uruguay datasets It must be a URI. This is validated do not have valid values in some by regex. of their fields, such as “downloadURL”. 56 of the 169 datasets from Brazil It must be a URI. This is validated do not have valid values in some of by regex.
their fields, such as “url”.
Colombia Colombia Colombia ‘accessURL’ is a re- 1058 of the 3365 datasets for Colom- The property should contain the acquired property bia do not have the “accessURL” cess URL of the dataset distribu
property. tion. ’issued’ is a required property ‘downloadURL’ is a required property
In the sample dataset from Colom- The property should contain the bia, none does not have the “issued” publication date of the dataset. property. 2308 dataset from Colombia does not have the “downloadURL” property.
URL that allows direct download of the dataset distribution, links directly to a downloadable file in a given format.
Table 2 is shown below, with the classification of aspects found grouped by type, some of the errors are expected because the repositories of these countries do not comply with the open data package of Argentina46. It should be noted that the types of errors indicated in Table 2 are the validations used by the software prototype proposed by the recommendations suggested by 44Perfil de Aplicación Nacional de Metadatos para Datos Abiertos. paquete-apertura-datos/perfil-metadatos/
46Perfil de Aplicación Nacional de Metadatos para Datos Abiertos. paquete-apertura-datos/perfil-metadatos/ https://datosgobar.github.io/ https://datosgobar.github.io/ the good practice guides of Argentina47. This country was taken as the standardized criterion since it was one of the countries with the most detailed validations.
It is important to mention that compared to the results obtained in Table 2, these are aspects that were identified through the good practice guide of the Argentine Government 48. This means that, although it is detected as an aspect to be considered by the prototype made, it might not be a criterion that does not take you into account in the countries mentioned and identified as a sample. Also shows a column of recommendations for the aspects detected in the structure of public open data catalogues.
Based on the work carried out on the study of the interoperability treatment of the diferent Latin American countries, it can be observed that the concept of open government is increasingly booming, which allows transparency in government management for all citizens. Being able to share data in an open format together with your open data catalog will help diferent companies consume this data and work on it to analyze possible improvement scenarios in diferent citizens and their relevant contexts.
Collaborating with an adequate structure in open data catalogs will help improve the quality of interoperability management between the diferent software that share data so that they can be processed and thus obtain information with added value in citizen contributions. This work not only proposes a survey of the Latin American countries in aspects of the current state of interoperability but also proposes a validation prototype of the structure of these metadata of the data sets, that is, of the data catalogs. public open. These validations in these data structures are necessary to provide a higher quality in the data content that is available to all people.
In future lines, it is expected to increase the survey of countries in the context of open data interoperability, and on the other hand, concerning the programmed prototype, work will continue, adding new schemes that allow the management of the metadata of the diferent sets of data that are available on the government sites of the diferent governments at an international level. As another future proposal, possible aspects that could be standardized and that are common criteria between the diferent open data catalogs will be analyzed, this will allow a better reading, analysis, and interoperability between the diferent software, which will lead to openness of data and transparency in the management of public governments. https://datosgobar.github.io/ https://datosgobar.github.io/