Many countries use international rankings in the formulation of electronic government and electronic governance (E-GOV) strategies. This practice seems to require some systematization, considering the limitations of these indexes as recognized by the Institutions that produce them and by the literature. Based on the scanning of national strategies and interviews with public officials, this exploratory study resulted in a proposal of an E-GOV strategy formulation framework taking into account international rankings. Despite the existence of certain constraints and dependence of stakeholders, the investigation revealed that rankings are relevant in this scenario and impact strategy formulation. The study concludes that this framework would be useful and should be flexible, instructive, easy to use, comprehensive, coparticipative, and effective. A research agenda for its development finalizes the article, arguing that it can be an opportunity to improve E-GOV strategy formulation processes and the knowledge in the field.
Acknowledgement: This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020.
The acronym E-GOV is used in this paper to refer to e-Governance, the public sector's use of IT to
improve information and service delivery, encouraging citizen participation in the decision-making
process, and making governments more accountable, transparent, and effective
E-GOV actions and initiatives seem to be developed under a strategy, which reminds the
traditional strategic alignment between IT and organizations' business stated years ago
These are just two examples of existing rankings that are currently used in E-GOV strategy
formulation. However, why are they used, to whom are they important, what role can they perform,
and particularly, how can they be used in strategy formulation are remaining open questions which
this research aims to address. Answers to these questions will naturally provide a set of relevant
insights that can inform the establishment of a prescriptive framework, or a method
The rest of the paper is structured as follows: section 2 contextualizes the E-GOV strategies scenario involving international rankings; section 3 describes the research design, while section 4 presents the results and a discussion about the characteristics of a future framework for the formulation of E-GOV strategies taking into account international rankings; and section 5 concludes the paper.
The use of Information Technology (IT) to enhance and transform government and governance is
not a simple task. The point is "how exactly the core government functions like providing public
services and infrastructure, formulating and implementing public policies, maintaining social order
and security, operating social programs, and promoting economic growth should be performed in
both physical and digital worlds"
Although there is no universally accepted definition of strategy
This definition refers to the strategy content and also, to the strategy formulation process,
something that the literature in management has already explored. Although the association of
organizational strategy classifications to public agencies has limited relevance
E-GOV strategies are a top-level document that addresses strategic directions, goals, components,
principles, and implementation guidelines
A strategy can support the management of investments and adequate evaluation through
performance indicators. The assessment of E-GOV has proven to be an important but complex aspect
due to the various perspectives involved, the difficulty of quantifying mostly qualitative objectives,
and the contexts of use
International institutions regularly undertake significant studies to produce rankings of countries
on a wide range of features, including information technology
However, existing E-GOV strategy formulation frameworks do not take into account
international rankings
Table 1 shows that international rankings are used across a wide range of countries that differ in their geographic location and size, economic status, population size, and development stage. Countries have been selected among the 193 United Nations Member States, and only documents formally published were considered. Procedures included web-based search engines and, when necessary, further contact with country authorities. Furthermore, Table 1 shows that international rankings play an important role in different phases of the process of E-GOV strategy formulation, including diagnosis, contextualization, and the definition of goals.
After establishing evidence of the use of international rankings in the formulation of E-GOV
strategies, it became pertinent to gain a deeper understanding of the way and why rankings are
used. This was viewed as important to ascertain whether a research line on the subject is worth
pursuit
This exploratory study was carried out in Brazil between July and August 2019 and involved semi-structured recorded interviews with nine Brazilian public officials. Six of them were high-level government executives such as National Deputy Secretaries, Directors, and a CIO. The remaining three were Senior Advisors working directly with high-level executives, not necessarily the ones that were part of the first group. The selection of respondents looked to include roles typically involved in national E-GOV strategy formulation. Eight of them worked in at least one of the three Brazilian E-GOV strategies formulated since 2015. Additional prudency has been taken to cover public officials still involved in digital public policies nowadays.
Questions have been selected to perceive the importance of a future framework for the
formulation of E-GOV strategies, taking into account international rankings. Although the
importance of these rankings as a tool for policy definition, program prioritization, and strategy
formulation has already been stated
These questions prove to be convenient to establish some features of the framework, grounded
on content analysis. So, a re-analysis procedure of all the responses was conducted sought to "define
the objectives for a solution", consistent with the Design Science Research methodology
This section presents the data analysis and main results of the interviews, along with a discussion about them. For anonymization reasons, respondents are identified into brackets using the acronym "rsp", followed by "_x", where "x" varies from 1 to 9. Respondents 1 to 6 are high-level government executives; respondents 7 to 9 are technical advisors. Interview excerpts are delimited by quotation marks and followed by the respondent's identification. 4.1
This statement was formulated based on the answers received to the question "Are international rankings relevant (and why)?" All the respondents said that rankings are relevant, some with great emphasis. According to them, they are relevant because they offer: "historical data series to make comparisons with other countries" [rsp_1, rsp_2, rsp_4, rsp_8], "showing the country evolution over time" [rsp_2]. "They have a standard evaluation process, show country experiences, create a gamification between countries and offer an independent evaluation method" [rsp_4]. They also "generate an international, productive and collaborative dialogue about efficient public policies" [rsp_4], "with technically-sense recommendations, not mandatory rules, coming from trustful International Institutions" [rsp_8].
They are relevant due to their utility: "they are useful to mid and long-term planning" [rsp_1], "to formulate public policies, independent of government level" [rsp_9], "to identify gaps and strengths in policies and strategies formulation" [rsp_7]. They are "now driving the whole work of the Presidency's public policies analysts, especially in competitiveness agenda" [rsp_8]. They are also useful to "guarantee a slot in the political agenda, a necessary condition to engage the Public Administration into any effort" [rsp_1, rsp_2].
Nonetheless, respondents reminded that some limitation exists. "Despite the importance of an international institution's brand, some rankings are relevant only at the technical level, case of UN/EGDI. Others, like WB/DB, reached a higher status, driving the public policy and changing the governance process, including the actors involved" [rsp_3]. "Rankings are a simplification result of many indicators, and indicators presume resulted-oriented work, something that not all government institutions are. For this reason, rankings are relevant, but not always, especially when exists some criticism about the ranking owner" [rsp_6]. 4.2
This statement resulted from the interpretation of the answers provided to the question “For whom?”. According to the interviewees, rankings are relevant for public officials, public agents, and policymakers [rsp_1, rsp_3, rsp_4, rsp_7]; for politicians, political actors, and Ministers [rsp_1, rsp_2, rsp_3, rsp_4]; for international investors [rsp_3]; and for top-ranked countries [rsp_2]. But, not for citizens [rsp_1, rsp_6, rsp_8, rsp_9]. “Rankings have different importance to different actors” [rsp_4].
"Policymakers look for good practices worldwide, while political actors use to play with the ranking rules" [rsp_4] looking for political gains. "The relevance is higher for politicians than technicians" [rsp_8]. "If a ranking endorses a public policy that a government high executive believes, a low-cost initiative that climbs positionings, relevance grows" [rsp_5]. Their "theoretical fundamentals are relevant to institutional leaders, inducing good practices, leading to institution improvement" [rsp_6]. "Depending on the importance of publisher to the country, ranking rules superpose countries' decisions, because their meaning of success, a premium work, an international recognition" [rsp_4]. "Top-ranked countries use them as a soft power mechanism, to influence other ones" [rsp_2]. However, in the opinion of some respondents, rankings are not relevant for citizens. Citizens are mainly and naturally concerned with self-demands, short-range plans, day-to-day issues, and transactional services [rsp_1, rsp_6, rsp_8, rsp_9]. 4.3
This statement resulted from the analysis of answers provided to the question "How international rankings impact E-GOV strategy formulation?". All respondents endorsed that rankings impact government strategies, including E-GOV. "They did impact former strategies, do it to currents, and will impact next ones" [rsp_4]. Authorities "usually keep up with strategic deliverables and associated measures, what lead to international rankings" [rsp_2]. "They are useful to expose a situation and support a point-of-view with decision-makers" [rsp_6]. “Despite the inexistence of a formal orientation to government agencies, there are high-level government units, the Presidency, uses them to legitimate their work" [rsp_8].
They also motivate a result-oriented approach, "especially when allowing regional comparisons or similar-context-countries comparisons" [rsp_2]. "If we consider that a public policy is a set of government programs and projects, each one of them with its success indicators, it is natural some impact of rankings and respective indicators" [rsp_8]. Especially in E-GOV, "international rankings give a general direction of which policy aspects are considered relevant, which ones might have more priority" [rsp_7].
However, despite being "a good tool to support planning, it cannot be the unique" [rsp_2]. Rankings should be the "result of a public policy, a great result if it allows better positions" [rsp_2]. Their use is constrained "by local reality and the local context" [rsp_1], they are "not a decisionmaker, don't decide which public policy will be done" [rsp_2]. "They are a factor, an important reference, a parameter, a message to the Estate, but with superficial influence" [rsp_3]. 4.4
This statement resulted from the replies gathered for the question "Would a prescriptive framework be useful for the formulation of E-GOV strategies taking into account international rankings (and why)?". In short, the framework would be useful [rsp_1, rsp_2, rsp_3, rsp_4, rsp_8, rsp_9]. "As an information source, it has the potential to support the decision-process" [rsp_2], "with limitations, but more beneficial than harmful" [rsp_4]. "If it offers a direct correlation between functionality and components (process, efficiency) and results and outputs (efficacy, effectiveness), it will be valuable, something that assures impact on the user perspective. It does not make sense a lot of digital transformed public services with no citizens using it, a good position in EGDI ranking, but no eservices users. It is not about the process; it is all about client satisfaction" [rsp_3].
Ranking specific criteria has been highlighted: "certain measurements turn possible to identify core E-GOV structures that must exist, like Digital ID, E-procurement, or a Unique Authentication Service" [rsp_1]. Finally, the necessity to include other rankings into the framework scope has been mentioned, like "trust measurements, for example" or others "that capture citizens perceptions, something that UN/EGDI does not do, would be very useful" [rsp_9]. 4.5
Data analysis of the interviews confirmed the relevance of the subject between the set of practitioners. Results confirmed that a framework for the formulation of E-GOV strategies, taking into account international rankings, would be useful. Some constraints and limitations became also evident after the analysis. These elements seemed to be very relevant as inputs for the design process of the new framework. Aimed at finding newer and deeper inputs, a re-analysis of all the responses was conducted to define the objectives for a solution. It involved the screening of all responses, independently of the questions, and the results achieved are presented in Table 2.
Interview's excerpt that supports the interpretation Flexible: "Strategies should not be exclusively guided by rankings, but also by local adjustable to the reality, local context."[rsp_1]; "Different countries have different contexts, but country context. E-GOV objectives are very common and similar. The final objective is mostly the same, for example, the unique digital identity. 'What to do' does not vary, 'how to do' varies according to a given country. So, the implementation should be guided by the context."[rsp_1]; "It is mandatory a balance between ranking measures and country priorities."[rsp_2]; "It will depend on the country's maturity. Lower maturity, higher-ranking importance. Higher maturity, lower importance to them. Rankings are more important to countries which use them as a persuasive tool."[rsp_2]; "It is not easy to transfer a public policy from a country to another because it depends on the country's trajectory."[rsp_4]; "Prudence should exist to avoid a choice of a 'best practice' that does not fit to a country. A translation is needed, based on context analysis." [rsp_4].
Instructive: "All our strategic initiatives use core E-GOV structures: interoperability, single supports the sign-on, digital ID; all these come from International rankings."[rsp_1]; "This learning process framework, as a source of information, […] has the potential to support and the decisions."[rsp_2]; "UN/EGDI not yet trespassed the technical barrier to a association of political level, what WB/DB did. Maybe because technicians do not know how rankings the evaluation process occurs and how indicators are measured."[rsp_3]; characteristics "Rankings are biased, have preconceptions and distortions because they to E-GOV aggregate indicators and some lack of updates. Why? To maintain the historical purposes. series and do not lose the comparative feature. This is an eternal tradeoff."[rsp_4]; "Rankings facilitates to find references of what is a good policy implementation, to learn from it. It also shows which policies need to be improved."[rsp_7].
Easy-to-use: "Rankings can be used as a checklist of strategic items, a minimum set of what simplifies the has to be done, a development guide."[rsp_1]; "Something that supports use of rankings interpretation, helping to make the correlation between what society international needs and what is evaluated, a translator for public officials and the rankings in E- society."[rsp_3]; "As much we lower the transaction costs, allowing transfer GOV strategy formulation efforts to the execution, it will be helpful."[rsp_4]; "UN/EGDI was formulation. not studied in detail, due to a traditional lack of time to build a strategy"[rsp_9]. Comprehensive: "UN/EGDI ranking is good, but not enough to check impact. Other indexes, broadly covers such as Edelman Trust Barometer, UN/Human Development Index, the E-GOV WB/Doing Business, are rankings with potential to evaluate digital purposes government impact."[rsp_1]; "If the framework uses more than one ranking, preferably those with a focus on results, it could work." [rsp_3]; "… not only EGOV rankings but others based on citizens perceptions, something that UN/EGDI is not, rankings that measure trust, for example, would be very useful." [rsp_9] Co-creative: "Citizens are short-term oriented, with day-to-day concerns […], politicians are enables the mid-term, need to legitimate what they do and why, every 3 or 4 years […] participation of public officials need to construct a long-term agenda, delivering outputs in multiple short and mid-term to balance the citizens and politicians' expectations"[rsp_1]. stakeholders. "Rankings show the common criteria used to evaluate other countries by global institutions, what impacts local authority's political agenda-setting up" [rsp_1]; "They are proxies of impact, not impact itself. Citizens' objective is to obtain a retirement pension, not an online service to order a pension. Note that if it happens seamlessly, better. Sometimes, rankings risky guide countries to be efficiency, not effectiveness." [rsp_4]; "In a conflict scenario between technicians and politicians, it would be useful if the framework brings technical and rational arguments, based on evidence, evidencing the impact of these indexes."[rsp_4]; "Considering that international rankings influence a national E-GOV strategy formulation, its design and governance should involve multiple stakeholders from multiple sectors, to avoid agenda capture by interest groups."[rsp_7].
Effective: "Rankings did not show 'how to do', they are a checklist of what a country delivers an E- has/has not. If the framework focuses on the E-GOV strategies formulation GOV strategy process, it will help countries." [rsp_3]; "A framework oriented by international after a complete rankings would allow a better performance in our work (strategy formulation formulation)." [rsp_8]; "The ideal process to build an E-GOV strategy involves process. a nominated coordinator, multi-disciplinary professionals, and multiple government agencies teamwork. A vision definition, a context analysis, a public consultation, an action plan, a benchmarking with similar countries, and a monitoring structure."[rsp_9].
According to the analysis, the future framework should be flexible, instructive, easy-to-use, comprehensive, co-creative, and effective. This list of framework characteristics seems to be compatible with the literature explored previously in this article. The framework should be flexible because E-GOV is a particular case of ICT application in government, a complex and diverse institution that varies according to countries' organization, political structure, population, size, economy, and so on. It should be an easy-to-use artifact, once the strategy formulation is a complex process. It must be comprehensive, since E-GOV features a variety of purposes to link, simultaneously, to different international indexes. The co-creative characteristic can be associated with e-governance processes, which involve many stakeholders. Finally, effectiveness is a natural goal of e-governance processes and strategies.
Many countries use international rankings as an important source of information in the process of E-GOV strategy formulation. Examples of these rankings are the United Nations E-Government Survey and the World Bank Ease of Doing Business Report, but this study unveils a wider reality, showing a list of nine national E-GOV strategies with references to the European Commission Digital Economy and Society Index, the World Economic Forum Global Competitiveness Report, and others. These countries cover different contexts regarding geographic location, economic status, population size, and development stage.
Evidence showed that international rankings are performing roles such as diagnosis, contextualization, and definition of goals. Findings also demonstrated that this practice is valued by politicians, public officials, international investors, and countries, but not by citizens. The exploratory study confirmed that the construction of a framework for the formulation of E-GOV strategies, taking into account international rankings, would be useful, and features of the future framework have been identified: it should be flexible, instructive, easy to use, comprehensive, coparticipative, and effective. The research agenda encompasses the framework development, to support public officials to better use these indexes and support the E-GOV strategies formulation. However, other opportunities exist in exploring the relationship between international rankings and other E-GOV themes. Limitations of the study can be considered the number of countries in the national strategy's inventory and the fact the interviewees were from a single country.
Wagner Silva de Araujo Wagner Silva de Araujo is a Research Assistant at the United Nations University Operating Unit on PolicyDriven Electronic Governance (UNU-EGOV) and a PhD Candidate at the Department of Information Systems, School of Engineering, University of Minho. He is also a researcher at Centro ALGORITMI / University of Minho. He has more than 25 years of experience in the Brazilian Government in positions like the CIO of the National Prosecuting Agency, National Director of Digital Government, and Chief Advisor of the National Secretary of Digital Government. His academic interests focus on strategic planning and design of digital government and digital governance initiatives.
Delfina Soares Delfina Soares is the Head of the United Nations University Operating Unit on Policy-Driven Electronic Governance (UNU-EGOV). She has been associated with the Operating Unit since 2015, when she joined as Adjunct Associate Professor. She has strong ties to UNU-EGOV’s host university, the University of Minho, where she has held various positions over the past 18 years, including Lecturer and Assistant Professor at the Department of Information Systems and as a Researcher with Centro ALGORITMI. Her areas of research and expertise include electronic governance at a national, local, and sectorial level, electronic government interoperability and cross-agency collaboration, electronic democracy and electronic participation, and electronic governance measurement and monitoring.
João Alvaro Carvalho