eGovernment's primary strategic priority is to offer electronic services to citizens. However, the complex structure of public organizations often makes it difficult for citizens to find the services they need. In response, governments have adopted web portals as a single point of contact. This research paper evaluates the usability of the Greek government's official web portal called "gov.gr". The study reveals that users encounter usability problems due to the ineffective performance of the search engine, the lack of a minimalist design, and difficulties navigating within the system. To improve the portal, the study recommends upgrading the search engine, focusing on a minimalist redesign, and implementing specific features such as a visible 'home' button.
One of the main strategic priorities of eGovernment is to provide electronic services to citizens. However, the complex structure of public organizations has made this initiative challenging, as citizens often struggle to locate the services they require. To address this, governments have embraced web portals as a single point of contact for online service delivery linking all government services and information to a central location.
In 2020 in Greece "gov.gr" has been launched as the official government web portal1. It offers citizens access to 1,709 digital public services offered by ministries, organizations, authorities, and the country's regions. Since its launch, the portal has gained significant popularity, and aims to be the primary point of contact for citizens and businesses in the public sector.
Ensuring user-friendliness in gov.gr design is crucial for its success. However, usability issues can hinder the user experience. To address this, researchers from the University of Macedonia, employees of the Greek Ministry of Digital Governance (MDG), and an expert of “Expertise France” collaborated closely in a joint project to evaluate gov.gr usability. Thus, the present study aims to disseminate the results of this ongoing project driven by the following research questions:
RQ1: Can users easily navigate gov.gr without any obstacles, doubts, or questions? RQ2: What obstacles or difficulties might users encounter while navigating gov.gr? RQ3: What are the proposed solutions to improve gov.gr usability?
The rest of the paper is structured as follows. In Section 2, we outline the overall project that funded this research. In Section 3, we provide the background and related work of the study. Section 4 outlines the methodology used to evaluate the usability of gov.gr, while Section 5 presents the evaluation results. Lastly, Section 6 discusses the research findings, presents conclusions, identifies study limitations, and provides recommendations for future work.
This research is part of a the “Next Generation of Digital Public Services” project which provides technical support in the areas of digital transformation and cybersecurity to the MDG. The project is funded by DG REFORM under the Technical Support Instrument (TSI 2022)2 and is implemented with the support of Expertise France. The main objective of the project is to implement a common Framework for the next generation of improved digital public services to the citizens via the “gov.gr” ecosystem.
The MDG, in an effort to ensure a homogeneous user experience across all domains and service categories offered by the new digital portal of the public administration (gov.gr), is aiming at creating the next generation, gov.gr ecosystem, which will operate as the single point of contact between citizens, businesses and the public sector. This new gov.gr platform will provide digital services with a common “aesthetic” identity, developed under a common framework (business and technological user-centric design) and offering automation by being linked to the underlying business processes, currently organized within the National Registry of Administrative Public Services “Mitos”3. Moreover, it will support cross border electronic services for all EU citizens.
The aim is for “gov.gr” to be redesigned to function as the digital service center, which will gather all solutions and information that citizens and businesses should be aware of in their interaction with the administration. In other words, it is intended to operate as the one and only point of contact between citizens, businesses and the public sector. For this purpose, a usability evaluation of the current form of "gov.gr" was deemed essential and that work is presented in this paper.
Usability refers to the interaction between humans and computers, including dimensions
such as learnability, efficiency, and user satisfaction [
The project's objectives and the associated research questions guided the selection of usability evaluation methods. We used heuristic evaluation and formative usability testing with the think-aloud protocol to gain insights from usability experts and real users, respectively. We also collected user feedback through satisfaction questionnaires.
Between October and December 2023, 332 students enrolled in the Department of Applied
Informatics at the University of Macedonia conducted a heuristic evaluation of gov.gr. To
ensure the accuracy and reliability of the results, the students enrolled in the "Information
Systems" course acquired the necessary skills. The students were divided into groups and
informed about the gov.gr website. 40 personas and 44 illustrative scenarios were created
to guide them through the 11 service categories (Table 1). Each group was given a persona
and scenario and asked to explore the website. Students individually evaluated gov.gr based
on Nielsen heuristics [
Between November and December 2023, 34 students from the Department of Applied Informatics at the University of Macedonia conducted a formative usability test to evaluate gov.gr. 67 novice users participated in the test, divided into six user groups. The students assigned tasks to the users, allowed them to freely browse the system, and gathered their thoughts and opinions while working with it. After completing the user testing, the group discussed their findings and prepared the final evaluation report.
Between October and December 2023, 57 out of 67 novice users and 191 out of 332
students participated in a study. They completed the SUS [
After reviewing the final assessment reports submitted by the students who conducted the
heuristic evaluation on gov.gr, we classified the findings into four main categories:
Navigation (Table 2), Content (Table 3), Terminology (Table 4), and Functionality (Table 5).
We also noted the number of student groups identifying each finding and its average
significance. Additionally, we used the Nielsen significance classification table to
characterize the importance of the findings, where 0= false alarm, 1= cosmetic problem, 2=
minor problem, 3= medium problem, 4= major problem, 5= usability catastrophe [
Usability issue 1.The website's home page and other pages contain too much information, leading to confusion for users. 2.The website lacks explanatory images that could assist the user in comprehending its content. 3.Some system parts and functionalities are not easily visible to users. 4.The font size, images, and buttons frequently fail to meet the user's requirements. 5.In the event of an error, the user is not provided with clear instructions on what to do. 1.It is challenging to find the desired service by navigating through the 11 service categories. 2.The wording of the services is problematic. 4 3 5 10 1 2 9 2 2 13/69 5/69 4/69 22/69 #Groups 28/69 5/69
Avg. significance 2,2 2
Usability issue 1.If the user misspells or types a service differently than its registered name on gov.gr, the search engine fails to return results. 2.In case English is selected, the service is executed in Greek. 3.Other than English, translation to foreign languages is not supported. 4.Users face difficulty in finding the desired service when browsing in English. 5.The mobile search feature does not suggest results to users. 2 2 7 7 #Users 60/67 40/67 27/67 26/69 21/69
1."I had trouble using the search bar to find the service I was looking for" 2."I found it challenging to navigate back to the homepage of gov.gr" 3."I found it inconvenient to constantly log into the system for running service processes." 4."I found it cumbersome to repeatedly enter information already known to the Public Administration." 5."I found the home page of gov.gr to be overwhelming with too much text, making it confusing and difficult to navigate."
Following the research findings, it has been identified that the search bar on the website and the way the services are categorized present usability issues for users. Users face difficulties in finding the services because of challenges in identifying the correct search terms, the lack of spelling error prevention, and the confusing service categories. Furthermore, the wording of services and the term "life events" is often conceptually ambiguous, which adds to the confusion for users. Users have found it challenging to use gov.gr due to the absence of minimalist design, overloaded pages, lack of images and infographics, poor use of colors and font size, and difficulty returning to the homepage. Additionally, constant reconnection to the system by entering personal details already known to the Public Administration has caused annoyance and a loss of willingness to consume the services.
According to the results of the SUS and SUPR-Q questionnaires, both novice users and undergraduate students had difficulty achieving their goals within the gov.gr application. Novice users rated gov.gr at 64.9%, while students rated it at 67.75% on the SUS scale, falling into the D-POOR category. The SUPR-Q questionnaire also revealed that novice users rated gov.gr at 3.5/5 (70% on the 100-point SUS scale), while students rated it at 3.57/5 (71.4% on the 100-point SUS scale), which is the B-GOOD category according to the proposed SUS scale. However, postgraduate students rated the gov.gr-native digital services as excellent, suggesting that usability issues may be related to locating digital services rather than their internal processes.
Finally, it has been observed that the statements in the SUPR-Q questionnaire related to
usability can predict SUS questionnaire score. Even without statistical testing, the evidence
strongly supports this finding, aligned with relevant literature [
This study evaluated the usability of "gov.gr," the web portal of the Greek government. The evaluation revealed that users experienced usability issues mainly due to problems with the search engine, the absence of a minimalist design, and difficulties in navigating the system.
To enhance gov.gr usability, we recommend upgrading the application's search engine. It is also essential to focus on a minimalist redesign of gov.gr, avoiding unnecessary information on the home and other application pages. To achieve this, gov.gr UI/UX designers should reconsider categorizing services at the conceptual level (for example, by adopting a more explicit mental schema) and the design level (by introducing images). It would also be helpful to implement specific functionalities such as a visible 'home' button to make it easier for users to navigate to the home page and within the application. Moreover, we propose rather than using the ten SUS questions, to use a pop-up menu that only includes the second statement of the standard SUPR-Q questionnaire ("It is easy to navigate within the website"), allowing users to assess their satisfaction simply and efficiently.
During our research in December 2023, the Greek government introduced a digital assistant (chatbot) on the portal. As a result, we have not investigated its impact on the usability of gov.gr, which is a limitation of our study. Further research is required to determine whether both the application's search bar and the digital assistant are necessary. If it is determined that an effective search engine is essential along with the digital assistant, appropriate measures will need to be taken to improve the search engine. In addition, as future work, it would be helpful to assess the digital assistant's usability and the extent to which it enhances the web portal's usability. This analysis will provide valuable insights that will enable us to understand the implications of introducing the digital assistant. Consequently, the results will help to improve the portal's usability and enhance users' overall experience. This research has been carried out for the Greek Ministry of Digital Governance under a project funded by the European Commission (DG REFORM) and implemented with the support of Expertise France.