<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Accessibility Evaluation Of Five Moroccan E-government Portals</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Mohammed Rida Ouaziz</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Laila Cheikhi</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ali Idri</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alain Abran</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Software Engineering Information Technology, École de Technologie Supérieure</institution>
          ,
          <addr-line>Montréal</addr-line>
          ,
          <country country="CA">Canada</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Software Project Management Team, ENSIAS, Mohammed V University in Rabat</institution>
          ,
          <country country="MA">Morocco</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>In the present era, e-government portals are viewed as an efective means of providing online services to citizens. It is imperative for everyone to guarantee equitable access to these services, irrespective of their abilities or limitations. Hence, accessibility is vital for those with disabilities and advantageous for everyone. Web Content Accessibility Guidelines (WCAG) provide a comprehensive set of guidelines, in terms of success criteria, that are widely utilized in various tools to assess the accessibility of websites or portals. By following these guidelines, developers can ensure that their websites are accessible to everyone. This paper reports on a study of the accessibility evaluation of five Moroccan e-government portals using the AChecker tool with respect to WCAG. The aim was to prepare recommendations to improve the overall user experience across Moroccan e-government portals. The evaluation results analysis showed that the portals have some common as well as diferent failed success criteria with diferent numbers of errors per WCAG principle and conformance level. In particular, all the portals exhibited varying degrees of adherence to the four principles, and most of the portals struggled to meet levels A and AA, and none of them achieve perfect conformance across all levels. To address this issue, a set of recommendations is provided for each portal to enhance accessibility. The lessons learned from this study can contribute to more inclusive e-government portals and provide insights for governments and designers to prioritize accessibility enhancements.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Accessibility</kwd>
        <kwd>e-government portals</kwd>
        <kwd>evaluation tools</kwd>
        <kwd>WCAG</kwd>
        <kwd>disabilities</kwd>
        <kwd>impairments</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Accessibility encompasses the design of products, services, environments, and digital content to
accommodate the diverse needs and abilities of people [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. In the context of the web, high accessibility
ensures that digital content and features can be perceived, understood, navigated, and interacted with
by everyone; therefore, no one is excluded from using the web efectively [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. E-government portals
are becoming increasingly crucial in today’s digital era, transforming government engagement with
citizens by providing eficient, transparent, and accountable platforms. They ofer a wide range of
services such as applying for permits, paying taxes, and accessing government documents. The 24/7
availability of these portals eliminates the need for physical visits, saving time and resources for both
governments and citizens.
      </p>
      <p>
        Accessibility in the e-government portals takes on significant importance as it should ensure equal
access to government services and information for all individuals, regardless of their abilities or
disabilities Therefore developers should design and build inclusive e-government websites in order to
ensure that individuals with various abilities, backgrounds, and requirements can actively participate in
and derive full benefits from the information, services, and features ofered by their digital platforms [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
E-government portals with high accessibility have the capacity to narrow the digital divide and foster
inclusivity within society by removing obstacles that could impede individuals with disabilities from
accessing and utilizing online services. Indeed, accessibility in e-government portals should enable
persons with permanent disabilities (or impairments) such as auditory, cognitive, learning, neurological,
speech, visual, and physical disabilities, to fully access online services [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], as well as persons with
temporary impairments, situational limitations, and an aging population [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        By incorporating accessible design principles and adhering to established guidelines provided by
the Web Content Accessibility Guidelines (WCAG) standard [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], governments can guarantee online
access to all people for online information and services and can benefit from the advantages of digital
governance.
      </p>
      <p>
        In this context, the accessibility evaluation of e-government portals is mainly performed using
WCAGbased tools, which provide a quick and easy way to test a large number of websites for accessibility
conformance [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. This evaluation is essential to detect and address accessibility barriers, ensure that
the portals are user-friendly for people with disabilities, and highlight potential improvements. Many
accessibility evaluation tools have been developed (e.g., AChecker1 , WAVE2 , CynthiaSays3 ) and used
for accessibility evaluations in the literature:
• Study [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] evaluated six home pages of Saudi Arabian universities using SiteImprove and WAVE.
      </p>
      <p>
        The evaluation showed that the homepage of Taibah University was more accessible than the
other universities’ home pages. Results also showed that SiteImprove outperformed WAVE in
detecting web accessibility issues.
• Study [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] evaluated Croatia’s most visited desktop and mobile websites using AChecker tool. The
study found that only two of the 14 analyzed Croatian news portals have fully accessible designs.
      </p>
      <p>
        It also highlighted the impact of diferent web browsers which may afect accessibility.
• Study [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] evaluated Pakistan’s central government websites, including all ministries and divisions,
using Total Validator and Functional Accessibility Evaluator tools. The findings revealed that most
websites were not developed according to the WCAG standards, making it dificult or impossible
for visually disabled persons to access government information.
• Study [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] evaluated the accessibility of three Moroccan e-government websites using the
AccessiWeb tool. The results indicated the presence of several accessibility issues across all three
websites, afecting the three WCAG levels (A, AA, and AAA) criteria. The findings showed
that none of the websites met the minimum level of accessibility, necessitating corrections to
next levels issues to improve web content’s perceivability, operability, understandability, and
robustness for users with disabilities.
• Study [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] evaluated the impact of mobile learning on the inclusion of students with disabilities at
Mohamed V University in Rabat. The findings showed that mobile learning significantly enhanced
educational inclusion, motivation, and potential realization for students with disabilities. The
study highlighted the crucial value of mobile devices in promoting accessible and inclusive
education, suggesting the need for a thorough analysis of their contributions to students with
reduced mobility.
• Study [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] investigated the potential of using smartphones and tablets as alternative learning tools
for assistive technology (AT) devices in under-developed and developing countries like Morocco.
They compared seven free Google Play medical apps with seven traditional AT devices in terms of
functionality and afordability. The findings suggested that mobile learning through these devices
can provide accessible and cost-efective solutions for students with disabilities, addressing issues
such as hearing and visual impairments, autism, and speech articulation disorders.
      </p>
      <p>
        Furthermore, a 2021 review study on digital accessibility [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] reported that few studies were performed
by Moroccan researchers in this context, in particular about e-government [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] and education [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ],
[
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. This motivated us to focus in this study on e-government Moroccan portals, a context in which
accessibility evaluation is scarce.
      </p>
      <p>This study reports on an evaluation study of five Moroccan portals using the AChecker accessibility
tool and analyzes the results according to a set of criteria related to WCAG principles, conformance
1https://achecks.org/AChecker/
2https://wave.webaim.org/
3https://www.tpgi.com/cynthiasays-com-accessibility-website-scan-announcement/
levels, accessibility problems, as well as hypertext markup language (HTML), and cascading style sheet
(CSS) validation. The aim was to prepare recommendations to improve the overall user experience
across Moroccan e-government platforms. The contribution of this study is insights into the accessibility
status of the evaluated e-government portals, facilitating the identification of areas for enhancement
and ensuring a more inclusive online environment for all users.</p>
      <p>The subsequent sections of this paper are organized as follows: Section 2 ofers an overview of
the WCAG, and Section 3 introduces the AChecker accessibility tool. The methodology and results
of this study are outlined in Sections 4 and 5, respectively. Section 6 encompasses the discussion,
recommendations, and limitations, followed by the conclusions in section 7.</p>
    </sec>
    <sec id="sec-2">
      <title>2. WCAG: An Overview</title>
      <p>
        The Web Content Accessibility Guidelines (WCAG) established by the World Wide Web Consortium
(W3C) Web Accessibility Initiative (WAI) are a set of technical standards designed to ensure the
accessibility of web content to people with disabilities [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>
        WCAG content has evolved over the years; the first version (WCAG 1.0) was launched in 1999,
followed by WCAG 2.0 in 2008, WCAG 2.1, and WCAG 2.2 in 2018 and 2022, respectively. Each
version has undergone enhancements, striving to address a wide range of accessibility challenges for
disabled individuals. These guidelines recognize the changing nature of accessibility needs, aiming not
only to cater to specific disabilities, but also to enhance overall accessibility for users across diferent
devices. The WCAG 2.0 version, approved as an International Organization for Standardization (ISO)
40500 standard [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ], was used to evaluate the accessibility of some Moroccan e-government portals.
This guideline comprises four principles, 12 guidelines, and 61 success criteria classified into three
conformance levels [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], as described in the following paragraphs (see Figures 1 and 2 for a summarized
and detailed view of this guideline).
      </p>
      <p>
        The WCAG 2.0 guidelines aim to enhance the accessibility of content to users with diverse disabilities
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Each guideline includes a set of success criteria (SC) for specific objectives that the portal should
satisfy. These criteria are formulated as statements that can be evaluated as true or false when testing
specific web content [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
      </p>
      <p>
        Furthermore, an SC can be assessed into three conformance levels depending on the extent to which
accessibility guidelines are met (see Figure 2). These levels are useful to developers when designing and
building portals [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>• Level A refers to the guidelines that the website must adhere to ensure that all information is
accessible to users, including those with disabilities.
• Level AA refers to the guidelines that the website should follow to address significant accessibility
barriers and enhance the accessibility of information on a website.
• Level AAA ofers guidelines that could improve the usability of websites for individuals with
disabilities, making their browsing experience more comfortable.</p>
      <p>
        In addition, WCAG 2.0 proposes a set of four principles, referred to as “POUR,” that the website should
satisfy (Figure 1). Each principle includes a set of guidelines and their corresponding SC [? ]:
• Perceivable (P): “Users should be able to perceive information and user interface components,
ensuring that the presented information is not hidden from any of their senses.”
• Operable (O): “Users should be able to operate user interface components and navigate through the
interface, ensuring that the interface does not demand interactions beyond the user’s capability”.
• Understandable (U): “Users should be able to comprehend both the information and the functioning
of the user interface, ensuring that the content and operation are within their understanding
capabilities.”
• Robust (R): “Content should be reliable and adaptable to various user agents, including assistive
technologies, ensuring continued accessibility as technologies evolve.”
Moreover, to conform to WCAG 2.0, a webpage must adhere to the following set of requirements [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]:
• “One of the following levels of conformance is met in full”.
• “Conformance (and conformance level) is for full webpage (s) only, and cannot be achieved if part
of a webpage is excluded”.
• “When a Webpage is one of a series of webpages presenting a process (i.e., a sequence of steps
that need to be completed to accomplish an activity), all webpages in the process conform to
the specified level or better. (Conformance is not possible at a particular level if any page in the
process does not conform to that level or better.)”.
• “Only accessibility-supported ways of using technologies are relied upon to satisfy the success
criteria. Any information or functionality that is provided in a way that is not accessible and
supported is also available in a way that is accessibility supported”.
• “If technologies are used in a way that is not accessibility supported, or if they are used in a
non-conforming way, then they do not block the ability of users to access the rest of the page”.
Therefore, the WCAG 2.0 level compliance for a website is defined as follows [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]:
• Level A: “the minimum level of conformance when the webpage satisfies all the Level A success
criteria” [? ]. This corresponded to a total of 25 SC met.
• Level AA: when “the webpage satisfies all the Level A and Level AA success criteria” [ ? ]. This
corresponds to 38 SC to be met (13 (AA) + 25 (A)).
• Level AAA: when “the webpage satisfies all the Level A, level AA, and Level AAA success criteria”
[? ]. This corresponds to a total of 61 SC to be met (23 (AAA) + 13(AA) + 25(A)).
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. AChecker Accessibility Tool</title>
      <p>
        Many tools have been developed for accessibility evaluation; they difer in terms of features, but all
of them implement WCAG in diferent versions. The specificity of a tool refers to how many kinds
of issues it can find and explain; that is, a tool that can detect and describe a wide range of issues is
considered more powerful because it can ofer specicfi advice and suggestions to web developers and
make it more valuable to use [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ].
      </p>
      <p>
        The AChecker tool was chosen in this study based on a systematic review in [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] on automatic tools
used to evaluate university websites, which reported that it had been the most used for evaluation
purposes from 2011 to 2020. It has a wide range of customization options, covers most accessibility
guidelines [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], is frequently used in accessibility testing [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], and can check WCAG conformance and
principles.
      </p>
      <p>
        In addition, AChecker goes beyond accessibility evaluation by ofering HTML and CSS validation
checks [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. Its validation process entails checking webpage scripts against syntax rules and standard
specifications. Within this context, validating the HTML is important for evaluating web accessibility
[
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. Assistive technologies rely on these established standards when interacting with both the HTML
and CSS [23]. This AChecker web accessibility tool uses Jigsaw, a W3C tool used for checking a website’s
code to determine its adherence to formatting standards such as n HTML and CSS guidelines [24].
AChecker is a free web-based tool designed to evaluate websites for conformance with WCAG 2.0.
Notably, it stands out for its informative approach to error reporting; for each identified error, the tool
provides a comprehensive explanation of the issue and its impact on users and ofers multiple potential
solutions to address it. This tool identifies three types of accessibility problems, as provided in [25].
• “Known problems: These are problems that have been identified with certainty as accessibility
barriers. Webpages must be modified to address these problems”.
• “Likely problems: These are problems that have been identified as probable barriers but require a
human to make a decision. The webpage likely needs to be modified to address these problems”.
• “Potential problems: These are problems that AChecker cannot identify, which requires a human
decision.”
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. Evaluation Methodology And Execution</title>
      <p>The motivation behind this study is to evaluate the accessibility of the five Moroccan e-government
portals. The evaluation methodology used included three steps: 1) selection of the sample of Moroccan
e-government portals, 2) performing the accessibility evaluation, and 3) analyzing the evaluation results.</p>
      <sec id="sec-4-1">
        <title>4.1. Step 1 - Selection of the sample of e-portals</title>
        <p>
          To select the five Moroccan e-government portals, several factors were considered to ensure relevance
and representativeness, with the focus on evaluating accessibility as in [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ], such as diversity of services,
user trafic and popularity.
        </p>
        <p>Moroccan portals were selected based on their importance in providing essential services to citizens
and residents. Indeed, those portals are from diferent governmental entities (recorded with “. gov.ma”),
including from diferent sectors, thus ofering diverse e-government services in Morocco to develop
recommendations for enhancing user interaction with similar e-government services. Details regarding
the websites’ functionalities and services may not be disclosed for confidentiality reasons.</p>
        <p>Furthermore, the selected portals are among the most visited in Morocco. High user trafic indicates
that these portals are essential to a large number of citizens. By focusing on portals with substantial
daily interactions, our study addresses accessibility issues that impact many users. Notably, these
websites number of visitors range from 10.000 to 7.2 million visits per year according to SimilarWeb 4 .
This highlights their importance and frequent use. Evaluating these sites ensures that our findings and
recommendations will be relevant and benefit a significant portion of the population.</p>
        <p>Besides, the selected portals are popular and provide services eficiently, making them preferred
choices for users. Indeed, these websites are among the most visited websites in Morocco according to
SimilarWeb. By selecting them, we aim to assess and enhance the accessibility of platforms that are
already widely trusted and utilized. Improvements made to these sites can therefore lead to a more
inclusive and user-friendly experience for a broader audience.</p>
        <p>It should be noted that Moroccan websites are multilingual including (Arabic, French, English, and
Amazigh). All website versions had the same design and content, and the Arabic version was chosen for
the evaluation. For confidentiality purposes, these portals are referred to as Portals A, B, C, D, and E.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Step 2 - The Evaluation Erocess and Tools</title>
        <p>The AChecker tool was selected to evaluate the adherence of the chosen portals to the WCAG 2.0
standards mentioned in Section 2, and for its robust feature set, as outlined in Section 3.</p>
        <p>The evaluation itself took place with the e-government portal versions available on December 3,
2023, at 20:37 hours, utilizing Firefox 124.0.2 (64-bit) on Windows 11, with each portal’s homepage
being analyzed.</p>
        <p>The evaluation process checks the website’s HTML and CSS codes against the established accessibility
and coding standards. For instance:
• HTML content elements, such as images, links, headings, and form controls were evaluated to
ensure compliance with the WCAG criteria.
• The syntax and structure of the HTML markup were evaluated to ensure the proper nesting of
elements and appropriate attribute usage.
• CSS styles were evaluated for compliance with CSS specifications, including rules for layout,
typography, and color.</p>
        <p>The evaluation results were documented in five main parts, including known problems, likely
problems, and, when selected, HTML and CSS validation issues. In the evaluation report, failed SC were
listed alongside their corresponding guidelines, SC name, level of conformance, principle, check name,
and instructions on how to rectify the issue to achieve compliance with the specific SC.</p>
      </sec>
      <sec id="sec-4-3">
        <title>4.3. Step 3 - The Analysis of Evaluation Results</title>
        <sec id="sec-4-3-1">
          <title>The evaluation results were analyzed based on to four criteria:</title>
          <p>1. WCAG principles to identify the extent to which portals meet the “POUR” principles.
2. Conformance levels to see if the portals adhere to WCAG 2.0 standards.
3. Accessibility problems that the portals encounter and need resolution.</p>
          <p>4. HTML and CSS validation issues to be addressed to conform to the guidelines.</p>
          <p>The analysis was performed using qualitative and quantitative methods to develop insights into the
portals’ conformance with WCAG 2.0, standards, and recurrent accessibility issues.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Evaluation Results Analysis</title>
      <p>This section presents the overall results of the evaluation followed by the WCAG principles, conformance
levels, accessibility problems, and validation issues reported by the AChecker tool.</p>
      <sec id="sec-5-1">
        <title>5.1. Overall results</title>
        <p>The AChecker accessibility evaluation results of WCAG failed success criteria (SC) (designated by tool
as “Known problems) are reported for portal A to E in terms of the number of errors. It also reports
the total number of errors per portal, per failed SC, per WCAG principle, and conformance level, and
at the bottom, the number of failed SC. From Table 1, it can be observed that portals have diferent
occurrences of each failed SC (i.e., number of errors) per principle and level. A more detailed analysis is
presented next based on WCAG principles, conformance levels, accessibility problems, HTML, and CSS
validation.</p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. WCAG principles</title>
        <p>This subsection presents an analysis of the WCAG principles (Perceptible, Operable, Understandable,
and Robust) (see Figure 3).</p>
        <p>• Perceptibility:
• Operability:
– Portal E is the least perceptible with the highest number of errors, followed by Portals A, C,
and D, which face perceptibility challenges primarily related to “non-text content”, “resize
text”, and the “contrast”.
– Portal B is the most perceptible with the lowest number of errors, and encounters dificulties
in only two SC.
– Portal C demonstrates strong operability with a low number of errors, making user interface
components and navigation accessible.
– Portals B and E which face some operability issues, particularly in “link purposes”, then
Portals A and D with the highest number of errors, which exhibits notable operability
emphasizing the need for substantial improvements.
• Understandability:
• Robust:
– Portals A, D, and C exhibit lower levels of understandability due to several errors related to
“labels or instructions”, highlighting the necessity of a comprehensible digital environment.
– Portal B stands out as the most understandable: it shows no errors in this principle and
presents information efectively for the user.
– All portals show good adherence to the robust principle, with only one error identified
in portals A and E. This suggests that the portals have implemented coding practices and
content presentation approaches that align well with the diferent browsers and assistive
technologies.</p>
        <p>The accessibility of each portal across the four principles is presented in Figure 4 in terms of the
percentage of SCs satisfied per principle (regardless of the number of errors per failed SC):
• Portal B is the most perceptible (in red) with 91% (two failed SC among 22 of this principle),
followed by Portal C, D, and E with 87% (three failed SC) each, and Portal A with 82% (four failed
SC).
• All Portals are operable (in blue), with 90% (two failed SC among 20 of this principle, each).
• Portals A, C, D, and E are the most understandable (in green) with 95% (one failed SC among 17 of
this principle), while Portal B is the best with 100% conformance to this principle (no failed SC).
• Portals A and E are less robust (in black) with 50% (one failed SC among the two of this principle),
while the other three portals reached 100% because no failed SC was reported.</p>
        <p>To summarize, each of the portals showed good adherence to WCAG 2.0, with a range between 4 and 8
of failed SC among the 61 proposed by the guidelines (see Table 1). However, none of the portals fully
adhered to these four principles. Therefore, addressing a specific SC within each principle is important
for overall accessibility improvement across all portals.</p>
      </sec>
      <sec id="sec-5-3">
        <title>5.3. Conformance Levels</title>
        <p>This subsection presents an analysis of the WCAG conformance levels (A, AA, and AAA) (see Figure 5)
with the total number of errors for the failed SC of the five portals. As observed:
• The number of errors in Level A (in blue) varies across most Portals A particularly in Portals A
(72), D (66), and E (29). These errors represent critical accessibility barriers that directly hinder
the usability of these portals for disabled users.
• Portal E presented the highest number of errors at level AA (94) (in purple). These errors, often
categorized as "known problems" by AChecker, can significantly impact the user experience.
The other portals also exhibit errors (ranging from 5 to 21), but with significantly lower
occurrences. Understanding these errors and their impact across all portals is crucial for prioritizing
improvement eforts.
• Portals A (4), C (13), and D (2) present moderate numbers of errors at the AAA level (in yellow)
associated with advanced accessibility features. While not as frequent as in the A and AA
levels, their presence warrants attention, particularly when they pose barriers to specific user
groups. Interestingly, Portals B and E show no failed SC, suggesting that they excel at handling
accessibility at this highest level. However, it is important to note that the absence of failed SC at
the AAA level does not imply that a website is entirely accessible. To achieve this, the website
needs to meet all SCs of the three levels.</p>
        <p>• Among the 25 SC of Level A (e.g., the minimum level of conformance)
– Portal B satisfied 23 SC (92% conformance rate), demonstrating notable progress in enhancing
accessibility.
– Portals C and D achieved 88% conformance with each other (22 satisfied SC per portal).
– Portals A and E achieved 80% conformance (with 20 satisfied SC per portal), highlighting
the need for improvements to meet the foundational accessibility standards.
• Among the 13 SC of level AA (e.g., the mid-range level of conformance),
– All five portals shared the same percentage of conformance at this level, with 85% each (e.g.,
11 satisfied SC per portal).</p>
        <p>– All portals failed on two SC related to “resizing text and “headings and labels”.</p>
        <p>• Among the 23 SC of level AAA (e.g., the highest level of conformance),
– Portal B and E stand out with 100% conformance each.
– Portals D, A, and C achieved 96% conformance at this level, with one failed SC per portal.
– However, we cannot conclude that these portals meet the AAA level; all of them have errors
in levels A and AA.</p>
        <p>To summarize, each portal shows a diferent level of conformance to each level, and no portal has
reached the three levels of conformance. For example:
• Portal A showed 81 errors, with conformance rates of 80% (level A), 85% (level AA), and 96%
(level AAA).
• Portal B had 41 errors, achieving 92% (level A), 85% (level AA), and 100% (level AAA) conformity.
• Portal C exhibited 52 errors, with conformance rates of 88% (level A), 85% (level AA), and 96%
(level AAA).
• Portal D demonstrated 78 errors, achieving 88% (level A), 85% (level AA), and 96% (level AAA)
conformity.
• Portal E displayed 123 errors, with conformance rates of 80% (level A), 85% (level AA), and 100%
(level AAA).</p>
        <p>The identified errors indicate the need for continuous enhancements to meet all SC of accessibility
conformance levels recommended by WCAG 2.0.</p>
      </sec>
      <sec id="sec-5-4">
        <title>5.4. Accessibility Problems and Validation Issues</title>
        <p>As stated previously, the AChecker tool reports three types of accessibility problems (see Section
3); “known problems” related to failed success criteria have already been discussed in Section 5, while
those readily identifiable by automated tools “Likely problems” and those beyond the tool’s capacity for
detection “Potential problems” [25] are analyzed in this section. As it can be remarked:
• Portal E emerges with the highest count of “likely problems, " suggesting a higher likelihood of
existing accessibility issues that may impact the user experience. In contrast, Portal C had the
lowest count, indicating potentially better adherence to accessibility guidelines. This variation
across portals highlights the diverse landscapes of the accessibility challenges faced by each
portal. The most detected problem in all five portals is related to “change on request” SC in level
AAA, which highlights a specific concern related to links and anchors that open a new window,
potentially missing a warning.
• Portal E once again leads with the highest count of “potential problems”, which signifies a larger
set of issues that may impact accessibility in the future, requiring proactive action. Conversely,
Portal C maintained its position with the lowest count, setting a positive precedent for future
accessibility considerations. One of the most recurrent detected problems is related to the “use of
color” SC in level A: this highlights a specific concern related to images that may contain text
with poor contrast.</p>
        <p>The CSS validation highlights potential styling issues that can impact the visual presentation of
portals. Portals A, B, C, D, and E all exhibited varying counts of CSS validation issues, suggesting
potential challenges in maintaining a consistent and efective visual design. This signifies the need for
vigilance in ensuring that styling choices do not compromise the accessibility and usability of portals.
Some CSS errors detected by the tool are related to incorrect values of padding, unknown parameters,
or incorrect use of CSS styling attributes.</p>
        <p>The absence of HTML validation errors across all portals indicates strong adherence to proper
HTML syntax and rules, which confirms the efective coding practices implemented in the design and
implementation of these portals, suggesting that the structural foundation of the webpages is consistent
with established standards.</p>
        <p>These insights allow for the identification of targeted improvements in accessibility problems,
particularly in addressing specific SC and CSS validation issues across portals. Adherence to HTML standards
is commendable and provides a strong foundation for accessibility.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Discussion, Recommendations and Limitations</title>
      <p>This study has evaluated and reported on the accessibility of five Moroccan e-government portals (A, B,
C, D, and E) in conformance with WCAG 2.0 using the AChecker tool.</p>
      <p>Accessibility evaluation analysis was performed using the number of errors followed by the SC per
portal. As previously mentioned, the number of errors does not necessarily reflect their severity, as
diferent errors afect accessibility to varying degrees. However, it provides a general overview of the
accessibility of each portal. Notably, the evaluation outcomes underscore that, while Portal E has the
most severe accessibility challenges with a high number of errors, other portals may also encounter
significant barriers to accessibility, albeit with diferent error distributions.</p>
      <p>The next analysis in terms of the SC, regardless of the number of errors, allowed for additional
insights into each portal’s conformance with the WCAG. The main findings are summarized as follows,
with corresponding recommendations to enhance the accessibility of portals:
• Concerning the WCAG principles, all five portals exhibited varying degrees of adherence across
the four principles. While some portals demonstrate strengths in certain areas, such as
perceptibility or operability, others show notable challenges, particularly in areas related to understandable
and robust principles. In particular, Portal E presents significant perceptibility issues, whereas
Portal B has stronger operability features. However, none of the portals fully adhered to all the
WCAG principles, highlighting the need for targeted improvements to achieve comprehensive
accessibility.
• For conformance levels, the findings revealed a mixed level of conformance across portals. Most
portals struggle to meet the level A conformance SC, with notable challenges identified in Portals
A, D, and E. Level AA conformance also presents challenges for all portals, with Portal E exhibiting
a notably high number of failed SC. Portals B and E demonstrate strong conformance to the AAA
level. It is important to note that none of the portals achieved perfect conformance across all
levels, which underlines the importance of continuous improvement eforts to meet the diverse
needs of users with disabilities.
• Concerning accessibility problems, the findings revealed significant variations across portals.</p>
      <p>Portal E exhibited the highest number of problems, suggesting a higher likelihood and a larger
set of existing and future accessibility issues. In contrast, Portal C consistently had the lowest
count, indicating better adherence to accessibility standards. The most recurrently detected issues
were related to specific SC, such as “change on request” and “use of color” highlighting concerns
related to links, anchor behavior, and contrast in images.
• Regarding HTML validation, all portals demonstrated full adherence to proper HTML syntax
and rules with no reported validation error. This reflects efective coding practices and ensures
that the structural foundation of webpages is aligned with the established standards. In terms
of CSS validation, varying counts of validation issues were detected across all portals, signaling
potential challenges in maintaining consistent and efective visual designs. While the counts
were moderate, they underlined the importance of vigilance in ensuring that styling choices do
not compromise accessibility and usability. Examples of CSS validation issues include errors in
padding, sidebars, sections, headers, anchors, and menu-icon styling.</p>
      <p>
        In conclusion, these findings highlight the importance of enhancing accessibility across all evaluated
portals to ensure a more inclusive and user-friendly online experience for all citizens. These results
confirm the finding of the study [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] in the Moroccan context. This study has identified that none of
the evaluated websites met the minimum WCAG 2.0 level A with significant barriers across all four
principles: perceivability, operability, understandability, and robustness. For level AA, five criteria were
not respected in each website, afecting the principles of perceivability, operability, and understandability.
For level AAA, 11 to 13 criteria were not met across the websites, indicating substantial non-compliance
with advanced accessibility standards. This emphasizes the need for enforceable accessibility standards
and more comprehensive research to ensure inclusive access to digital services in Morocco.
      </p>
      <p>Table 4 provides a set of recommendations for each portal. By following these recommendations, the
accessibility of these five portals can be improved, and their content made more accessible to a wider
range of users, which will improve the overall user experience for everyone.</p>
      <p>Portals
Portal A</p>
      <p>Examples of recommendations
Prioritize improvements in “non-text content”, “info and relationships”,
“contrast”, “text resizing”, “Headings and Labels” and “link purposes”, to establish
a foundational structure for accessibility at level A. Examples of actions to
undertake are presented as the following:
• Ensure each image used as an anchor has descriptive alt text identifying</p>
      <p>its purpose or function.
• Provide a concise and descriptive label for each input element.
• Evaluate and adjust text and background colors to ensure a contrast</p>
      <p>ratio of 7:1 for standard text and 4.5:1 for larger text.
• Replace &lt;i&gt; elements with &lt;em&gt; or &lt;strong&gt; elements to provide</p>
      <p>semantic meaning along with styling.
• Ensure each &lt;label&gt; element contains descriptive text that identifies</p>
      <p>the associated input element.
• Ensure that only &lt;h1&gt; or &lt;h2&gt; follows an &lt;h1&gt; header.</p>
      <p>• Ensure that headers following an &lt;h3&gt; are &lt;h1&gt;, &lt;h2&gt;, &lt;h3&gt;, or &lt;h4&gt;.</p>
      <p>Portal B</p>
      <p>Direct eforts towards resolving issues in “non-text content”, “info and
relationships”, “contrast”, “text resizing”, “Headings and Labels” and “link purposes”,
such as:
• Make sure to add an alt attribute to &lt;img&gt; elements.
• Replace Bold elements &lt;b&gt; with &lt;em&gt; or &lt;strong&gt;.
• Ensure to add &lt;fieldset&gt; and &lt;legend&gt; elements to the form for</p>
      <p>each group of radio buttons.
• Ensure each anchor (&lt;a&gt;) element contains descriptive text or a title</p>
      <p>attribute.
• If an image is used within the anchor, add descriptive alt text to the</p>
      <p>image.</p>
      <sec id="sec-6-1">
        <title>Portal C and D</title>
        <p>Prioritize addressing errors related to “language of page”, “contrast”, and “text
resizing” as top priorities to establish a solid foundation for accessibility at
level A by taking into consideration these examples of actions:
• For HTML documents, add the lang attribute with a valid ISO-639-1
two-letter language code to the opening &lt;html&gt; element.
• For XHTML documents, add both the lang and xml:lang attributes
with a valid ISO-639-1 two-letter language code to the opening &lt;html&gt;
element.
• Evaluate and adjust text and background colors to ensure a contrast
ratio of 7:1 for standard text and 4.5:1 for larger text.</p>
        <p>• Make sure to remove the &lt;font&gt; element used in the document.</p>
      </sec>
      <sec id="sec-6-2">
        <title>Portal E Enhance operability by focusing on issues in “link purposes”, “headings”, and other operable elements to improve operability at level A conformance, such as:</title>
        <p>• Ensure each anchor (&lt;a&gt;) element contains descriptive text or a title
attribute.
• Ensure that only &lt;h1&gt; or &lt;h2&gt; follows an &lt;h1&gt; header.
• Ensure that headers following an &lt;h3&gt; are &lt;h1&gt;, &lt;h2&gt;, &lt;h3&gt;, or &lt;h4&gt;.</p>
      </sec>
      <sec id="sec-6-3">
        <title>Special consideration to perceivable principle as the portal faces the highest</title>
        <p>number of problems in “text resizing” due to the excess use of italic text.
Examples of actions to be taken are:
• Replace &lt;i&gt; elements with &lt;em&gt; or &lt;strong&gt; elements to provide
semantic meaning along with styling.</p>
      </sec>
      <sec id="sec-6-4">
        <title>Other actions to be taken concerning “labels or instructions” and “parsing”, such as: • Adding text to the &lt;label&gt; element. • Making sure that the id attribute is unique in the document.</title>
        <p>As for any study, these findings are limited by inherent limitations of the evaluation methodology
used, including
• Sample size: The study was limited by the small number of websites that were evaluated.</p>
        <p>Although the chosen Moroccan websites ofer valuable insights, the findings may not be fully
representative of all Moroccan e-government portals. Although the sample of portals is small,
the findings revealed that accessibility needs remedies because the evaluated portals encounter
many problems that should be addressed to reach higher levels of conformance.
• Evaluation approach: The evaluation relied primarily on an automated testing tool to identify
accessibility errors. Although this tool is useful, it may not capture all accessibility issues, leading
to potential gaps in the evaluation. Therefore, for a more extensive evaluation, it is suitable to
use multiple tools to benefit from their strengths and mitigate their weaknesses.
• Lack of user testing: This study did not include direct user feedback or testing, which could
provide valuable insights into the actual user experience and challenges faced by individuals with
disabilities when navigating websites. Therefore, complementing automated testing with user
and accessibility expert evaluations, user testing, and expert reviews is recommended for a more
comprehensive evaluation of website accessibility [26].
• Static analysis: The evaluation was based on a snapshot of the websites at a specific point in
time. Changes to websites after the evaluation period could afect the accuracy and relevance of
the findings. The evaluated portals can benefit from the WCAG recommendations, and additional
evaluation studies in the near future should be conducted to observe the progress of these portals
in terms of accessibility achievements.</p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>7. Conclusion</title>
      <p>In this study, the accessibility of five Moroccan e-government portals was evaluated with respect to
WCAG 2.0, using the AChecker tool. The results were analyzed according to WCAG principles, levels
of conformance, accessibility problems, HTML, and CSS validation. The main findings are summarized
as follows.</p>
      <p>• None of the portals fully met all the WCAG principles, emphasizing the necessity for focused
enhancements to ensure comprehensive accessibility. Indeed, the insights summarized per
the WCAG principle revealed distinctive challenges and performance across portals. While
certain portals excel in specific areas, such as perceptibility or operability, others face significant
challenges, particularly in terms of understandability and robustness. Portal E struggles with
perceptibility issues, whereas Portal B exhibits stronger operability features.
• All portals failed to meet the WCAG conformance levels, indicating a lack of accessibility as a
quality factor for the website. There are common accessibility challenges at level A across all
portals, emphasizing fundamental accessibility issues, especially for people with disabilities. Level
AA conformance revealed operability challenges, emphasizing the need for advanced accessibility
measures. The AAA conformance level indicated varying degrees of challenges, emphasizing the
commitment to providing an optimal user experience.
• Accessibility problems were identified in all portals, with Portal E having the highest count.</p>
      <p>HTML validation across all portals was flawless, indicating a strong adherence to proper HTML
syntax. However, varying numbers of CSS validations present potential challenges in maintaining
a consistent and efective visual design.</p>
      <p>Although accessibility is an important aspect of any website, especially e-government portals, few
eforts have been performed in this context. A set of recommendations was provided to help improving
accessibility not only of the five e-portals in this study, but also for other e-portals for an improved
positive user experience for a diverse range of citizens. Future work is needed to evaluate the same
portals using two other tools, and then provide the results to the e-government portals to tackle
accessibility issues identified and reevaluate iteratively until they reach a high degree of adherence to
the WCAG 2.0 standard. Furthermore, evaluating additional e-government websites using other tools,
complemented by manual evaluations would help identify and address a broader range of accessibility
issues across various platforms and therefore ensure comprehensive accessibility results.
[23] I. Doush, Z. AlMeraj, Evaluating the accessibility of kuwaiti e-government websites, Jordanian</p>
      <p>Journal of Computers and Information Technology (JJCIT) 5 (2019) 1.
[24] J. Team, What is jigsaw ?, 2020. URL: https://jigsaw.w3.org/css-validator/, accessed on June 15,
2024.
[25] M. B. Almourad, M. Hussein, F. Kamoun, Z. Wattar, Analysis of wcag 2.0 data accessibility success
criterion of e-government websites, Periodicals of Engineering and Natural Sciences 7 (2019)
496–503.
[26] H. S. Al-Khalifa, I. Baazeem, R. Alamer, Revisiting the accessibility of saudi arabia government
websites, Universal Access in the Information Society 16 (2017) 1027–1039.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Henry</surname>
          </string-name>
          , Introduction to web accessibility,
          <year>2024</year>
          . URL: https://www.w3.org/WAI/fundamentals/ accessibility-intro/,
          <source>accessed on March 15</source>
          ,
          <year>2024</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Henry</surname>
          </string-name>
          , Accessibility: It's about people,
          <year>2023</year>
          . URL: https://www.w3.org/WAI/people/,
          <source>accessed on March 3</source>
          ,
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>R.</given-names>
            <surname>Bonacin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Melo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. A.</given-names>
            <surname>Simoni</surname>
          </string-name>
          ,
          <string-name>
            <surname>M. C. C.</surname>
          </string-name>
          <article-title>Baranauskas, Accessibility and interoperability in e-government systems: outlining an inclusive development process</article-title>
          ,
          <source>Universal Access in the Information Society</source>
          <volume>9</volume>
          (
          <year>2010</year>
          )
          <fpage>17</fpage>
          -
          <lpage>33</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>J. B.</given-names>
            <surname>Shadi</surname>
          </string-name>
          Abou-Zahra,
          <article-title>Diverse abilities</article-title>
          and barriers,
          <year>2024</year>
          . URL: https://www.w3.org/WAI/ people-use-web/abilities-barriers/,
          <source>accessed on March 25</source>
          ,
          <year>2024</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>J. B.</given-names>
            <surname>Shawn Lawton</surname>
          </string-name>
          <string-name>
            <surname>Henry</surname>
          </string-name>
          , Shadi Abou-Zahra,
          <article-title>Older users and web accessibility: Meeting the needs of ageing web users</article-title>
          ,
          <year>2024</year>
          . URL: https://www.w3.org/WAI/older-users/, accessed on June 25,
          <year>2024</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>I. J. G.</given-names>
            <surname>Vanderheiden</surname>
          </string-name>
          , W. Chisholm,
          <article-title>Web content accessibility guidelines (wcag) 2.0, 2008</article-title>
          . URL: https://www.w3.org/WAI/older-users/, accessed on June 25,
          <year>2024</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>R.</given-names>
            <surname>Ismailova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Inal</surname>
          </string-name>
          ,
          <article-title>Comparison of online accessibility evaluation tools: an analysis of tool efectiveness</article-title>
          ,
          <source>IEEE Access 10</source>
          (
          <year>2022</year>
          )
          <fpage>58233</fpage>
          -
          <lpage>58239</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>A.</given-names>
            <surname>Alsaeedi</surname>
          </string-name>
          ,
          <article-title>Comparing web accessibility evaluation tools and evaluating the accessibility of webpages: proposed frameworks</article-title>
          . information,
          <volume>11</volume>
          (
          <issue>1</issue>
          ),
          <fpage>40</fpage>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>A.</given-names>
            <surname>Pirša</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Stanić</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Štracak</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Todorović</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Vdović</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Žilak</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Vuković</surname>
          </string-name>
          , Ž. Car,
          <article-title>Frontend solution for enhancing web sites accessibility</article-title>
          ,
          <source>in: 2015 13th International Conference on Telecommunications (ConTEL)</source>
          , IEEE,
          <year>2015</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>M.</given-names>
            <surname>Bakhsh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Mehmood</surname>
          </string-name>
          ,
          <article-title>Web accessibility for disabled: a case study of government websites in pakistan</article-title>
          ,
          <source>in: 2012 10th International Conference on Frontiers of Information Technology, IEEE</source>
          ,
          <year>2012</year>
          , pp.
          <fpage>342</fpage>
          -
          <lpage>347</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>I.</given-names>
            <surname>Boussarhan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Daoudi</surname>
          </string-name>
          ,
          <article-title>The accessibility of moroccan public websites: evaluation of three e-government websites</article-title>
          ,
          <source>Electronic Journal of E-government</source>
          <volume>12</volume>
          (
          <year>2014</year>
          )
          <fpage>67</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>S. BOUMAGHZEL</surname>
          </string-name>
          ,
          <string-name>
            <surname>M. H. ALAOUI</surname>
          </string-name>
          , Massalek atarbiya wa atakwine (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>J.</given-names>
            <surname>Ismaili</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. H. O.</given-names>
            <surname>Ibrahimi</surname>
          </string-name>
          ,
          <article-title>Mobile learning as alternative to assistive technology devices for special needs students</article-title>
          ,
          <source>Education and Information technologies 22</source>
          (
          <year>2017</year>
          )
          <fpage>883</fpage>
          -
          <lpage>899</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>F. E.</given-names>
            <surname>Chadli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Gretete</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Moumen</surname>
          </string-name>
          ,
          <article-title>Digital accessibility: a systematic literature review</article-title>
          ,
          <source>in: SHS Web of Conferences</source>
          , volume
          <volume>119</volume>
          ,
          <string-name>
            <given-names>EDP</given-names>
            <surname>Sciences</surname>
          </string-name>
          ,
          <year>2021</year>
          , p.
          <fpage>06005</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>A.</given-names>
            <surname>Al-Faries</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. S.</given-names>
            <surname>Al-Khalifa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. S.</given-names>
            <surname>Al-Razgan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Al-Duwais</surname>
          </string-name>
          ,
          <article-title>Evaluating the accessibility and usability of top saudi e-government services</article-title>
          ,
          <source>in: Proceedings of the 7th International Conference on Theory and Practice of Electronic Governance</source>
          ,
          <year>2013</year>
          , pp.
          <fpage>60</fpage>
          -
          <lpage>63</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>L. G. R. B.</given-names>
            <surname>Caldwell</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M</given-names>
            <surname>Cooper</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Vanderheiden</surname>
          </string-name>
          ,
          <source>Understanding wcag 2.0</source>
          ,
          <year>2016</year>
          . URL: https: //www.w3.org/TR/UNDERSTANDING-WCAG20/complete, accessed
          <source>on March 15</source>
          ,
          <year>2024</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>G.</given-names>
            <surname>Brajnik</surname>
          </string-name>
          ,
          <article-title>Comparing accessibility evaluation tools: a method for tool efectiveness</article-title>
          ,
          <source>Universal access in the information society 3</source>
          (
          <year>2004</year>
          )
          <fpage>252</fpage>
          -
          <lpage>263</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>M.</given-names>
            <surname>Campoverde-Molina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Luján-Mora</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Valverde</surname>
          </string-name>
          , Accessibility of university websites worldwide:
          <article-title>a systematic literature review</article-title>
          ,
          <source>Universal Access in the Information Society</source>
          <volume>22</volume>
          (
          <year>2023</year>
          )
          <fpage>133</fpage>
          -
          <lpage>168</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>M.</given-names>
            <surname>Alshamari</surname>
          </string-name>
          ,
          <article-title>Accessibility evaluation of arabic e-commerce web sites using automated tools</article-title>
          ,
          <source>Journal of Software Engineering and Applications</source>
          <volume>9</volume>
          (
          <year>2016</year>
          )
          <fpage>439</fpage>
          -
          <lpage>451</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>N.</given-names>
            <surname>Kesswani</surname>
          </string-name>
          ,
          <article-title>Accessibility analysis of indian government websites</article-title>
          ,
          <source>in: Conference Proceedings of ICDLAIR2019</source>
          , Springer,
          <year>2021</year>
          , pp.
          <fpage>179</fpage>
          -
          <lpage>189</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>R. A.</given-names>
            <surname>Galvez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. E.</given-names>
            <surname>Youngblood</surname>
          </string-name>
          , e
          <article-title>-government in rhode island: what efects do templates have on usability, accessibility</article-title>
          , and mobile readiness?,
          <source>Universal Access in the Information Society</source>
          <volume>15</volume>
          (
          <year>2016</year>
          )
          <fpage>281</fpage>
          -
          <lpage>296</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>M.</given-names>
            <surname>Akram</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. B.</given-names>
            <surname>Sulaiman</surname>
          </string-name>
          ,
          <article-title>A systematic literature review to determine the web accessibility issues in saudi arabian university and government websites for disable people</article-title>
          ,
          <source>International Journal of Advanced Computer Science and Applications</source>
          <volume>8</volume>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>