Accessibility testing of web applications:

A Systematic Literature Review

Sai Teja Paritala Ranjith Satla Sai Teja Bale
Information Technology Information Technology Information Technology
Montclair State University Montclair State University Montclair State University
Montclair, USA Montclair, USA Montclair, USA
Paritalas1@montclair.edu Satlar1@montclair.edu Bales1@montclair.edu

Abstract– This comprehensive study of the literature Making digital places inclusive is at the core
examines the state of web application accessibility of web application accessibility testing. It involves
testing techniques. The study identifies important creating and designing web apps in a way that can be
approaches, tools, and issues associated with assuring
used by people with a range of disabilities, including
accessibility through an analysis of a subset of papers. A
thorough overview is produced by synthesising
cognitive, motor, visual, and aural impairments [1].
empirical research, automated testing technologies, and The accessibility of web apps becomes crucial
common problems. Cross-question synthesis reveals because these programmes are the foundation of
links across approaches, resources, and difficulties, online interactions, including daily tasks and
enhancing our comprehension of their complex necessary services. Bridging the digital divide and
interrelationships. An investigation of patterns and offering equal chances to everyone, regardless of
trends explores how accessibility testing procedures ability, is the larger societal objective.
have changed over time. Reflection on quality The prevalence of web apps in contemporary life
assessment scrutinises methodological rigour and
underscores the significance of conducting
strengthens the credibility of the outcomes.
Professionals in the field are guided by implications for accessibility testing. These programmes are now
practice, which closes the gap between research findings necessary for everything from e-commerce websites
and useful application. The evaluation ends with to educational platforms that enable distance study
suggestions for future research, providing a basis for and streamline online transactions [2]. It is not only a
further development in the area of accessibility testing matter of convenience but also of fundamental rights
for online applications. to be able to use and navigate these platforms.
Therefore, web accessibility is a necessary rather
Keywords–Web Application, Accessibility Testing, than an optional component of ethical and
Systematic Literature Review, Methodologies, Tools, responsible digital development.
Challenges, Empirical Studies, Automated Testing, Testing for accessibility in web applications
Common Issues, Cross-Question Synthesis, Patterns and
Trends, Quality Assessment, Implications for Practice,
is very important since it guarantees that online
Industry Professionals, Future Work. environments are accessible to everybody. This
testing promotes inclusivity in a world that is
I. INTRODUCTION becoming more and more reliant on technology by
accommodating a range of skills, including visual,
Web application accessibility testing has hearing, motor, and cognitive impairments. It
emerged as a crucial aspect of technological progress embodies ethical digital development concepts and
in the constantly changing digital world. Web goes beyond compliance. The essential phase in
applications must be accessible to users with a wide today's web apps, which range from e-commerce
range of abilities due to the widespread influence of centers to educational platforms, is accessibility
web-based technologies in daily life, which includes testing. It guarantees that all users, regardless of
everything from e-commerce centres to educational ability, may interact with the platform without
platforms. Web application accessibility testing is difficulty, encouraging equal chances and reaffirming
vital for reasons other than compliance; it upholds the the dedication to creating a digital space that is
values of equality and inclusivity, making the digital inclusive and accessible to everyone.
world accessible and inclusive for all.
B. Existing Gap in Research
A. Significance of Web Application Accessibility Although there is general agreement about
Testing the importance of web application accessibility, there
is a clear vacuum in the literature regarding the contrast problems, and screen reader compatibility in
particular field of accessibility testing for these apps. order to pinpoint areas that need more focus and
The majority of existing research has tended to focus possible directions for innovation.
on broad concepts and standards for web It can be concluded that the goal of this
accessibility, frequently ignoring the unique systematic literature review is to make a valuable
difficulties brought about by the dynamic and contribution to both academic debate and practical
interactive character of web applications [3]. guidance for individuals working on accessible
The difference becomes more pronounced as online apps. This review aims to be a cornerstone in
technology advances and single-page applications furthering the cause of web accessibility by
and dynamic content become commonplace. The synthesising existing information, filling in present
inability of traditional accessibility testing gaps, and suggesting future research directions. The
approaches to adjust to these new technologies ultimate objective is to create a digital environment
creates a gap in our knowledge of how to test modern where web applications that exemplify the values of
online apps for accessibility. The dearth of thorough inclusion and equitable access are not only
research focusing on this particular specialty technically sound but also available to users with a
obstructs advancements in the area and prevents the range of abilities.
creation of customized testing approaches.
Furthermore, even though the fundamentals II. BACKGROUND
of web accessibility are widely known, there are very
few resources that thoroughly examine the plethora A. Introduction to Web Application Accessibility
of tools and platforms available for testing the Testing
accessibility of web applications. The variety of these The methodical assessment of web
instruments, each with advantages and disadvantages, applications to make sure they are inclusive and
presents a difficulty for professionals and scholars useable by people with different abilities is known as
looking for direction in negotiating this complex web application accessibility testing. It addresses the
terrain. difficulties presented by dynamic web technologies,
going beyond the scope of traditional testing. User-
C. Proposal of the Current Research centric design and compliance with international
This systematic literature study attempts to accessibility standards like WCAG depend on this
fill in these gaps by elucidating the complexities of testing. It tries to get rid of typical accessibility
web application accessibility testing. The review problems by taking into account elements like
aims to close the gap in the literature by usability and assistive technologies [5]. Web
concentrating on approaches, resources, and application accessibility testing is becoming more
difficulties associated with web application and more important as technology advances. It helps
accessibility testing. In order to support researchers, create an inclusive digital environment where users
practitioners, and developers who are working to of all abilities may easily access and interact with
improve the accessibility of web applications, it seeks web apps.
to offer a thorough and nuanced overview of the state
of the subject. B. Web Accessibility Principles and Guidelines
Three main research topics serve as the Creating inclusive digital places starts with
framework for this evaluation. First and foremost, the adhering to web accessibility principles and
review aims to shed light on the approaches and guidelines. The guidelines for creating accessible
strategies currently used in web application web content are outlined by these principles, which
accessibility testing. This means figuring out which are included in standards such as WCAG (Web
cutting-edge techniques, successful tactics, and best Content Accessibility Guidelines). They direct
practises exist in this particular field of testing [4]. developers in creating online applications that
The review's second objective is to examine the wide accommodate users of varying abilities by placing an
range of instruments and platforms that are accessible emphasis on perceivability, operability, and
for using when testing web apps for accessibility. It understandability. Respecting these rules is essential
aims to go beyond simple listing by offering a to guaranteeing a fair online experience. These
comparative assessment to help practitioners make standards, which define precise accessibility
well-informed judgements about which tools to use. requirements, enable developers to produce digital
Finally, the paper explores typical accessibility material that is both compliant and, more crucially,
problems and obstacles that arise throughout the globally accessible. This strengthens the fundamental
testing phase. The evaluation looks at concerns principles of inclusion and equal access in the digital
including keyboard navigation challenges, colour sphere.
 F. Methodologies in Web Application Accessibility
C. Evolution of Web Technologies and Challenges Testing
online accessibility testing has new issues as Web application accessibility testing
a result of the advancements in online technologies, methodologies include a variety of techniques to
such as dynamic content and single-page guarantee thorough assessments. They include both
applications. Testing approaches need to be automated testing that makes use of tools and scripts
reevaluated because traditional procedures find it and manual testing that requires human participation.
difficult to adjust to the dynamic and interactive While automated testing increases productivity,
character of current online applications. As the digital manual testing enables more nuanced evaluation of
world develops, maintaining accessibility is harder the user experience. The testing process is optimised
and harder [6]. One of the challenges is in when the two approaches are combined. The online
maintaining usability while addressing the application's specific accessibility features and
accessibility implications of emerging technologies. evaluation criteria will often determine which option
In order to maintain online applications' universal is best. Robust accessibility testing necessitates a
accessibility, it is imperative that testing procedures balance between automatic and manual techniques,
be adjusted to the always changing web architecture. providing a comprehensive view of the application's
This emphasises the dynamic nature of accessibility accessibility and enabling developers to handle a
difficulties and the necessity for thorough testing wide range of potential problems.
methodologies.
G. Tools and Platforms for Web Accessibility Testing
D. Key Concepts in Web Accessibility Testing There are numerous web accessibility
The three main ideas of web accessibility testing tools and platforms available that provide a
testing are assistive technologies, user experience, range of features to assess and improve digital
and usability. Web apps must be straightforward and inclusion. These technologies make it easier to find
easy to use for people of different abilities in order to and fix accessibility problems. They range from
be considered usable. The entire interaction is taken automated scanners to manual testing platforms.
into account while designing a user experience, with Among the notable instances are Axe, WAVE, as
accessibility being a key element. Accessibility is well as Pa11y [8]. The web application's nature and
greatly facilitated by assistive technologies, such as the particular testing requirements are major factors
screen readers and alternate input devices. in tool selection. By helping practitioners choose the
Comprehending these ideas is essential to developing best tools for their requirements, comparative
successful testing plans. Including assistive assessments guarantee a customised and successful
technologies, taking into account the user experience accessibility testing strategy. The dynamic landscape
as a whole, and integrating usability all work together of web accessibility testing is highlighted by the
to create web applications that are not only variety of tools that are available, offering both
technically sound but also truly inclusive, meeting developers and testers flexible solutions.
the needs of all users.
H. Common Accessibility Issues in Web Applications
E. Legislation and Standards in Web Accessibility Typical web application accessibility
The legal framework governing online problems are things that people with impairments run
accessibility standards and legislation requires and against on a regular basis. These concerns include
directs the development of inclusive digital issues with colour contrast, keyboard navigation
environments. Laws that require compliance with challenges, and compatibility with screen readers.
standards like WCAG emphasise web accessibility Providing an inclusive user experience requires
on a global and regional level [7]. Ensuring addressing these issues. While keyboard navigation
accessibility for all users requires organisations offers smooth interaction for individuals unable to
producing online apps to adhere to these regulatory use a mouse, screen reader compatibility guarantees
guidelines. These rules support the moral requirement that visually impaired users may access content.
of equal access and provide guidelines for developing Enhancing colour contrast also accommodates
accessibility testing methods. Comprehending the persons with vision difficulties. When creating online
legal environment pertaining to web accessibility is applications that prioritise accessibility and meet the
essential for maintaining legal compliance as well as various needs of all users, it is critical to identify and
cultivating an inclusive digital culture dedicated to address these typical problems.
offering accessible experiences for people with a
range of abilities. I. User-Centric Design and Inclusive Development
 In order to create online applications that
serve a broad user base, two fundamental ideas are L. Summary
inclusive development and user-centric design. The It can be concluded that the background section
end-user experience is given priority in user-centric provides an overview of the complex field of web
design, which guarantees the usability and application accessibility testing. Every subsection
accessibility of web applications. Beyond merely advances a thorough understanding of the subject,
adhering to standards, inclusive development fosters covering everything from guiding concepts and core
a culture of universal design by integrating ideas to the ever-changing difficulties presented by
accessibility considerations throughout the developing technology. The backdrop lays the
development lifecycle [9]. These guidelines place a groundwork for an organised investigation of the
strong emphasis on knowledge, empathy, and gaps in the current body of research by highlighting
responsiveness to user requirements, creating digital the importance of user-centric design, legal
environments that are both authentically inclusive frameworks, and a variety of testing approaches.
and technically competent. When inclusive Through the analysis of case studies and new
development methods and user-centric design are developments, the part emphasises how web
combined, web apps that are accessible to users with accessibility is flexible. Using this basic
different abilities and easy to use are produced. understanding, the next sections will conduct a
thorough literature study and provide
J. Emerging Trends in Web Application Accessibility recommendations and solutions to further the field of
The dynamic landscape of digital inclusion web application accessibility testing.
is reflected in the emerging trends in web application
accessibility. The use of artificial intelligence (AI) to III. LITERATURE SEARCH METHOD
improve accessibility testing is becoming more
popular, tackling issues like content comprehension A. Research Questions
and automatic image identification. With the
increasing use of smartphones, mobile accessibility is TABLE I. RESEARCH QUESTIONS
becoming more and more important, highlighting the
necessity for flexible and accessible design. PWAs, SL. Research Focus on Existing
or progressive web applications, are becoming more No Question Literature
and more popular, necessitating creative accessibility
solutions. These developments highlight how RQ1: According to
dynamic web accessibility is and how constant the existing
adaption to new technologies is required. In a digital literature, what are
world that is developing quickly, developers and the key Key methodologies and
testers who are dedicated to improving the methodologies and techniques in web
accessibility of online apps must keep up with these 1
techniques application
advancements. employed in accessibility testing.
accessibility
K. Case Studies and Best Practices testing of web
For developers and testers, case studies and applications?
best practises in web application accessibility provide
useful information. Analysing successful RQ2: According to
implementations in the real world teaches us the existing
important lessons about designing digital experiences literature, what are
Tools and platforms
that are accessible. These case studies show obstacles some
available for
and their resolutions that arose during the 2 tools/platforms
accessibility testing of
development process, demonstrating practical tactics. that can be used
web applications.
Successful apps have yielded best practises, which for accessibility
provide useful guidance for incorporating testing of web
accessibility into online development [10]. This applications?
knowledge-sharing promotes a community-driven
approach to web accessibility by highlighting RQ3: According to
exemplary cases and condensing best practises. It Common accessibility
the existing
also ensures that developers learn from both 3 issues and challenges in
literature, what are
successes and failures in their quest to create web web application testing.
the most common
applications that are universally accessible.
 accessibility issues 1.
and challenges Accessibilit ("Accessibility
encountered in y testing of testing" OR "Web
accessibility web accessibility
testing of web applications evaluation") AND
applications? methods ("web applications"
OR "websites" OR
"online applications")
B. Search Strings AND ("methods" OR
"techniques" OR "best
A thorough method of acquiring pertinent material is practices" OR
reflected in the search strings developed for the "approaches")
systematic literature review on web application
accessibility testing. The high-level search string
2.
highlights "Accessibility testing" or "Web
Tools and ("Web accessibility
accessibility evaluation" in relation to terms linked to
platforms testing tools" OR
web applications and methodologies for RQ1 which
for "Software for web
focuses on methodologies and techniques. Using
accessibilit application accessibility
phrases like "methods," "techniques," "best
y testing of testing") AND
practises," and "approaches," the comprehensive
web ("comparison" OR
search string further refines the search and guarantees
applications "evaluation" OR
a focused examination of the literature covering the
"review" OR "selection"
primary methodologies and techniques used in web
OR "best" OR
application accessibility testing.
"popular")
The high-level search string for Research Question 2
(RQ2), which focuses on tools and platforms,
includes terms like "Web accessibility testing tools" 3.
and "Software for web application accessibility Common ("Common issues
testing." By adding phrases like "comparison," accessibilit encountered in web
"evaluation," "review," "selection," "best," along with y issues in accessibility testing"
"popular," to the comprehensive search string, the web OR "Frequent
search is narrowed down and it becomes easier to application challenges in web
find articles that cover different platforms and tools testing accessibility
for accessibility testing web apps [11]. evaluation" OR
The high-level search string for Research Question 3 "Typical problems in
(RQ3), which focuses on frequent accessibility web accessibility
difficulties, includes terms associated with problems testing") AND
found during web accessibility testing. With terms ("screen reader
like "screen reader compatibility," "keyboard compatibility" OR
navigation difficulties," "alt text implementation "keyboard navigation
issues," and "colour contrast problems," the difficulties" OR "alt
comprehensive search string focuses in on relevant text implementation
literature and ensures a focused examination of issues" OR "colour
common problems encountered in web application contrast problems”)
accessibility testing [12].

TABLE II. SEARCH STRINGS 4.

Accessibilit "Accessibility testing
Stri High-Level Detailed search y testing strategies for
ng search string strategies websites" OR "Web
string for websites accessibility
evaluation methods"
OR "Techniques for
web accessibility
 D. Selection Criteria
testing" OR "Web
TABLE III. INCLUSION AND EXCLUSION CRITERIA
application
accessibility
assessment strategies" Inclusion Criteria Exclusion Criteria
OR "Website
accessibility testing 1. Articles that focus 1.Studies not related
best practices" OR on web application to any of the research
"Effective strategies accessibility testing questions
for web accessibility methodologies and
evaluation" OR techniques.
"Approaches to assess
website accessibility"

2. Include sources 2.Studies not in

that describe English
methodologies and
To summarise, the search strings have been carefully techniques for
designed to encompass a wide range of literature accessibility testing of
pertaining to accessibility testing for online web applications.
applications. This ensures a methodical and
exhaustive investigation of techniques, resources, and
obstacles within this field.
3. Include sources 3.Information that
C. Search Execution that discuss tools and does not pertain to
platforms for web application
A rigorous search procedure was used to conduct the accessibility testing of accessibility testing or
systematic literature review on web application web applications. related topics.
accessibility testing across a number of reliable
sources, including ScienceDirect, IEEE Xplore,
Google Scholar, and ACM Digital Library [13]. The
systematic execution of the searches throughout 4. Include studies 4. papers with no full-
several sessions made sure that all pertinent literature comparing the text availability.
was thoroughly examined. Iteratively, the effectiveness of
comprehensive and high-level search strings created different tools
for every study issue were used, enabling
optimisation and refining.
Initially, a broad net was to be cast, and the results
were to be gradually filtered to include the most 5. Include sources 5. Papers published
relevant sources. The goal of this iterative that describe common before 2010
improvement was to achieve a balance between accessibility issues
specificity and inclusivity. The databases that were and challenges in the
picked were chosen because they included a wide context of web
range of academic articles related to web applications.
accessibility. The methodical approach to search
execution improves the validity and scope of the
literature retrieved, offering a strong basis for the
review's later phases. Certain criteria that are intended to guarantee
● Databases: Google Scholar, IEEE Xplore, timeliness, relevance, and dependability are applied
ACM Digital Library, ScienceDirect. while choosing which articles to include in this
● Journals:International Journal, review. The following inclusion requirements will be
Requirements Engineering Journal, applied to articles that are subject to review:
Empirical Software Engineering. ● Focus on online Application Accessibility
● Additional Sources: Reference lists from Testing: The main focus of articles should
primary studies. be on the approaches and strategies involved
 in testing online applications for information is accessible for an exhaustive
accessibility. This criterion guarantees that review, papers without full-text availability
the materials chosen are in direct alignment will be disqualified.
with the main theme of the study. ● Pre-2010 Publication Date: Due to the
● Approaches and Strategies Description: constantly changing web technology and
Sources that offer a thorough explanation of accessibility standards landscape, articles
methods and strategies created especially for published previous to 2010 will be given
evaluating the accessibility of web apps are less weight in favour of more current and
included. This guarantees that the updated material. This chronological
information significantly advances constraint guarantees the findings' validity
knowledge of useful strategies in this field and usefulness in modern settings.
[14].
● Explain of about Tools and Platforms: This
section will contain articles that talk about E. List of selected papers
the tools and platforms that are used while
assessing the accessibility of online
applications. This criterion attempts to cover
a wide range of materials that deal with
theoretical approaches as well as the real-
Fig 1: “Overview of the steps in the systematic literature review
world implementation of pertinent and Steps included to generate the search strings” [1]
instruments.
● Comparison Studies: Included are studies
that evaluate the relative merits of several TABLE IV. LIST OF PAPERS SELECTED
web application accessibility testing tools.
This criterion makes it easier to investigate S Papers Links Authors and
empirical data on the effectiveness and L Year
results of various accessibility testing .
instruments. N
● Accessibility Problems and Difficulties: o
Sources outlining typical accessibility .
problems and difficulties in the context of
1 Empirical https:// Campoverde-
web applications are included in the
. studies on ieeexplore.ie Molina, M.,
inclusion criteria. This guarantees a
web ee.org/ Lujan-Mora, S.
thorough comprehension of the
accessibility iel7/6287639 and Garcia,
environment, including obstacles and
of /8948470/09 L.V., 2020.
possible fixes.
educational 092982.pdf
In contrast, in order to expedite the selection
websites: A
process, the following exclusion criteria will be
systematic
implemented:
literature
● Irrelevance to Research Questions: In order
review
to keep this review focused and coherent,
studies that do not directly address the 2 Bility: https:// Vontell, A.R.,
research questions that inform it will be . automated dspace.mit.e 2019.
omitted. accessibility du/
● Non-English Studies: In order to maintain a testing for bitstream/
unified and easily understandable analysis, mobile & handle/
articles that have not been published in web 1721.1/1216
English will be disqualified for linguistic applications 85/11020575
reasons. 44-MIT.pdf?
● Relevance to Accessibility Testing: In order sequence=1
to preserve the review's specificity, anything &isAllowed
that does not concern web application =y
accessibility testing or related subjects will
be omitted [15]. 3 A heuristic https:// Acosta-Vargas,
● Limited Accessibility (No Full-Text . method to ieeexplore.ie P., Salvador-
Availability): To guarantee that the
 evaluate web ee.org/ Ullauri, L.A. web publication/
accessibility iel7/6287639 and Luján- accessibility 334596375_
for users /6514899/08 Mora, S., 2019. Bibliometric
with low 822682.pdf _Analysis_of
vision _Global_Sci
entific_Liter
4 The current https:// Yan, S. and ature_on_We
. status of dl.acm.org/ Ramachandran, b_Accessibil
accessibility doi/abs/ P.G., 2019. ity/links/
in mobile 10.1145/330 5d33ef00a6f
apps 0176 dcc370a51df
1d/
5 Accessibility https:// Alshayban, A.,
Bibliometric-
. issues in dl.acm.org/ Ahmed, I. and
Analysis-of-
Android doi/pdf/ Malek, S.,
Global-
apps: state of 10.1145/337 2020, June.
Scientific-
affairs, 7811.338039
Literature-
sentiments, 2
on-Web-
and ways
Accessibility
forward
.pdf
6 Comparing https:// Alsaeedi, A.,
1 Evaluating https:// Alajarmeh, N.,
. web www.mdpi.c 2020.
0 the link.springer. 2022.
accessibility om/2078-
. accessibility com/article/
evaluation 2489/11/1/40
of public 10.1007/
tools and /pdf
health s10209-020-
evaluating
websites: an 00788-7
the
exploratory
accessibility
cross-country
of webpages:
study
proposed
frameworks
E. Methodological Rigor
7 AccGuideLin https:// Ara, J. and Sik-
. er: Towards ieeexplore.ie Lanyi, C.,
a Modelling ee.org/ 2023, May.
Approach of abstract/
Web document/
Accessibility 10223541/
Requirement
s following
WCAG 2.2 Fig 2: “Steps included in the selection process” [1]
8 Tools for an https:// Rubano, V.,
. Innovative link.springer. Vitali, F. and The approach places a high value on
Approach to com/ Lengua, C., thoroughness while searching the literature while
Web chapter/ 2022, June. preserving repeatability and transparency. Following
Accessibility 10.1007/978- established inclusion and exclusion standards
3-031- guarantees a methodical and targeted approach to
05028-2_6 paper selection. Through the methodical inclusion of
papers that directly answer the research questions and
9 Bibliometric https:// Aidi Ahmi, the exclusion of irrelevant or non-English studies, the
. analysis of www.researc R.M., 2019. rigorous procedure seeks to improve the
global hgate.net/ trustworthiness of the review. The thorough
scientific profile/Aidi- documentation of the search approach offers an open
literature on Ahmi/ road map for future repeatability [16]. This
dedication to methodological rigour protects the
 state of web accessibility testing methodologies is
provided by the synthesis through the identification
of trends, patterns, as well as gaps in the literature
[17]. It presents a balanced analysis of the methods
and tools discussed in the literature, with perceptive
details on their applicability and potential areas for
improvement. Through the process of identifying
underlying themes, this analytical method contributes
to the advancement of knowledge in the field and
lays the groundwork for informed decisions.

I. Quality Assessment:
Selected papers are critically evaluated
throughout the quality assessment process, with an
emphasis on each dependability and methodological
robustness of the source. Each paper's level of
evidence is evaluated systematically, taking into
account several aspects like sample size,
methodological rigour, and research design. This
procedure guarantees a critical approach to the
selection of literature for the review, highlighting
sources that make a significant contribution to the
research issues. This phase gives the synthesised
insights more credibility by offering a nuanced
assessment of the strength of evidence. This allows
readers and researchers to evaluate the validity and
review's integrity, promoting trust in the selection dependability of the findings offered in the literature
procedure and, eventually, enhancing the solidity of review.
the synthesised findings. IV. RESULTS

G. Data Extraction A. Introduction to Results:

The focus of the data extraction step is on The findings introduction includes a brief
methodically summarising important details from summary of the selected publications, giving an
chosen papers. This entails pulling out specifics overview of the range of literature covered in the
about the techniques used, the tools talked about, and review. The audience is more prepared for the
important conclusions that are made in the literature. findings that will be discussed later when the
The procedure guarantees a thorough understanding introduction explains the organisation in light of the
of the field of web accessibility testing techniques. research questions. The selected papers together
Upholding constant uniformity in the extraction of address a variety of web application accessibility
data is crucial, since it enhances the dependability of testing-related topics, such as challenges, resources,
the results. By meticulously documenting relevant and testing methodologies. This section describes the
facts, this stage ensures that the material acquired is systematic procedure that ensures the intricate
in conformity with the established study objectives. problems related to accessibility testing are explained
This establishes an organised framework for more in an understandable manner by pulling knowledge
research. Sustaining consistency in the extraction of from the literature. The alignment of the structure
data enhances the validity of the study findings and with the research topics facilitates a methodical
boosts the reliability of the combined insights. exploration of significant issues and advancements in
the field of accessible online applications.
H. Synthesis of Literature
A comprehensive synopsis of the most B. Research Question 1: Key Methodologies and
significant findings from the selected literature is Techniques in Accessibility Testing (RQ1)
”
included in the literature synthesis, which
methodically addresses every research topic. This In response to Research Question 1 (RQ1),
phase makes an effort to compile all of the data the results section unveils a comprehensive
acquired from diverse sources into a coherent presentation of findings derived from an analysis of
narrative. A comprehensive overview of the current selected papers. The compilation summarises the
many approaches and strategies used in the field of applications, making it challenging to achieve
web application accessibility testing as described in thorough coverage.
the literature [18]. The careful distillation of • Subjectivity: Results may vary based on the
commonalities and variances among studies yields a individual tester's expertise and perspectives,
deeper picture of the prevalent practises. The section introducing a level of subjectivity.
integrates information from multiple sources to create
a cogent narrative that sheds light on the intricacies III. User Testing:
of accessibility testing methodologies [19]. This Overview: User testing involves real users, including
synthesis clarifies the current patterns while also those with disabilities, interacting with the web
highlighting anomalies that contribute to a thorough application. Observations, feedback, and user
and contextualized knowledge of the primary tactics experiences are collected to understand how
employed to ensure the accessibility of web individuals with diverse abilities navigate and use the
applications. application.[13]
Benefits: User testing provides authentic insights into
Some Key Methodologies and Techniques in the real-world experiences of individuals with
Accessibility Testing are: disabilities. It helps identify usability challenges and
ensures that the application is genuinely accessible to
I. Automated Testing Tools: its intended audience.
Overview: Automated testing tools, such as Axe, Drawbacks:
WAVE, and Pa11y, are designed to automatically • Resource-Intensive: Conducting user tests with
scan web applications and identify accessibility a diverse group of participants can be
issues. These tools follow established guidelines like resource-intensive, requiring time, effort, and
WCAG (Web Content Accessibility Guidelines) and potentially financial resources.
can efficiently detect common problems like missing • Difficulty in Recruiting: Finding participants
alternative text for images, improper heading with specific disabilities for targeted testing
structures, or keyboard navigation issues.[8] may be challenging.
Benefits: Automation allows for a quick and
systematic evaluation of large-scale applications. It IV. Heuristic Evaluation:
helps catch common accessibility issues in an Overview: Heuristic evaluation involves experts
efficient manner.[8] assessing a web application against a set of
Drawbacks: predefined heuristics or principles. These heuristics
• Limited Context Understanding: Automated often align with established accessibility guidelines,
tools may struggle to fully understand the and the evaluation helps identify potential issues
context of certain user interactions, leading to without involving actual users.[3]
false positives or negatives. Benefits: Heuristic evaluation is a quick and
• Inability to Test User Experience: They may systematic method to uncover potential accessibility
not capture the overall user experience, issues. Experts can provide valuable insights based
including subjective aspects like the clarity of on their knowledge of accessibility principles.[3]
content. Drawbacks:
• Limited to Expertise: Results heavily depend
II. Manual Testing: on the expertise of the evaluators, and they
Overview: Manual testing involves human testers might not capture the diverse range of user
carefully reviewing and interacting with a web experiences.
application to identify accessibility barriers. Testers • May Miss User Perspectives: Heuristic
use assistive technologies, such as screen readers or evaluations may not fully represent the
keyboard navigation, to navigate through the perspectives of users with different abilities.
application and assess its overall usability and
accessibility.[11] V. Functional Testing with Accessibility
Benefits: Manual testing provides a nuanced Considerations:
understanding of the user experience. Testers can Overview: This approach integrates accessibility
uncover issues that automated tools might miss and considerations into regular functional testing
assess the actual usability of the application. processes. Testers evaluate the functionality of the
Drawbacks: application while also assessing its adherence to
• Time-Consuming: Manual testing can be time- accessibility standards, ensuring that both aspects are
consuming, especially for large and complex thoroughly tested.[12]
Benefits: By combining functional testing with going beyond a simple list [20]. This section provides
accessibility checks, this method ensures that a thorough overview of the accessibility testing tool
accessibility is considered an integral part of overall ecosystem by summarising information from many
application quality. It allows for a comprehensive sources. This comprehensive analysis contributes to
assessment of the application's features and our understanding of popular tools' potential
accessibility.[12] influence as well as usefulness in improving web
Drawbacks: application accessibility by identifying and critically
• May Not Capture All Issues: While this assessing their various elements.
approach ensures that accessibility is
considered, it may not capture all potential Some Tools and Platforms for Accessibility Testing
accessibility issues without specific focus on are:
accessibility testing.
• Integration Challenges: Integrating I. Axe:
accessibility considerations seamlessly into Overview: Axe is an open-source accessibility testing
functional testing may pose challenges. tool known for its user-friendly interface, aiding
developers and testers in identifying and addressing
VI. Cross-Browser and Cross-Platform Testing: accessibility issues within web applications. It
Overview: Accessibility should be consistent across operates as a browser extension or integrates into
different browsers and platforms. This method different development environments, automating the
involves testing the web application on various testing process and providing actionable insights for
browsers (e.g., Chrome, Firefox, Safari) and resolution [3].
platforms (e.g., Windows, macOS, mobile devices) to Benefits: Axe stands out for its user-friendly
identify any inconsistencies or issues related to interface, accessible to developers and testers.
accessibility.[16] Automated testing capabilities streamline the
Benefits: Cross-browser and cross-platform testing identification of common accessibility issues,
ensure that users have a consistent and accessible offering real-time feedback during development. Its
experience regardless of the technology they use. It simplicity enhances the efficiency of accessibility
addresses the diverse ways people access web testing, aiding in the creation of inclusive web
content.[16] applications.
Drawbacks: Drawbacks: Axe's focus on in-browser assessments
• Cannot Cover All Scenarios: It may be might limit its versatility across various development
impractical to test on every possible browser environments. Users may face a learning curve,
and platform combination, leading to potential especially with advanced features. Additionally,
oversights. being a browser extension, Axe may not provide a
• Resource-Intensive: Testing across multiple comprehensive view of accessibility in offline
platforms and devices can be resource- contexts.
intensive, especially for organizations with
limited testing resources. II. WAVE (Web Accessibility Evaluation Tool):
It's essential to recognize that no single method is Overview: WAVE is a comprehensive suite of tools
foolproof, and a combination of these methods is designed to assist authors in enhancing the
often necessary to achieve a comprehensive accessibility of web content. It provides visual
evaluation of web application accessibility. feedback through browser extension or an online
Overcoming these drawbacks often involves careful interface, aiding in identifying potential accessibility
planning, continuous monitoring, and a commitment barriers and offering guidance on improvement [11].
to addressing issues as they arise during the testing Benefits: WAVE injects visual indicators into web
process. pages, offering a clear representation of accessibility
status. Its browser extension provides comprehensive
C. Research Question 2: Tools and Platforms for feedback, aiding in a more intuitive understanding of
Accessibility Testing (RQ2) potential issues.
The results section carefully gathers and Drawbacks: WAVE's sensitivity might lead to false
synthesizes information from the chosen articles in positives, requiring users to discern genuine issues.
response to Research Question 2 (RQ2), with an Dependence on specific browsers could limit its
emphasis on web application accessibility testing utility in diverse development environments.
tools and platforms. This synthesis evaluates the
instruments mentioned in the literature based on their III. Pa11y:
unique characteristics and intended applications,
Overview: Pa11y is an automated accessibility testing themes that capture the most frequent challenges
tool that seamlessly integrates into the development faced while seeking to provide accessibility [21]. The
pipeline, ensuring accessibility checks throughout analysis provides a thorough understanding of the
development. It follows the Web Content methods used in the literature to address these issues
Accessibility Guidelines (WCAG) and generates and includes potential remedies [12]. The results of
detailed reports for evaluation [19]. this synthesis throw light on the complex terrain of
Benefits: Pa11y integrates into continuous integration accessibility concerns and offer valuable perspectives
processes, ensuring accessibility checks. for scholars and practitioners who aim to tackle these
Customizable rules and a command-line interface obstacles in the online application testing domain.
offer flexibility, enhancing overall efficiency in
automated testing scripts. Some Common Accessibility Issues and Challenges
Drawbacks: Configuring certain aspects of Pa11y are:
may be complex for some users. Reliance on
generated reports, lacking visual feedback, may I. Compatibility Across Devices and Browsers:
require additional interpretation. Challenge: Ensuring consistent accessibility across
IV. Axe Accessibility Checker: various devices and browsers poses a significant
Overview: Axe Accessibility Checker is a browser challenge. Web applications must cater to users with
extension providing real-time feedback on the diverse assistive technologies and use different
accessibility status of web pages. Focused on user- browsers and devices. Achieving uniform
friendliness, it identifies issues such as missing accessibility in this varied landscape requires
alternative text and color contrast violations [7]. thorough testing and adaptation to address potential
Benefits: Axe Accessibility Checker provides real- compatibility issues [8].
time feedback, fostering immediate issue resolution. II. Dynamic Content and Single Page Applications
Its user-friendly interface caters to developers of (SPAs):
varying expertise levels, and an active community Challenge: Web applications with dynamic content
ensures ongoing improvements. and SPAs often update and change elements without
Drawbacks: Similar to other browser extensions, Axe full page reloads. This dynamic nature can present
Accessibility Checker's primary focus on in-browser challenges for screen readers and other assistive
assessments might limit its scope. Users might technologies to interpret and convey changes
encounter a learning curve with advanced features. effectively. Testing must account for these dynamic
interactions to guarantee an accessible user
V. AChecker: experience [6].
Overview: AChecker is an open-source web III. Keyboard Accessibility:
accessibility evaluation tool available as an online Challenge: Many users with disabilities rely on
service or for local installation. It supports multiple keyboards or alternative input devices to navigate
accessibility standards and generates detailed reports web applications. Ensuring that all interactive
for identifying and addressing potential issues [22]. elements are keyboard accessible, focusable, and
Benefits: AChecker's open-source nature allows user provide meaningful feedback is crucial. Keyboard
contributions and customization. Supporting multiple navigation should be intuitive and efficient,
accessibility standards, it offers flexibility, and the addressing challenges such as focus traps and proper
option for online service or local installation adds to order of navigation [3].
user convenience. IV. Complex User Interfaces and Rich Interactions:
Drawbacks: AChecker's interface may be perceived Challenge: Modern web applications often feature
as less intuitive than more streamlined tools. While complex user interfaces with rich interactions, such
automating certain checks, it might not cover all as drag-and-drop functionalities and interactive
aspects of automated testing, necessitating widgets. Ensuring the accessibility of these elements
supplementary evaluations. can be challenging, as assistive technologies may
struggle to interpret and convey the intended user
D. Research Question 3: Common Accessibility experience. Testing should validate that all users,
Issues and Challenges (RQ3) regardless of ability, can interact meaningfully with
Research Question 3 (RQ3) is addressed in these features [11].
the results section with a brief review of common V. Inadequate Alternative Text for Non-Text Content:
accessibility problems and obstacles that are Challenge: Providing alternative text for non-text
described in the chosen web application testing content, such as images and multimedia, is a
literature. Through a process of synthesising data fundamental aspect of web accessibility. However, a
from a variety of sources, the section names recurrent common challenge is ensuring that alternative text is
not only present but also descriptive and contextually thorough analysis of the calibre and dependability of
relevant. Inadequate or missing alternative text can the chosen articles is conducted. This part offers
hinder users with visual impairments from readers a critical viewpoint by recognising and
understanding the content, emphasizing the need for assessing the methodological foundations. This
thorough testing and adherence to accessibility encourages a careful reading of the summarised
guidelines [2]. findings and their implications for the larger
discussion of web application accessibility testing.
E. Cross-Question Synthesis
Research Questions 1, 2, and 3 (RQ1, RQ2, H. Implications for Practice and Industry:
and RQ3) are combined in the cross-question The part pertaining to implications for
synthesis in the findings section to provide a coherent practise and industry provides significant insights
story that highlights common themes and that are obtained from the combined research on
relationships in the field of online application accessibility testing for online applications. It
accessibility testing [15]. This part provides an outlines how the field's practical approaches might be
overall view by methodically integrating information informed by the methods, tools, and insights that
on tools, concerns, challenges, and techniques. This have been learned about common challenges. This
thorough overview highlights the interplay and section also considers the wider ramifications for
interconnectedness of essential factors, going beyond industry experts that perform accessibility testing [8].
individual question analysis. This synthesis It provides professionals with the knowledge they
contributes to a comprehensive understanding of the need to improve their testing procedures and adjust to
area and directs future research and practical new trends by condensing the useful suggestions
applications by giving readers a nuanced grasp of the from the chosen papers. With the ever-changing
complex relationships between approaches, tools, and world of technology and user experience, the insights
the frequent issues faced in assuring accessibility offered aim to close the gap between research and
[16]. application, promoting a more efficient and
knowledgeable approach to web application
F. Discussion of Patterns and Trends accessibility testing.
The patterns and trends section explores the
results through a perceptive analysis that reveals V. CONCLUSION
complex relationships between the tools, concerns, The main conclusions and learnings from
challenges, and methods used in the chosen papers. the study project on web application accessibility
This section provides an in-depth analysis of the testing techniques are summarised in the Conclusion
complex interactions between these aspects, going section. Combining data from a few chosen studies,
beyond a cursory investigation. Examining the ways this section provides a thorough summary of the
in which approaches and tools interact, as well as the approaches, resources, and difficulties in this field. It
ways in which general problems connect to particular becomes apparent that many approaches and
difficulties, the talk reveals evolving patterns in the instruments are utilised to tackle accessibility
field of web application accessibility testing [18]. It concerns; yet, recurring difficulties endure. The
also examines how these patterns have changed over cross-question synthesis contributes to a
time, giving the analysis a temporal component. This comprehensive knowledge by illuminating complex
comprehensive analysis not only advances relationships between approaches, resources, and
knowledge of existing procedures but also establishes difficulties. Examining patterns and trends helps to
a foundation for predicting future advancements in clarify how accessibility testing is developing. The
the industry. reflection on quality evaluation emphasises the
requirement of exercising caution when interpreting
G. Quality Assessment Reflection results based on the methodological rigour of certain
The reflection on quality evaluation part studies.
delves further into the issue of how reliable the data In the future, the implications for business
that was taken from the chosen publications is. The and practise stress practical advice for experts
integrity of the results is examined by a critical conducting accessibility testing. The study opens the
reflection on the methodological rigour utilised in door for more research by pointing up potential areas
these research. This discussion provides insights into for improvement and examination. This work
the strengths and potential limitations of the data establishes a basis for future research by pointing out
offered by taking into account issues including gaps and new trends. In addition to providing an
sample size, study design, and overall research overview of the state of the art, the Conclusion
technique [17]. The results are more credible when a section paves the way for further research and
encourages ongoing advancements in web application [21] Kong, P., Li, L., Gao, J., Liu, K., Bissyandé, T.F. and Klein,
J., 2018. Automated testing of android apps: A systematic
accessibility testing procedures. literature review. IEEE Transactions on Reliability, 68(1), pp.45-
66.

REFERENCES
[1] Lenberg, P., Feldt, R. and Wallgren, L.G., 2015. Behavioral
software engineering: A definition and systematic literature
review. Journal of Systems and software, 107, pp.15-37.
Campoverde-Molina, M., Lujan-Mora, S. and Garcia, L.V., 2020.
Empirical studies on web accessibility of educational websites: A
systematic literature review. IEEE Access, 8, pp.91676-91700.
[2] Vontell, A.R., 2019. Bility: automated accessibility testing for
mobile applications (Doctoral dissertation, Massachusetts Institute
of Technology).

[3] Acosta-Vargas, P., Salvador-Ullauri, L.A. and Luján-Mora, S.,

2019. A heuristic method to evaluate web accessibility for users
with low vision. IEEE Access, 7, pp.125634-125648.
[4] Yan, S. and Ramachandran, P.G., 2019. The current status of
accessibility in mobile apps. ACM Transactions on Accessible
Computing (TACCESS), 12(1), pp.1-31.
[5] Alshayban, A., Ahmed, I. and Malek, S., 2020, June.
Accessibility issues in Android apps: state of affairs, sentiments,
and ways forward. In Proceedings of the ACM/IEEE 42nd
International Conference on Software Engineering (pp. 1323-
1334).
[6] Alsaeedi, A., 2020. Comparing web accessibility evaluation
tools and evaluating the accessibility of webpages: proposed
frameworks. Information, 11(1), p.40.
[7] Ara, J. and Sik-Lanyi, C., 2023, May. AccGuideLiner:
Towards a Modelling Approach of Web Accessibility
Requirements following WCAG 2.2. In 2023 IEEE International
Conference on Smart Information Systems and Technologies
(SIST) (pp. 10-15). IEEE.
[8] Aidi Ahmi, R.M., 2019. Bibliometric analysis of global
scientific literature on web accessibility. Nternational Journal of
Recent Technology and Engineering (IJRTE), 7(6), pp.250-258.
[9] Alajarmeh, N., 2022. Evaluating the accessibility of public
health websites: an exploratory cross-country study. Universal
access in the information society, 21(3), pp.771-789.
[10] Rubano, V., Vitali, F. and Lengua, C., 2022, June. Tools for
an Innovative Approach to Web Accessibility. In International
Conference on Human-Computer Interaction (pp. 97-115). Cham:
Springer International Publishing.
[11] Doğan, S., Betin-Can, A. and Garousi, V., 2014. Web
application testing: A systematic literature review. Journal of
Systems and Software, 91, pp.174-201.
[12] Paiva, D.M.B., Freire, A.P. and de Mattos Fortes, R.P., 2021.
Accessibility and software engineering processes: A systematic
literature review. Journal of Systems and Software, 171, p.110819.
[13] Campoverde-Molina, M., Lujan-Mora, S. and Garcia, L.V.,
2020. Empirical studies on web accessibility of educational
websites: A systematic literature review. IEEE Access, 8,
pp.91676-91700.
[14] Ara, J., Sik-Lanyi, C. and Kelemen, A., 2023. Accessibility
engineering in web evaluation process: a systematic literature
review. Universal Access in the Information Society, pp.1-34.
[15] Ara, J., Sik-Lanyi, C. and Kelemen, A., 2023. Accessibility
engineering in web evaluation process: a systematic literature
review. Universal Access in the Information Society, pp.1-34.
[16] Cisneros, D., Huamán Monzón, F. and Paz, F., 2021.
Accessibility evaluation of E-Government web applications: A
systematic review. In International Conference on Human-
Computer Interaction (pp. 210-223). Springer, Cham.
[17] MACAKOĞLU, Ş.S. and Peker, S., 2022, June. Web
accessibility performance analysis using web content accessibility
guidelines and automated tools: a systematic literature review. In
2022 International Congress on Human-Computer Interaction,
Optimization and Robotic Applications (HORA) (pp. 1-8). IEEE.
[18] Campoverde-Molina, M., Luján-Mora, S. and Valverde, L.,
2023. Accessibility of university websites worldwide: a systematic
literature review. Universal Access in the Information Society,
22(1), pp.133-168.
[19] Dias, J., Carvalho, D., Paredes, H., Martins, P., Rocha, T. and
Barroso, J., 2021, December. Automated Evaluation Tools for
Web and Mobile Accessibility: A Systematic Literature Review. In
International Conference on Innovations in Bio-Inspired
Computing and Applications (pp. 447-456). Cham: Springer
International Publishing.
[20] Akram, M. and Sulaiman, R.B., 2017. A systematic literature
review to determine the web accessibility issues in Saudi Arabian
university and government websites for disable people.
International Journal of Advanced Computer Science and
Applications, 8(6).