Complete Guide to Automation Accessibility Testing

Introduction: WhyRelying onTools
Alone Isn’t Enough
As digital accessibility becomes increasingly important, manyteams
incorporate automation tools to check iftheirwebsite or application is
accessible.
To meet this growing need efficiently, many QAteams and developers
turn to automation accessibilitytesting tools. These tools promise to
speed up testing, catch compliance issues early, and integrate
seamlessly into CI/CD pipelines. Their power lies in their abilityto
quickly scan large codebases and identify common violations of
standards like WCAG.
CompleteGuidetoAutomation
AccessibilityTesting

Ifyou haven’t already, check out our previous blog – Everything
You Needto KnowAboutAccessibilityTesting – where we debunk
common misconceptions and explore why accessibility is much
broaderthan screen readers and visual impairments.
In this blog, we will examine what automation accessibilitytesting
tools promise, what they provide based on real-world testing, and
why human judgment will remain a key component ofthe process. We
will also look at the most common tools today – what they do well,
where theyfail, and howto best use them in your QA process.
Table ofContent
Introduction: Why Relying on Tools Alone Isn’t Enough
PopularAutomation AccessibilityTesting Tools
Axe by Deque Systems
Steps to Run the Tests
Test Results:
Google Lighthouse
Steps to Run the Lighthouse AccessibilityTest:
Test Results:
WAVE byWebAIM
Steps to Run the WAVE AccessibilityTest
Test Results:
Pa11y
Steps to Run Pa11y
Prerequisites:
Test Results (Pa11y Output)
Accessibility Insights by Microsoft
Steps to Run Accessibility Insights Test
Test Results:
What Do These Tools Promise?
Best Practices: Using Automation the Right Way

Use Automation Tools for Speed and Coverage
Rely on Manual Testing for Real User Scenarios
Educate Your Entire Team
Don’t Rely on “Green Reports” as a Final Verdict
Automation AccessibilityTools: What They Catch vs. What
They Miss
Why Manual AccessibilityTesting Still Matters
Determine IfAlt Text Is Meaningful
Assess Logical Focus Order and Navigation Flow
Understand Complex Interactions
Simulate Real Screen Reader Experiences
Sense Confusion, Delight, or Frustration
Conclusion: WhyAutomation Tools Can’t Replace Manual
AccessibilityTesting
PopularAutomationAccessibility
TestingTools
Let’s break down the most widely used accessibilitytesting tools,
their pros and cons, and where they stand in real-world usage.
Axe byDeque Systems
What It Is:
One ofthe most respected and widely used accessibilitytools,
available as a
browser extension, CLI tool, or integration forframeworks like
Selenium, Cypress, and Playwright.
Promises:
Accurate WCAG checks
Seamless CI/CD integration

Lowfalse positives
Enterprise-level dashboarding with Axe Monitor
What It Delivers:
Excellent for spotting basic WCAG 2.1 violations
Great developer experience
Doesn’t check alt text meaning orvisual clarity
Needs manual testing for logical flow and interaction
Use Case:
Use it during development and CI builds. Pair it with manual
screen readertesting for best results.
Example (BrowserExtension):
To demonstrate automation accessibilitytesting, we used Axe
DevTools by Deque Systems on the SauceDemo Login Page.
Below is a step-by-step guide with real results and screenshots.
Stepsto RuntheTests
Step 1:InstallAxe DevTools
Install the Axe DevTools Chrome extension.
Step 2: OpentheWebsite
Navigate to the target site – in our case:
https://www.saucedemo.com/v1/
Step 3: Open DevTools
Right-click on the page → Click Inspect or press Ctrl + Shift
+ I.
Step 4: Switchtothe “axe DevTools”tab
You’ll find a newAxe DevTools tab inside Chrome DevTools.
Step 5: Click“Analyze.”
Axe will begin scanning the page and show a list of

accessibility issues it detects.
Test Results:
Here are the actual findings from the SauceDemo login screen using
Axe DevTools:
Issues Detected (Total: 3)
1. The <html> element must have a lang attribute
2. Images must have alternative text (missing alt attribute)
3. Zooming and scaling must not be disabled
Screenshot:
KeyTakeaway:
Even polished-looking websites can miss critical accessibility
attributes. Automationtools like Axe catch these early in
development, but for a complete accessibility audit, always
complement automation with manual testing (screen readers,
keyboard navigation, etc.).

Google Lighthouse
What It Is:
Afree tool built into Chrome DevTools. Runs accessibility audits
as part of performance and SEO checks.
Promises:
Simple, quick accessibility score
Checklist of key issues
No installations needed
What It Delivers:
Quick overviewwith accessibility score
Detects low contrast, missing labels, and bad button usage
Score can be misleading – passing doesn’t mean “fully
accessible”
Limited rule set compared to Axe
Use Case:
Great for a fast audit orto report improvements. Not reliable for
deep testing.
To demonstrate automation accessibilitytesting, we used
Lighthouse on the SauceDemo Login Page.
Below are the step-by-step guide with real results and screenshots.
Stepsto Runthe LighthouseAccessibilityTest:

Go to https://www.saucedemo.com/v1/ in Chrome.
Step 2: Open Chrome DevTools
Right-click anywhere on the page → Select Inspect or press
Ctrl + Shift + I.
Step 3: Gotothe “Lighthouse”tab
Click on the “Lighthouse” tab inside Chrome DevTools.
Step 4: ConfiguretheAudit
Uncheck all categories except Accessibility
Choose the mode: either Mobile or Desktop
Click the Analyze page load
Step 5:Viewthe Report
After a few seconds, Lighthouse will generate a detailed
accessibility report with a score out of 100 and a list of
issues.
Test Results:
Accessibility Score: 82 / 100
Common issues detected byLighthouse:
1. Missing alt attributes for meaningful images
2. Buttons without accessible names
3. Insufficient contrast between background and foreground text
4. Form elements not associated with labels
ScreenShot

WAVE byWebAIM
What It Is:
Avisual accessibility evaluation tool. Available as a browser
extension and API.
Promises:
Clearvisual feedback on accessibility error
Educational tool for learning WCAG issues
API for developers and QAteams
What It Delivers:
Great for quick visual checks
Highlights structure and ARIA landmarks
Doesn’t scale well for large apps or CI/CD
Can clutter pages and sometimes false-flag
Use Case:
Ideal for learning and quick audits on smaller pages.
Install theWAVE extension.
Visit a webpage and click the WAVE icon.
It overlays icons to show errors like missing alt text or bad ARIA
roles.
Example:

To demonstrate manual visual accessibility inspection, we used
the WAVE browser extension on the SauceDemo Login Page.
Below is a step-by-step guide along with real test results and
optional screenshot areas.
Stepsto RuntheWAVEAccessibilityTest
Step 1: InstalltheWAVE Extension
Go to the Chrome Web Store and install the WAVE Evaluation
Tool.
Navigate to https://www.saucedemo.com/v1/ in your
browser.
Step 3: LaunchWAVE
Click the WAVE extension icon in your browsertoolbar.
WAVE will analyze the page and overlayvisual indicators
directly on the UI.
Step 4: Reviewthe Results
Look for icons on the page:
Red icons: Accessibility errors
Yellow icons: Warnings (potential issues)
Green icons: Structural elements (like ARIA
landmarks or heading tags)
Step 5: Usethe SidebarforDetails
The WAVE sidebar provides a categorized list of:
Errors
Alerts
Features
Structural elements
Each issue includes:
Description
WCAG reference
HTML snippet

Test Results:
Category Count Description
Errors 4
– 1× Missing alternative
text (on image) – 2×
Missingform label
(Username & Password
fields) – 1× Language
missing orinvalid
(missing lang attribute
in <html>)
Alerts 2
– 1× No page regions
(missing semantic
landmarks like <main>,
<nav>, etc.) – 1× Missing
first level heading (<h1>
tag notfound)
Structural Elements 2
– 2× Heading level 4
(<h4> used, but no <h1>
to <h3> levels)
Screenshots:

Pa11y
What It Is:
An open-source command-line tool for running accessibility
reports using the Axe engine.
Promises:
Fast automation tests
CLI-friendlyfor DevOps pipelines
Headless browsertesting
What It Delivers:
Easyto set up in CI/CD
Covers standard WCAG checks
Lacks advanced dashboards
Misses interaction-level issues
Use Case:
Use in automation test suites to catch regression issues.
Example (Command Line):

npx pa11y https://example.com
Output:
Shows a terminal-friendly list of issues like contrast errors,
missing labels, etc. Great for CI.
CI/CD Example:
Add the command to your pipeline YAMLto fail builds on
accessibility errors.
Example:
Tool Used: Pa11y – Command-line accessibilitytesting tool
WebsiteTested: https://www.saucedemo.com/v1/
Execution Mode: Headless (CLI)
Stepsto Run Pa11y
Prerequisites:
Node.js installed
Pa11y installed globallyvia npm: npm install -g pa11y
RuntheTest:
pa11yhttps://www.saucedemo.com/v1/
Test Results (Pa11yOutput)
IssueType Count Description
– Missing alt attribute on

Error 4
the mascot image – No
<label> on
username/password
fields – Missing lang
attribute on <html>
Warning 1
– Improper heading level
usage (skipping heading
levels directlyto <h4>)
Screenshot:
AccessibilityInsights byMicrosoft
What It Is:
A browser extension and Windows tool that tests against WCAG
using automation and guided manual checks.
Promises:
Rich manual testing experience
Guidance forWCAG compliance
Integration with Azure Boards
What It Delivers:

Well-structured guided tests
Focuses on completeness, not speed
Slight learning curve
Not ideal for large-scale regression
Use Case:
Perfect for exploratorytesting with guidance.
Install AccessibilityInsights.
Open your site → Launch the tool from the toolbar.
Use the FastPass or Assessment mode.
Get a guided walkthrough for manual checks (keyboard,
landmarks, etc.).
Example:
Tool Used: AccessibilityInsightsforWeb – a Chrome/Edge
extension developed by Microsoft
WebsiteTested: https://www.saucedemo.com/v1/
Execution Mode: Manual and Automation (both available)
Stepsto RunAccessibilityInsightsTest
Step 1: Installthe Extension
Download the AccessibilityInsightsforWeb extension
from the Chrome Web Store orthe Microsoft Edge Add-ons
store.
Step 2: OpentheTargetWebsite
Navigate to https://www.saucedemo.com/v1/ in Chrome
or Edge.
Step 3: LaunchAccessibilityInsights
Click the AccessibilityInsights icon in the browsertoolbar

→ Select “FastPass” (automation test) or “Assessment”
(detailed manual test).
Step 4: Run FastPass
FastPass includes:
Automation checks (using axe-core)
Tab stopsvisualization
Click “Start” to run the test.
Test Results:
Issue Category Count Example
Automation Check
Failures
3
– Image missing alt
attribute – Inputs
missing <label> – Missing
lang on <html>
Tab StopsVisualization 0
– Shows tab order is
present, but inputs lack
labels, affecting screen
reader usability
Screenshot:

Get ProfessionalAccessibilityQA
Support
Need help implementing these accessibilitytools effectively? Ensure your
website is fully accessible and compliant with expert guidance.
What DoTheseTools Promise?
Automation accessibilitytesting tools are often marketed as
powerful, all-in-one solutions for achieving digital accessibility
compliance. Their key promises include:
Automation Detection ofWCAGViolations
Most tools claim they can automatically detect violations against the
WCAG (Web Content Accessibility Guidelines) as applied to your entire
website or application. They often promote near-instant coverage
across WCAG 2.1 levels A, AA, and even AAA, suggesting
comprehensive issue identification with minimal manual intervention.
QuickIdentification ofAllAccessibilityIssues
One ofthe strongest selling points is speed – tools often claim to
detect all major accessibility issues within seconds. With just a click
or a build trigger, they promise to uncover everything from missing alt
text to improper heading structures and color contrast failures.
Elimination ofManualTesting
Some tools suggest that an automated scan is sufficient to meet your
compliance standards. Therefore, some tools will advocate reduced or
even no need for manual accessibilitytesting performed by experts or
Explore OurServices

users with disabilities.
Seamless Integrationwith CI/CD Pipelines
Modern accessibilitytools are often built into the CI/CD (Continuous
Integration/Continuous Deployment) process. The marketing often
touts that the automation tests can be run effortlessly everytime
there is a commit or build, so it can be flagged earlier in the
development cycle, and keep accessibility at the front of developers’
minds going forward.
ButWhat’sthe Reality?
Most automation tools get 30%-50% ofthe actual accessibility
issues identified.
They are excellent at identifying code-levelviolations (like
missing ARIAattributes or improperform labeling).
However, they cannot assess visual context, userintent, or
interaction-based issues, like whether a modal trap exists, if
link text is meaningful, or if a screen reader user can navigate
intuitively.
Automation tools also struggle with dynamic content, custom
components, and JavaScript-heavy UIs where accessibility is
related to behavior and not just markup.
Best Practices: UsingAutomationthe
RightWay
Automation is powerful, but when used in the right context. Here’s
howto blend automation tools with human insight for maximum
coverage and usability.

UseAutomationToolsforSpeed and
Coverage
Quickly catch common WCAG violations (e.g., missing labels,
color contrast)
Ideal for earlydevelopment and CI pipelines
Helps prevent regressions by catching known issues
consistently
Relyon ManualTestingforReal User
Scenarios
Validate keyboard navigation,focus order, and screen reader
flow
Confirm meaningful alttext, headings, and context
Detect subtle issues like confusingforms, popups, orlive
content changes
EducateYourEntireTeam
Accessibility is not just a QAtask – it involves developers,
designers, and product owners
Encourage using accessible components and semantic HTML
from the start
Share insights from manualtesting in retrospectives and
grooming sessions
Don’t Relyon “Green Reports” as a
FinalVerdict
A passing score from a tool doesn’t meanthe experience is
accessible
Tools don’t understand intent, empathy, ortaskcompletion

Always follow up with a real human evaluation
CombineTools + ManualTesting Strategically
Project Stage Whatto Use WhyItWorks
Development
axe DevTools, Google
Lighthouse
Fast, inline checksto
catch issues early
QATesting
axe CLI, Pa11y,
AccessibilityInsights
(manual)
Combines automation
with guided manual
testingforfull
coverage
UAT/ FinalTesting
NVDA, JAWS,
VoiceOver, keyboard-
onlynavigation
Real-worldtestingto
verifyusabilityfor
assistivetech users
AutomationAccessibilityTools:What
TheyCatchvs.WhatTheyMiss
WhatToolsTypicallyDetectWell
WhatTools CommonlyOverlookor
Misjudge
Missing alt attributes on <img> tags
Inadequate ormisleading alttext,
like “image123” or irrelevant
descriptions
Incorrect semantic elements (e.g.,
using <div> instead of <button>)
Ambiguous link/buttontext, like
“Click here” or “Read more,” without
context
Insufficient colorcontrast ratios
Real-world contrast issues, such as
glare or color conflicts, under different
lighting conditions
Missing oremptyform labels
Unclearorvague labels, such as
“Field” or unlabeled form sections that
confuse screen reader users

Keyboardtraps orlackofkeyboard
access
Logicaltab order, meaningful
grouping, or presence and usefulness
of “Skipto content” links
Misuse ofARIAroles orattributes
UnnecessaryorredundantARIA
usage, or lack of awareness that native
HTML might suffice
Incorrect ormissing landmarkroles
True navigability for screen reader
users – i.e., how intuitive and effective
the landmarks feel in use
Non-descriptive button labels, like
<button></button>
Contextual relevance, e.g., whether a
button’s label explains what action it
performs
Presence offocus indicators
Visibilityand clarity offocus
indicators in real usage scenarios
Use ofheadings out oforder(e.g., h3
afterh1)
Document structure clarity, such as
whether headings provide a
meaningful content hierarchy
WhyManualAccessibilityTesting Still
Matters
While automation checkers like Axe, Lighthouse, and WAVE are great
for identifying the easy and cursory accessibility issues, they only
address part ofthe issue. Here’s what they cannot do – this is why
human judgment is imperative and irreplaceable:
Determine IfAltText Is Meaningful
Automation tools can identify missing alt attributes, but they cannot
identify ifthe description conveys useful content.
Forinstance:
“Portrait of Dr. Jane Smith, Nobel Laureate” → Useful

“image123.jpg” or picture → Useless
A human being is the only one who can identify ifthe image was
meaningful content or decorative, and whetherthe alt text achieves
that purpose.
Assess Logical Focus Orderand
Navigation Flow
Tools can confirm keyboard access, but they can’t assess whetherthe
tab order makes sense.
Humanstest:
Is it possible to navigate from search to results without jumping
around the page?
Is there a visible focus indicatorthat you can see?
Logical flow is critical for users relying on a keyboard or assistive
tech, and only manual testing can fullyvalidate it.]
Understand Complex Interactions
Components like modals, dynamic dropdowns, sliders, and popovers
can behave unpredictably:
Does focus stay inside the modal while it’s open?
Can you close it using onlythe keyboard?
Is ARIA used appropriately, or is it misapplied?
Automation tools may not fully grasp interaction nuances, especially
when JavaScript behaviors are involved.
Simulate Real Screen Reader

Experiences
Automation tools don’t use screen readers – humans do.
Manual testing helps answer:
Can a screen reader user complete atask?
Is the narration logical, or is it confusing?
Are landmarks and headings trulyhelpful?
Only a manual test can confirm ifthe experience is navigable,
informative, andtask-completable.
Sense Confusion, Delight, or
Frustration
Accessibility is not just about compliance – it’s about usabilityand
dignity.
Automation tools can’t measure:
Is the experience pleasant orfrustrating?
Are error messages clearandtimely?
Can someone complete the task without askingforhelp?
These insights come onlyfrom real human experience – whether it’s a
tester, a userwith a disability, or a QA professional.
Manual testing adds empathy and human experience to the process –
something no tool can automate.
Conclusion:WhyAutomationTools
Can’t Replace ManualAccessibility
Testing
Automation accessibilitytools are invaluable allies. They speed up

audits, catch repetitive code issues, and fit seamlessly into modern
DevOps pipelines. They help you do more, faster – but they can’t do it
all.
No matter how advanced a tool claims to be, it can’t judge meaning,
can’t feel frustration, and can’t guarantee usabilityfor someone
navigating your app with a screen reader or only a keyboard.
True accessibility demands a human perspective.
It requires manual testing, empathy, and sometimes even direct
feedback from people with disabilities. That’s howyou uncover real
barriers and craft experiences that are not just compliant, but
welcoming and usable.
Remember:
Automation helpsyou checkthe rules.
Manualtesting helpsyou servethe people.
Witness howourmeticulous approach and cutting-edge
solutions elevated qualityand performanceto newheights.
Beginyourjourneyintotheworld ofsoftwaretesting excellence.
To knowmore referto Tools &Technologies & QAServices.
Ifyouwould liketo learn more aboutthe awesome serviceswe
provide, be sureto reach out.
HappyTesting 🙂
TAGS: Originally Published By :- Jignect Technologies

Complete Guide to Automation Accessibility Testing

More Related Content

Similar to Complete Guide to Automation Accessibility Testing

More from digitaljignect

Recently uploaded

Complete Guide to Automation Accessibility Testing