Web Accessibility Testing Methodology: A Systematic Approach to Inclusive QA

Accessibility testing is not a single action — it is a layered methodology combining automated scanning, manual keyboard testing, screen reader evaluation, and user testing. No single tool or method catches every accessibility barrier. Automated tools find approximately 30-40% of WCAG issues (Deque University’s research consistently shows this range). The remaining 60-70% require human judgment — understanding context, evaluating content quality, and experiencing the interface as assistive technology users do.

The Four-Layer Testing Approach

Layer 1: Automated Scanning

What it catches: Missing alt text, insufficient color contrast, missing form labels, duplicate IDs, missing landmarks, invalid ARIA attributes, heading hierarchy violations.

What it misses: Alt text quality, logical reading order, keyboard trap detection, screen reader announcement quality, content comprehension, focus management logic.

Tools:

• axe DevTools (Deque): Browser extension and CI integration. The most widely used automated accessibility testing engine. Runs against the rendered DOM, catching dynamic content issues.
• WAVE (WebAIM): Browser extension with visual overlay showing errors, alerts, and structural elements directly on the page.
• Lighthouse (Google): Built into Chrome DevTools. Runs axe-core under the hood with a simplified scoring interface.
• Pa11y: Command-line tool for CI pipeline integration. Supports HTML CodeSniffer and axe-core engines.
• IBM Equal Access Checker: Browser extension with unique rules beyond axe-core coverage.

Best practice: Run at least two automated tools, as their rule sets differ. A page passing axe may fail IBM’s checker and vice versa.

Layer 2: Keyboard Testing

Keyboard testing verifies that every interactive element is operable without a mouse. This catches issues automated tools cannot detect:

Keyboard Testing Checklist

1. Tab order: Press Tab repeatedly through the entire page. Does focus move in a logical sequence that matches the visual layout?
2. Focus visibility: Is there a visible focus indicator on every focusable element? Can you always tell where focus is?
3. Interactive elements: Can you activate every button, link, checkbox, radio button, and select with keyboard alone (Enter, Space, Arrow keys as appropriate)?
4. Custom widgets: Do tabs, modals, date pickers, accordions, and comboboxes follow their W3C keyboard interaction patterns?
5. Focus traps: When a modal or dialog is open, does focus stay within it? When it closes, does focus return to the trigger?
6. Skip links: Does pressing Tab once from the top of the page reveal a skip link? Does activating it move focus to the main content?
7. No keyboard traps: Can you Tab away from every element? Is there any component that captures focus without an escape mechanism?
8. Escape key: Do menus, dialogs, tooltips, and dropdowns close on Escape?

How to Keyboard Test

Unplug your mouse (or disable your trackpad). Navigate the complete user journey — from landing page through task completion — using only the keyboard. Time yourself. If you cannot complete the task, keyboard users cannot either.

Layer 3: Screen Reader Testing

Screen reader testing evaluates whether assistive technology can access and interpret all content and functionality. This is the most skill-intensive testing layer.

Screen Reader and Browser Combinations

Test with the combinations that represent the majority of assistive technology users:

Screen Reader	Browser	Platform	Market Share
NVDA	Chrome or Firefox	Windows	~40%
JAWS	Chrome or Edge	Windows	~30%
VoiceOver	Safari	macOS/iOS	~20%
TalkBack	Chrome	Android	~5%

At minimum, test with one Windows screen reader (NVDA is free) and VoiceOver on macOS/iOS.

Screen Reader Testing Checklist

1. Page title: Does the page title announce correctly? It should describe the page uniquely.
2. Landmarks: List all landmarks (NVDA: NVDA+F7 > Landmarks). Are all major page sections represented with descriptive labels?
3. Headings: List all headings (NVDA: NVDA+F7 > Headings). Does the heading hierarchy form a logical outline of the page?
4. Images: Navigate to each image. Is alt text present, accurate, and contextually appropriate? Are decorative images hidden?
5. Forms: Navigate to each form field. Is the label announced? Are required fields indicated? Do error messages announce when they appear?
6. Dynamic content: Trigger dynamic updates (notifications, loading states, live search results). Are changes announced through live regions?
7. Custom widgets: Interact with every custom widget. Are roles announced correctly? Are states (expanded, selected, pressed, checked) communicated?
8. Links: List all links (NVDA: NVDA+F7 > Links). Does each link have descriptive text that makes sense out of context?
9. Tables: Navigate data tables cell by cell. Are row and column headers announced contextually?

Learning Screen Readers

If you are new to screen reader testing, start with VoiceOver (built into macOS) or NVDA (free download for Windows):

• VoiceOver: Cmd+F5 to enable. Use VO keys (Ctrl+Option) with arrow keys to navigate.
• NVDA: Download from nvaccess.org. Use Insert key as the NVDA modifier. Tab navigates interactive elements; arrow keys navigate all content.

Practice with websites you know well before testing your own. Understanding normal screen reader behavior helps you identify abnormal behavior.

Layer 4: User Testing With People With Disabilities

The gold standard. Real users with disabilities reveal barriers that no tool or tester without disabilities will catch:

• Cognitive accessibility: Is the content understandable? Is the workflow intuitive? Are instructions clear?
• Motor accessibility: Can users with motor impairments interact with all elements in a reasonable time? Are touch targets adequate?
• Low vision: Does the interface work with screen magnification? Are zoom levels supported? Do color choices work for color blindness?
• Real assistive technology configurations: Users have customized settings, preferred tools, and workflow patterns that testing environments do not replicate.

Recruit users with diverse disabilities. A blind screen reader user finds different issues than a user with motor impairments who uses switch access. Both perspectives are essential.

Integration Into Development Workflow

Shift Left: Test Early

• Design phase: Check color contrast, touch target sizes, and information hierarchy in design mockups
• Component development: Run axe-core on each component in isolation (Storybook addon-a11y)
• Feature development: Keyboard test each feature before code review
• Code review: Check for semantic HTML, ARIA correctness, and label associations

CI/CD Pipeline

Integrate automated testing into your continuous integration pipeline:

# Example: axe-core in CI
- name: Accessibility audit
  run: |
    npx axe-cli https://staging.example.com --exit

Set thresholds: no new critical or serious issues allowed. Track moderate issues for resolution in subsequent sprints.

Regular Audits

Beyond continuous testing, conduct periodic comprehensive audits:

• Quarterly: Full automated scan plus keyboard test of critical user journeys
• Bi-annually: Screen reader testing of all major features
• Annually: Third-party audit or user testing with people with disabilities

WCAG Conformance Testing

For formal WCAG conformance assessment:

1. Define scope: Which pages and user flows are being assessed
2. Identify WCAG level: AA is the standard target for most organizations
3. Test against each Success Criterion: Walk through all applicable WCAG 2.2 Success Criteria (approximately 50 for Level AA)
4. Document findings: For each criterion, record pass/fail/not applicable with evidence
5. Prioritize remediation: Critical issues (keyboard traps, missing alt text, no focus indicators) before minor issues (suboptimal heading hierarchy)

Accessibility Testing Tools Reference

Tool	Type	Cost	Best For
axe DevTools	Browser extension	Free (basic)	Quick automated scans
axe DevTools Pro	Browser extension	Paid	Guided manual testing
WAVE	Browser extension	Free	Visual accessibility overlay
Lighthouse	Built into Chrome	Free	CI integration, scoring
Pa11y	Command line	Free	CI pipeline automation
ARC Toolkit	Browser extension	Free	Color contrast, ARIA
Colour Contrast Analyser	Desktop app	Free	Precise color contrast measurement
NVDA	Screen reader	Free	Windows screen reader testing
VoiceOver	Screen reader	Free (built-in)	macOS/iOS screen reader testing

Common Testing Mistakes

1. Relying solely on automated tools: They catch 30-40% of issues. The most impactful barriers (keyboard traps, poor focus management, incomprehensible screen reader experience) require human testing.
2. Testing only the homepage: Test complete user journeys — search, filter, add to cart, checkout, account creation, error recovery.
3. Testing only with one screen reader: NVDA, JAWS, and VoiceOver behave differently. A feature that works in NVDA may fail in VoiceOver.
4. Not testing dynamic content: Pages that load content asynchronously, display notifications, or update in real time need specific testing for those dynamic behaviors.
5. Testing in isolation: Test the full flow, not individual pages. Focus management issues often manifest at page transitions and step changes.

Building an Accessibility Testing Practice

Start small and expand:

1. Week 1: Install axe DevTools. Run it on your most-trafficked pages. Fix critical issues.
2. Month 1: Add keyboard testing to your development workflow. Unplug the mouse once per feature.
3. Month 3: Learn one screen reader. Test your critical user journeys monthly.
4. Month 6: Integrate automated testing into CI. Set a zero-new-critical-issues policy.
5. Year 1: Conduct a formal audit. Establish an accessibility testing rhythm with regular screen reader and user testing.

Key Takeaways

Effective accessibility testing combines four layers: automated scanning (30-40% coverage), keyboard testing (interaction barriers), screen reader testing (information access), and user testing (real-world experience). No single layer is sufficient alone. Integrate automated testing into CI, make keyboard testing part of development workflow, and schedule regular screen reader evaluations. Start with the basics — axe DevTools and a keyboard — and build toward comprehensive testing with multiple screen readers and real users with disabilities. Accessibility is not a checkbox; it is an ongoing quality practice.