Skip to content
Journal

Engineering

Accessibility Audits That Actually Stick

Headshot of Grace Nakamura
Grace Nakamura
August 22, 2023 · 3 min read

We have been on both sides of bad accessibility audits. We have received them: a 90-page PDF of automated scanner output, delivered with an invoice and no follow-up. And early on, we probably delivered a version of one. The pattern is always the same. The report lands, a sprint of fixes happens, everyone feels responsible, and eight months later the product has regressed to roughly where it started because nothing about how the team builds actually changed.

This year we rebuilt our audit practice around a simple observation: an accessibility audit is a change-management project wearing a technical costume. The findings are the easy part. Making them permanent is the work.

Automated scanning is the floor, not the audit

Tools like axe catch maybe a third of real issues, and they catch the mechanical ones: missing alt attributes, contrast failures, unlabeled inputs. Necessary, nowhere near sufficient. The failures that actually lock people out live above that layer: focus order that jumps across the page after a modal closes, custom dropdowns that eat arrow keys, error messages announced to nobody, animation that cannot be paused.

So our audits are task-based. We pick the five journeys that matter commercially, then run each one three ways: keyboard only, screen reader (VoiceOver and NVDA), and 200 percent zoom. On a recent commerce audit for Northwind, the automated scan of the checkout produced eleven findings. The task-based pass produced the one that mattered: the promo code field's validation error was visually adjacent but programmatically orphaned, so screen reader users heard nothing, retried, and abandoned. No scanner will ever find you that, and that single fix was worth the engagement.

Fixes belong in the system, not the page

The biggest structural change we made: findings are triaged by where the fix belongs, not by severity alone. A contrast failure on one banner is a page fix. The same failure appearing forty times is a token fix. An inaccessible dropdown is a component fix, made once in the design system and inherited everywhere.

We deliver every audit finding tagged one of three ways:

  • Token or system level: fix once in the design system, audit the diff, done everywhere.
  • Component level: fix in the shared library, add a regression test.
  • Content level: fix in the page, add to the editorial checklist.

Systems-level fixes are the highest-leverage work in the entire engagement, and in a traditional severity-sorted report they get buried under a hundred instance-level duplicates.

Making it stick

The last third of our audit engagements now produces no findings at all. It produces guardrails: axe checks wired into CI so known-fixed issues cannot silently return, a keyboard pass added to the definition of done for any new component, and one working session where the client's engineers fix real findings with us rather than receiving them. Engineers who have fixed a focus trap once, with a screen reader talking at them, write different code forever after.

# CI: fail the build on regressions in audited flows
- name: axe smoke test
  run: pnpm exec playwright test a11y/ --reporter=line

The honest sales pitch we give clients is this: a report costs less and does less. If the audit does not change your tokens, your component library, and your definition of done, you have not bought accessibility. You have bought a snapshot of your inaccessibility, suitable for framing. With European accessibility requirements tightening and US litigation volume still climbing, snapshots are getting expensive. Systems are cheaper. They always were.

Building something this could apply to?

We take on a small number of flagship projects each quarter.

Start a project