We use automated scanners on every engagement. They're quick, they don't get tired, and they catch a lot of real problems. So this isn't an article about how tools are bad. It's about what a clean report actually tells you — and what it can't.
Start with the scale of the problem
Every year, WebAIM runs automated tests on the home pages of the top one million websites. In the 2026 report, 95.9% of those pages had detectable WCAG failures, averaging 56 errors per page. That's up from 94.8% and 51 errors a year earlier, reversing six years of slow improvement.
Two things are true at once here. Automated tools are clearly good at finding problems — they found 56 million of them. And those numbers are a floor, not a ceiling, because they only count what a machine can detect.
How much do scanners actually find?
It depends on how you count, which is why you'll see very different numbers quoted.
In 2017, the UK Government Digital Service built a page with 143 deliberate accessibility barriers and ran ten automated tools against it. The best single tool found around 37–41% of them, and the weakest found 17%. Even with all ten tools combined, 29% of the barriers went completely undetected.
Deque, which makes the axe engine, published a study in 2021 of more than 2,000 of its own audits and found that automated testing caught 57% of issues by volume. That's a vendor studying its own tool, so read it with that in mind — but it isn't really in conflict with the GDS result. Counting by volume favours the problems that appear hundreds of times on a page, like low-contrast text, and those happen to be the ones machines are best at. Counting distinct types of barrier, as GDS did, gives a lower number.
What scanners are good at
- Colour contrast between text and its background
- Images with no alt attribute at all
- Form fields with no programmatic label
- Missing page language and page titles
- Empty links and buttons
- Broken ARIA — invalid roles, or attributes that reference IDs that don't exist
These are real problems, and catching them automatically in your build pipeline is one of the cheapest accessibility wins available. If you're not doing it yet, start.
What they can't judge
The pattern is simple: a tool can check that something exists. It can't check that it makes sense.
- Alt text quality. A scanner sees alt="image1.jpg" and passes it. A person knows that's useless.
- Whether focus order is logical. Tools can confirm an element is focusable, not that focus moves in an order that matches the page.
- Keyboard traps in custom widgets. Date pickers, carousels, and modals are where keyboard users get stuck, and scanners rarely try operating them.
- Whether updates are announced. When an error appears or a cart total changes, does a screen reader user hear about it? No static scan can tell.
- Whether error messages help. "Invalid input" technically identifies an error; it doesn't tell anyone how to fix it.
- Heading structure that reflects meaning. A tool can confirm headings exist, not that they describe the content beneath them.
- Captions and transcripts. A tool can detect a caption track. It can't tell whether the captions are accurate or in sync.
- Whether the whole journey works. Every page can pass and the checkout can still be impossible to complete with a screen reader.
A word on "overlay" widgets
Some products promise to make a site accessible with a single line of JavaScript. They run into the same limits as scanners — they can't reliably understand what your content means — and they can interfere with the assistive technology people already use. Many disabled users actively avoid them. Fixing the underlying code is slower, but it's the only thing that actually works.
What a sensible testing mix looks like
- Automated checks in your CI pipeline, so the obvious problems never reach production.
- A manual expert review against WCAG 2.2 AA, using a keyboard, screen readers, and zoom.
- Usability testing with disabled people on the journeys that matter most.
- Regular retesting, because every release can introduce new barriers.
A green score from a scanner is a good sign. It just isn't the finish line. The only way to know your product works for disabled people is to check — with the tools, with expertise, and with the people themselves.