Be Wary of Accessibility Guarantees from Anyone

TL;DR: anyone promising you that a total solution to digital accessibility is coming, and they are the ones bringing it, may be lying.

Background

In 2016 I wrote Be Wary of Accessibility Guarantees from Vendors. At the time I was cautioning readers about libraries and frameworks and SaaS and so on.

For the most part, these are driven by teams who do not spend their days in the weeds as accessibility practitioners and, in most cases, have no connection to disability among their management or teams.

Then an overlay company framing itself as run by experts and practitioners, but having nothing more than venture funding at its back, outlined its bold vision (still on its site today in the metadata):

Our vision is to make the entire internet fully accessible to people with disabilities by 2025. By using automation rather than labor, the notion of a fully accessible internet (hundreds of millions of active websites and over 1.5 Billion in total) is for the first time ever a practical, attainable reality and not just a distant dream.

It is now 2025 and all evidence suggests that vision hasn’t worked out and was based on lies.

Foreground

Within the last week I watched a pitch from a digital accessibility consultancy that had this to say about its testing platform:

Well, today I am telling you that I and everyone at [redacted] is committed to achieving 100% automation or so close that the difference won’t matter for digital accessibility testing in the very near future.

Accompanying this was a slide titled The Journey to 100% Automated Testing that showed a timeline: 2000 at 0%, 2024 at 57%, and 2025 at 100%.

So that’s two companies telling us either testing or remediation will be solved by 2025 (this year).

And Yet

I know all the arguments for automation — computer vision can describe the images, LLMs can generate better code, this is the worst genAI will ever be, things will only improve, and so on.

I also know I am not the audience for the claims that all will be solved by 2025. Organizations looking for an easy fix are the audience. Organizations that are already on the “AI” hype train. Organizations that may want to do their best but bump up against real challenges at scale.

I am wary for reasons I have outlined in the past and which, so far, have not been proven wrong.

• WCAG itself can only be tested with automation at around the 30% level, with the remainder requiring understanding of context and user needs.
• The European Accessibility Act, clearly used as marketing pressure, goes beyond WCAG in parts.
• In my own comparison of automated tools, the best ones found under 15% of WCAG violations in a sample.
• The math has always been sketchy on many claims buttressing the more grandiose ones.
• Automation will always lack context and intent, which is critical to solutions that work best for people.
• Prior efforts have demonstrated garbage in / garbage out is a reality when training any automation.
• The false claims tied to many promises have resulted in lawsuits and user harm.
• Some proponents continue to move the goalposts when their visions do not come to pass.
• All of this assumes the resources can keep up with the demand.

Anecdata

Recently someone asked how to test a web page coded in canvas. The <canvas> element has no DOM. It exists primarily as an API for animation, scripting, and video. Think of it like Flash, but without the marketing.

This person had one tool in their toolbox, and it was an automated tool that inspects the DOM. Which canvas does not have. The automation couldn’t help.

Never mind all the poorly-tagged PDFs spit out of editors claiming to make accessible PDFs. Never mind native apps that even platform-provided testing tools cannot fully test. And so on.

These are cases where a tester needs to suit up, put on the assistive technologies, and climb into the experience to properly evaluate it. Instead, some vendors want to eat our seed corn.

Pandering

I feel like when anybody makes a claim that they can guarantee conformance with all of WCAG or some other maximum accessibility target, it pushes the rest of the industry down. They enter a race to the marketing bottom as companies continue to trot out messages grounded in what they want their prospects to hear, not reality of the challenges in helping people.

It’s performative and it’s harmful. Companies might as well award themselves perfect scores on some opaque and arbitrary scale within their own industry just to claim they are making things better.

Wrap-up

Anyone promising you that a total solution to digital accessibility is coming, and they are the ones bringing it, may be lying.