Be Wary of Accessibility Guarantees from Anyone

TL;DR: anyone promising you that a total solution to digital accessibility is coming, and they are the ones bringing it, may be lying.

Background

In 2016 I wrote Be Wary of Accessibility Guarantees from Vendors. At the time I was cautioning readers about libraries and frameworks and SaaS and so on.

For the most part, these are driven by teams who do not spend their days in the weeds as accessibility practitioners and, in most cases, have no connection to disability among their management or teams.

Then an overlay company framing itself as run by experts and practitioners, but having nothing more than venture funding at its back, outlined its bold vision (still on its site today in the metadata):

Our vision is to make the entire internet fully accessible to people with disabilities by 2025. By using automation rather than labor, the notion of a fully accessible internet (hundreds of millions of active websites and over 1.5 Billion in total) is for the first time ever a practical, attainable reality and not just a distant dream.

It is now 2025 and all evidence suggests that vision hasn’t worked out and was based on lies.

Foreground

Within the last week I watched a pitch from a digital accessibility consultancy that had this to say about its testing platform:

Well, today I am telling you that I and everyone at [redacted] is committed to achieving 100% automation or so close that the difference won’t matter for digital accessibility testing in the very near future.

Accompanying this was a slide titled The Journey to 100% Automated Testing that showed a timeline: 2000 at 0%, 2024 at 57%, and 2025 at 100%.

So that’s two companies telling us either testing or remediation will be solved by 2025 (this year).

And Yet

I know all the arguments for automation — computer vision can describe the images, LLMs can generate better code, this is the worst genAI will ever be, things will only improve, and so on.

I also know I am not the audience for the claims that all will be solved by 2025. Organizations looking for an easy fix are the audience. Organizations that are already on the “AI” hype train. Organizations that may want to do their best but bump up against real challenges at scale.

I am wary for reasons I have outlined in the past and which, so far, have not been proven wrong.

• WCAG itself can only be tested with automation at around the 30% level, with the remainder requiring understanding of context and user needs.
• The European Accessibility Act, clearly used as marketing pressure, goes beyond WCAG in parts.
• In my own comparison of automated tools, the best ones found under 15% of WCAG violations in a sample.
• The math has always been sketchy on many claims buttressing the more grandiose ones.
• Automation will always lack context and intent, which is critical to solutions that work best for people.
• Prior efforts have demonstrated garbage in / garbage out is a reality when training any automation.
• The false claims tied to many promises have resulted in lawsuits and user harm.
• Some proponents continue to move the goalposts when their visions do not come to pass.
• All of this assumes the resources can keep up with the demand.

Anecdata

Recently someone asked how to test a web page coded in canvas. The <canvas> element has no DOM. It exists primarily as an API for animation, scripting, and video. Think of it like Flash, but without the marketing.

This person had one tool in their toolbox, and it was an automated tool that inspects the DOM. Which canvas does not have. The automation couldn’t help.

Never mind all the poorly-tagged PDFs spit out of editors claiming to make accessible PDFs. Never mind native apps that even platform-provided testing tools cannot fully test. And so on.

These are cases where a tester needs to suit up, put on the assistive technologies, and climb into the experience to properly evaluate it. Instead, some vendors want to eat our seed corn.

Pandering

I feel like when anybody makes a claim that they can guarantee conformance with all of WCAG or some other maximum accessibility target, it pushes the rest of the industry down. They enter a race to the marketing bottom as companies continue to trot out messages grounded in what they want their prospects to hear, not reality of the challenges in helping people.

It’s performative and it’s harmful. Companies might as well award themselves perfect scores on some opaque and arbitrary scale within their own industry just to claim they are making things better.

Wrap-up

Anyone promising you that a total solution to digital accessibility is coming, and they are the ones bringing it, may be lying.

Update: 4 March 2025

I do not name the vendor whose messaging I cite in the Foreground section above. As a professional courtesy, I didn’t want this post to be associated with the vendor’s name when someone searches.

That said, there were some other more direct criticisms of that message that I think worth sharing:

• Values by Eric Eggert, 2 March 2025
• Karl Groves on LinkedIn (for the conversation), 28 February 2025
• Eyal Eshet on LinkedIn (for replies from the vendor), 28 February 2025

Unfortunately, you have to log into LinkedIn to see the full conversations, because LinkedIn is more interested in monetizing and tracking you than letting you read stuff.

Update: 6 March 2025

Deque has made the video private. Deque has also wiped it from its axe-con schedule. Deque has removed the machine-generated video transcript. Deque has removed the Masto post for the video. I have seen no mention from Deque on social media (including in the LinkedIn conversations where Deque’s CEO took part and welcomed the debate) that the video was being removed nor on its site.

While I did not archive the video, I did pull down the slides and transcript. The PDF slides are 1.9MB, so maybe don’t view them in the embed that follows.

And the machine-generated transcript (which you can grab as a plain text file):

I’m not a fan of revisionist history. I’ve spent too long fighting with overlay companies and their casual relationship with the truth to accept it when someone disappears their past statements.

The purge of the YouTube video also lost some of the real-time feedback people offered in the chat:

During the follow-up conversations on LinkedIn, Dylan (from Deque) made it a point to share his talk, The Future of the axe Platform, which he says goes into much more detail with much more nuance and offers a much more practical view of where we are right now.

Anyway, in light of all this, if you see Deque at CSUNATC this year (its folks are attending as opposed to using axe-con to directly compete), maybe you can ask them what happened.

Update: 7 March 2025

Late this afternoon, in the walled garden of the Deque Discord (an oddly apt name this last week), a Deque representative acknowledged removal of the video.

There was a bit more drama during the event (last week) where Deque deleted a critical post from one person and blamed it on a new moderator. This was on top of repeated Deque leadership statements that they want to engage (without actually engaging), requests to watch yet more Deque axe-con videos as explanation (or justification or revisionism), and generally not winning people over.

I am not a member of the Discord (because I don’t give my expertise away for free to companies who can pay for it), but if you are then head over to the #ai-a11y channel and scroll up to 25 February.

Update: 18 March 2025

While at CSUNATC I found out there was a post from Deque’s CEO that I believed would address all the stuff here. I read what I thought might be it, but confirmation on LinkedIn tells me the post is The journey toward a more accessible internet.

I am of the opinion it fails to address any of my concerns in its brief 673 words. It does not acknowledge…

• its axe-con talk;
• the claims in its axe-con talk;
• that it deleted its axe-con talk;
• that it failed to notify anyone it deleted its axe-con talk;
• that it failed to engage directly with the criticisms over its axe-con talk.

Instead it as an aspirational post meant for sales prospects, not practitioners. It puts the blame for the widening accessibility gap on automation and artificial intelligence (AI) but assures us automation and artificial intelligence (AI) can solve the problem it created (as stated in the graphic). Though here it also pitches its own team and how it will arm them with automation and artificial intelligence (AI).

The only quote of merit in the post:

To do this, automation is essential. Not to remove humans from the work, but to enable all humans to do their best work. Advanced automation does not replace accessibility expertise; it amplifies and scales it.

That is a harmless, common missive. It’s also a bit at odds with what I heard from an attendee of one of the CSUNATC Deque talks who walked away with the message that their job as a tester would be eliminated by axe. They were genuinely angry. I didn’t attend that talk and the slides don’t appear to be online, so I can’t even tell you what the exact message was. But if one person felt that way, then others likely did too.

I also did not attend the CSUNATC session for the revised version of the talk this post discusses. I already heard the message Deque chose to convey. I didn’t need to see it again nor its revisionist history variant. I certainly didn’t have time to burn doing so.

Finally, I think the alt text on the only graphic in that Deque post fails SC 1.1.1, but you may disagree.

Update: 1 April 2025

The accessibility team at Orange had a little April Fools fun with this in the post Tota11yMouse.

I appreciate the parody, which is clear to practitioners by the second paragraph:

In the field of accessibility, the expert community is formal: AI is going to revolutionize our businesses, and websites will, without a doubt, be 100% accessible within 2 or 3 years (see links at the end of the article).

All three links point to posts critical of Deque’s claims, including this post. There is a French version of the Tota11yMouse page as well.