No, APG’s Support Charts Are Not ‘Can I Use’ for ARIA

TL;DR: Despite claims, APG’s support charts are not ARIA support charts and they are not analogous to Can I Use. While brand new and likely to grow, their scope is still strictly APG patterns.

Why I Say This

I have opinions on the ARIA Authoring Practices Guide (APG).

I am critical of it in my Uncanny A11y post. I have campaigned against its use of menus for navigation. And its preference for grids over tables, including ARIA layout grids. I was frowny when, after being demoted from a W3C Note, it rebranded itself during Global Accessibility Awareness Day as a pattern library, spewing 404s, dropping warnings, and failing WCAG. I regularly remind people it was only ever meant to experiment with ARIA patterns in an idealized context, independent of support or HTML-first principles.

With all that, it would be fair to dismiss me now as simply being grumpy.

Unfortunately, my grump can also come off as beating up people, which is not my intent. Describing and defining any pattern is a heavy lift. Anyone working on pattern libraries in any capacity knows this. Attempting to follow the letter of the specs and working to make them accessible and well-supported is an even heavier lift. There is a lot of good faith effort in the APG content by smart people with the best intentions.

Yet I have spent years tearing broken APG patterns out of client projects because they were never thoroughly tested. Because devs assumed APG was “Official W3C” and safe to use, and nothing in its materials suggested otherwise (excepting the few warnings I fought to get added).

Of course I welcomed the news that APG was finally getting some support notes:

W3C blog: "Answering 'What ARIA can I use?'" by Matthew King
“Assistive Technology Support” tables in the ARIA Authoring Practices Guide show how 3 screen readers support 4 UI pattern implementations represents a sea change in accessibility engineering w3.org/blog/2023/04/answering…

I would not call support tables for the safest four patterns a sea change in accessibility engineering. Certainly not when testing with only two browsers (at launch) and three desktop screen readers.

The effort seems promising. It is not, however, anything like Can I Use. Despite the framing:

This “Can I Use…” like data showing how three screen readers support four UI pattern implementations represents a sea change in accessibility engineering. The data are the first taste of the fruits of nearly five years of collaboration, learning, community building, and process and infrastructure development.

[…] As of the time of this announcement, the APG also provides the equivalent of “can I use …” data, i.e., AT support tables, for four of the reference implementations.

Echoed elsewhere:

Now developers can see what patterns are supported in which browser and screen reader combinations. This is a big move in the direction of making the web more accessible, open and inclusive for screen reader users.

That messaging is working. I have already had people tell me they are excited that the Can I Use charts have come to the APG. Except they haven’t.

Comparison

This is the Can I Use support chart for <button>:

This is the APG support chart for <div tabindex="0" role="button"> when announced (18 April):

I am thrilled that since this launched two weeks ago Firefox has been added to the support table and the NVDA and JAWS versions are now nearly current. If this pace of updates is maintained then this resource can indeed be incredibly useful within a few months. But it took them five years to get here.

Regardless of those updates, the information it provides is not helpful at a glance. I see the ARIA button is supported at 96% in NVDA 2022.3.2. That is meaningless to me as a non-practitioner. 4% of users cannot use it? 4% of the features are broken? It breaks 4% of the time? What about in NVDA 2023? Or NVDA 2021?

If I do some digging I see that the 4% maps NVDA’s failure to convey the role when I use Ins + ↑, but only with 2022.4 and 2023.1 and Firefox 111. As an accessibility practitioner, that is somewhat useful and speaks to how much effort is needed for support testing (and why it takes so long). As a full-stack developer under a deadline, this is meaningless.

Let’s compare that to just the abstract from A11ySupport.io for button role (this link takes you to the full results):

The data in A11ySupport.io is certainly older, but it is also historical so you can see what versions were tested. As you dive into the results matrices for each set of discrete tests, they continue with the same level of detail.

APG Is Not ARIA

Another critical difference is that the four patterns APG cites (buttons, links, radio group, and alerts) are very nearly the simplest patterns. As such, the support tables roughly map to ARIA for now. As APG tests more of its patterns, they will no longer map one-to-one because they will represent complex widgets with opinionated implementations (layout grids!). Further, without adding support for touch devices, they may mis-represent to users how well ARIA itself is supported, regardless of problems with APG’s specific patterns.

The W3C press release is not so overt in asserting APG is ARIA, though it does imply APG is a specification (it is not):

[…] The objective of ARIA-AT stated in terms of “WCAG accessibility supported” is to build consensus for definitions of “actually works” that are specific to every accessibility semantic defined within a specification related to ARIA. It is starting with semantics employed in APG patterns.

Bocoup is more overt in its post ‘Can I Use…’, but for ARIA! (and not just in labeling the sample table as ARIA support tables):

[…] Today we’re announcing that the first round of test results from ARIA-AT are available and embedded on the APG so web developers can read about screen reader support for various ARIA features.

This (maybe unintentional) conflation of APG and ARIA continues to confuse designers and developers, and these support charts will only continue that.

Source Data

The data in the support tables comes from the ARIA and Assistive Technologies Community Group (ARIA-AT CG). The home page for ARIA-AT frames the challenges with divergent screen reader experiences and lays out its overall plans.

Today, different screen readers often yield conflicting experiences when presenting a web page, disadvantaging or even excluding some users. These differences also create accessibility design and test barriers for web developers.

On the other hand, browsers are interoperable for people who do not use assistive technologies. That is, different browsers provide equivalent experiences. Browser interoperability facilitates an inclusive web.

Assistive technology users deserve equal inclusion. The ARIA-AT project aims to empower equal inclusion by realizing interoperability for AT users.

An immediate concern is this can signal that differences in announcements between screen readers are a problem. My next concern is the focus on screen readers alone despite its name.

AT Is More Than Screen Readers

A11ySupport.io also does something APG’s tests do not — it tests AT (assistive technology) beyond screen readers. One of my ongoing complaints about APG is its focus on screen readers (others haven noticed). It is hard to know from what we see now if or how APG will represent more AT in its support tables.

[…] In 2018, the ARIA-AT CG set a five-year goal of testing 60 APG examples with three desktop screen readers and two mobile screen readers. Even with the scope limited to a select few screen readers, the goal was more ambitious than we anticipated. […]

ARIA is primarily for screen readers, though how accessible names are calculated can also impact voice control users. Because of that, the remit of ARIA-AT is already too narrow:

Since the initial phases of the ARIA-AT project (at least through 2023) are focused on screen readers, we’ll explain interoperability using screen readers as the assistive technology. […]

Real Patterns Must Support More Than AT

Remember that ARIA does not bring interactions along with it, requiring developers to script every behavior. Because ARIA-AT is not concerned with non-ARIA support, as source material it cannot address broader support in the APG patterns. For example, will the known lack of touch support for the APG slider pattern be represented in these support tables? Will a developer who wants copy-ready code understand that gap?

Most common patterns (ReactJS Datepicker, ag-Grid, etc.) fall down with assistive technologies, but they often start by considering mouse, maybe keyboard, and maybe even touch. The APG slider pattern may feel like a red herring in this post, but it is also a canary for the coal mine of basic support. When an APG pattern does not work outside of a screen reader, how will these support charts represent that? Will developers assume that if a pattern is supported in a screen reader that it is also supported outside that context?

My prior experience with devs looking at the APG slider pattern is they will assume those fundamentals are already covered and APG builds on top of functional patterns. Screen-reader-only support charts that are silent on everything else will exacerbate that.

Wrap-up

This is difficult work in an ever-changing landscape. I don’t want to belittle the effort.

What I want is no more hyperbole, the support notes to reflect all AT, confidence the patterns actually work, and conflation with an actual spec to stop. Current behavior genuinely makes my job harder and results in inadequate patterns working their way into projects. This makes for worse outcomes for users and distrust from the developer community that accessibility practitioners struggle to keep engaged.

From the perspective of an accessibility practitioner, I would have preferred the APG effort to mimic A11ySupport.io at the start or, maybe better, contribute to it instead.