In my ~20 years of responding to RFPs/RFQs, once organizations started to realize the value of accessibility (or fear of lawsuits), I saw more and more requests include a note on accessibility. In most cases this was just a single line item among many, often with nothing more than a checkbox: “Is your solution Section 508 compliant?”
Typically these requesting organizations don’t have the expertise within to evaluate whether a solution that has been delivered is genuinely accessible. More than once I have walked into a project after completion simply to deliver the bad news that the vendor failed to make good on its promise.
While at CSUN I saw a few conversations about third-party tools that have no accessibility built in at all, along with those that fail to meet claims made in their marketing or by their support teams.
As so many web sites and applications today are often built by hiring third-party developers or cobbling together features from different vendors, this something with which many organizations struggle.
Evaluating Vendors & Tools
Based on that ongoing frustration, I decided to propose a list of questions to ask and steps to take when evaluating a vendor or tool.
Look for an Accessibility Policy
Visit the web site of the vendor or tool maker. Look for an accessibility statement or policy, ideally linked prominently in the footer or similar. Read it and see what guarantees are made about the site, the organization, and how these are worded. This can be insight into an otherwise opaque tool or organization.
Look for Accessibility Tickets or Pull Requests
If you are evaluating an open source tool or if it maintains a bug tracking feature that you can see, then take a look through it to see what tickets are open. Look for an accessibility category. Look for how long tickets are open or merges sit. Check Stack Overflow for those tools that offload support there, as well as those that don’t.
21 October 2020: Find Issues with Accessibility Tag in GitHub Repos
In her a11yTO Conference talk, Sarah Higley demonstrated a tool she made that will allow you to see how many GitHub issues are tagged with accessibility for public repo, while also showing how long it takes for their first comment and time to close.
Look for Accessibility References on Social Media
Take to the Twitters (and/or the social platform where you have the most experience) with the company or tool name and the #a11y hashtag. See if people are raising issues, sharing tips, or discussing work-arounds. Pay attention to the tone and frequency.
Look at Community Engagement
Similarly to looking for references on social media, look to see if the organization or its representatives are engaging users at all, and then compare that to how they engage users with accessibility issues. A large consultancy may not have a developer relations team, nor would that be appropriate for a company that builds bespoke solutions, but a tool maker might (or should, at a certain size).
Ask Around
While social media platforms may be great tools to use from your chair, remember that there are plenty of places to get together and share information in meat space. Perhaps among your team, at local meet-ups, co-working spaces, conferences, and so on. My guess is you don’t interact with all those people on social media (nor do they all spend time there).
Ask for Demos
If a vendor is selling a solution or if you are just choosing an add-on for your site, ask for a demo that shows the accessibility features. It may be a kitchen-sink style hodge-podge, a previous project, or just some widgets. Take some time to evaluate them, starting with the quickest test you know first. After all, how ARIA may be used on overlays is moot if you can’t tab through the page (probably the easiest first test).
Ask for Posts / Articles
Some organizations have written about solutions, best practices, or just want to share some neat problem and solution combinations related to accessibility (or not). Look to see if these exist and pay attention to how they might have been solved.
Ask for a Written Guarantee
Finally, if you are paying for a solution and the vendor has assured you that it will provide accessibility, ask for a written guarantee. Make it a separate document; just one page. When people are asked to sign promises they tend to think a little harder about that promise. Whether you choose to ask for it up front or at the end (telling the vendor at the start to expect it) is up to you and your process, but make sure the vendor knows it is coming.
Write Testing into the Contract
Shortly after I posted this, Lainey Feingold pointed out another good tip:
File bugs. Contribute code fixes. Blog. Share the experience.
In the end, sometimes the only option that works is shaming an organization. Unfortunately, depending on the nature of the work your best bet may be to sue everybody.
Some Examples
It would be uncharacteristic of me to fail to highlight organizations that could do better. While none of them makes a promise to be accessible, the burden still lies on those making the request, so the techniques I outline above would quickly disqualify them.
TypeForm
TypeForm is currently a popular survey tool because of its visual style, but it is useless with a screen reader. Its opaque method of submitting bugs and requests, coupled with its stated (to me in email) unwillingness to incorporate even simple fixes, makes it a poor choice in any context. Though if you saw its (now corrected) failure to render results on Chrome/Android in the SitePoint CSS survey, you’d probably already have suspected this. A little testing would disqualify this as a tool immediately.
Update on TypeForm
Thanks to TypeForm asking me to file another bug on its ongoing inaccessibility (November 2017), I discovered that even the bug reporting tool is inaccessible (as I demonstrated in one video and then a second video). Typeform still has no plans to address it.
Slack
Slack isn’t a tool you add to a site, but it has become a popular collaboration platform for teams. While it has had an unfilled position for an accessibility product manager for a couple years, its unwillingness to eat its own dog food by requiring this person to move to San Francisco suggests that Slack is unwilling to make even a basic investment in making its product accessible. This says nothing of its use of a CAPTCHA on the application form. A little social media surfing should quickly disqualify Slack as a platform.
Wix
Wix is billed as a quick way to build a web site that is attractive. Both its tools and the sites it builds are known to be inaccessible. Some bug spelunking would reveal this and should immediately disqualify it as a viable solution, especially since adding accessibility isn’t something you decide to do only after it has enough votes.
Update on Wix: 27 May 2019
The link above that references voting for accessibility support now redirects to a page that neither accepts votes nor comments. You can find a 2016 version of that page in the Wayback Machine archive. Instead of fixing the problem in the intervening 3+ years, Wix decided to just hide the comments.
Disqus
If you want a third-party threaded discussion form, Disqus is quite the popular choice. It is also a pain to use with some assistive technology. In this case, if testing the tool is tricky you can always ask around. At CSUN more than a few hallway conversations were had, though sadly nobody could come up with a comparable accessible solution.
Polar
Polar is a polling tool (originally developed by Luke Wroblewski) that does not work with keyboards (as I noted in a mis-attributed bug report with WebCompat). Given that three of original developers are no longer there, its GitHub pages aren’t seeing any activity (as of 30 November 2016, the pages at github.com/input-factory have been purged and are not in the Internet Archive), and the Twitter account is dormant, you can expect accessibility issues probably won’t be getting addressed.
Sitemorse sells an automated testing product that presents its customers with a dashboard to show how a site is performing on a few different metrics, including accessibility. Unfortunately it appears from all the knowledge base materials that the accessibility testing is based on WCAG 1.0 (you have to expand the Accessibility bar first, then you can read about WCAG 1.0 and tables and WCAG 1.0 and link text). Given Sitemorse’s double claim that it is not an expert on WCAG (the second statement was in a comment it has since deleted), you should maybe take Sitemorse at its word and move along. I also made a PDF version of the post since, well, LinkedIn.
Update: March 28, 2016
Hah! VPAT = Very Packaged Alternate Truth (not Voluntary Product Accessibility Templates).
I am of the opinion that offering tips on how to validate a vendor (as I have done above) is good. Requiring people to provide their contact and employer information to get access to those tips is only intended to turn them into sales leads, and not so good.
After all, if you have just identified a lead then isn’t the intent to sell them your own services? That means those tips are a sales tool and should be treated with appropriate suspicion.
Update: July 12, 2016
Beware WCAG 1.0
If a vendor builds an entire testing model, automated or not, around WCAG 1.0 then you should be suspect. Given that WCAG 1.0 was superceded by WCAG 2.0 in 2008, and given that international laws and now plenty of case law all reference or incorporate WCAG 2.0, you should consider throwing any vendor out of your office if it even breathes WCAG 1.0.
There is a very real risk you will be addressing recommendations that will not only fail to protect you in the case of a lawsuit, but because of differences between WCAG 1.0 and 2.0, can open you up to greater risk.
Update: January 30, 2017
I have seen an increase in individuals and companies spinning up sites in WordPress and selling their overall UX consulting services, usually with accessibility as a featured line item. Everything I touched on above still applies. In addition, another quick test is to identify if the theme they are using is an off-the-shelf theme and, if so, if it is an accessibility-ready theme. Couple that with a quick check in the WayBack machine and you may find the site, and the business, only has an eight month shelf life.
Update: February 1, 2017
Sadly, even with all those checks in place you can get screwed.
What do *you* call it when a web firm takes contract to develop to WCAG but then claims no time is left in the sprint to make it accessible?
This is breach of contract. The next step is to seek damages. A concurrent step is to make others aware that this vendor is probably not up to the task.
Update: September 6, 2017
When there is no VPAT (because it is custom development work) you can use a questionnaire. Here is an example (it uses a Microsoft Word document):
At the Texas Department of Information Resources, we developed one such form, The Vendor Accessibility Development Services Information Request. Vendors must complete this form as part of any solicitation where development services are being procured. It asks for qualitative information regarding a vendor’s accessibility policies, organization, tools, training, corrective actions, and examples of prior accessible work. The completed form can be a significant indicator as to a vendor’s accessibility development capabilities; and can be used to gauge the level of confidence in the vendors testing and results, should that vendor be selected.
The rest of the post has some nuggets. Its overall message is that accessibility testing should always be the vendor’s responsibility, not the client’s.
A good starting point for businesses in achieving digital accessibility is to implement basic smart practices in vendor contracts.
This begins with a well-developed accessibility policy.
From there, businesses should carefully craft their RFPs and include accessibility as a contract term.
I think this follows nicely from the suggestions I make above for evaluating a vendor. After all, just because the vendor presents well doesn’t mean you just leave accessibility out of the contract.
Update: March 26, 2019
A marketing firm poses as the W3C:
And someone has mistaken them for the real deal, too. This, to my IANAL eyes, is a trademark infringement pic.twitter.com/leOzJXMOk8
A law firm uses scare tactics:
Yet another fear-based accessibility “expert” consultancy with a wildly inaccessible web site (focus indicators, form labels, and captions are a low bar, no?) – atlasaccessibility.com This crap is so damaging to the legitimate efforts! pic.twitter.com/Gac4sFAEjg
These are nefarious tactics that prey on fear and the under-informed. This doesn’t even count those who lie about their abilities. Be careful, ask for proof, build it into contracts.
Update: January 10, 2020
Nic Steenhout has written Checking 3rd Party Vendors’ Product Accessibility at the Knowbility blog. He has similar rules — ask for a VPAT, ask what standards they follow, do a keyboard test of the product (a great quick hit), build it into an RFP, and ask for their roadmap.
Update: December 7, 2021
I like this practical nugget from Erik Kroes about ING’s design system:
Using Lion does not ensure an accessible outcome. Sorry, it doesn’t work like that. What it does provide is a solid base for you to build your own design systems and/or components with. […] It’s your responsibility to use what’s provide, and build something that includes people.
Just after I posted this, Lainey let me know about the Accessible Technology Procurement Toolkit, a detailed set of documents, checklists, and resources for ensuring what you purchase will be accessible (or hopefully let you know if it isn’t before you purchase).
Good post thanks Adrian. Good accessibility is a collaborative effort undertaken by people for people, if buying a readymade software solution, check the credentials/passion of the team managing that code. I think you could use better examples though, there’s no point in naming and shaming companies who don’t actually guarantee accessibility in the first place.
I chose to name companies who have been informed, often repeatedly, that their products are inaccessible and that acknowledge it as fact, but choose to do nothing (or even push back).
I don’t shame companies who are likely unaware. I also don’t shame companies here who claim to work in accessibility but who create more problems than they solve (see notes on the Virgin America accessible site as an example), though they easily deserve it.
Leave a Comment or Response