Accessibility Reports: Context, Quality, Cadence, and Value
Is my site accessible? And how can I prove it?
These are the questions that a vast majority of our clients pose when it comes to WCAG compliance – whether that’s due to an internal desire to increase accessibility of content, or a need to align to legislative requirements.
Everyone wants the check mark. Everyone wants the binary yes/no answer. Everyone wants to press a button, create a report, and be done with it. Unfortunately, there is nothing that’s going to give that definitive result.
Accessibility is fluid. Even something as simple as “is this image accessible?” can be a matter for interpretation. Contextual images absolutely need Alt-text; decorative images or logos? Those are exempt. Can an automated tool differentiate between the two types of images? Not at all.
Even the best of reports aren’t able to be fully automated. There’s still a strong requirement to have a human review the results and provide context. Some positives – like the aforementioned logo – are actually false. And, depending upon the size of your site, they can have a dramatic impact on your accessibility score.
So if everyone wants a report, but no reports are 100 per cent plug-and-play successful, what should you do to get the value you want out of accessibility scans? Here are some high-level ideas to help you get important, contextualized, information that can benefit your accessibility efforts.
Know What You’re Testing For… And Why
The Web Accessibility Initiative – the stewards of the WCAG standards upon which pretty much every compliance standard are built – offers a report template that can help guide you through the process.
But what are you testing and why? Is it strictly technical, so as long as you dot the “i”s and cross the “t”s of WCAG compliance is everything good? Or are you looking at experiential accessibility? Is your site usable? Does it provide a positive and equitable experience for people with disabilities?
And what are the next steps after your report? Do you have people allocated to make the fixes? Do you have people to go through the report to flag false positives (and ideally have whatever tool you’re using skip those “errors” in the future?)
Knowing your motivations, what your goals are, and what resourcing you’re going to allocate to both evaluation and remediation can help determine what type of reports are for you.
What Report is For Me?
Armed with the knowledge of what you want and why, you can then explore the different types of reports that are available:
- Comprehensive Accessibility Audit Report: providing detailed information on website accessibility, with test results, flagged accessibility issues, and recommendations for improvement;
- Accessibility Compliance Report: confirms whether a site meets specific accessibility standards or guidelines;
- Accessibility Conformance Report: identifying specific WCAG success criteria that are met by a website and whether there are any known exceptions or limitations;
- Executive Summary Report: an easy-to-understand overview of the accessibility audit results, with recommendations for addressing any issues.
- Accessibility Roadmap Report: steps and timelines for implementing accessibility improvements on a website;
- Progress Tracking Report: updating the progress of accessibility improvements and highlighting remaining barriers to accessibility; and
- Usability Testing Report: a more manual process that shows how well people with disabilities can use a website. This is generally done through user testing with people with lived experience, and really focuses on the spirit of accessibility – ensuring that the experience is equitable.
There are a few tools at your disposal to run these tests. As well, some of the larger companies out there, like SiteImprove, can provide similar types of results. Tools like WCAG-EM and VPAT (Voluntary Product Accessibility Template) are great places to start.
What Am I Tracking?
There are some key things you want to track in a test: volume, severity, level of success, user experience, and progress.
- Volume: How many issues have been flagged? After remediation and exclusion of false positives, are those numbers decreasing? Are they increasing? If it’s the latter, understanding what content is impact them (e.g., is it all new content?) can help support remediation efforts. The problem may not be with the site, but rather the people inputting content – and that could require additional training. Do you have benchmarks in place upon which you can measure success?
- Severity: Is this a slight risk or a major risk? For example, though not ideal, offering certain content in an accessible format to users, upon request, is a way to fix minor issues. But if your accessibility challenges are such that they prevent people from completing a transaction, then that’s a different story.
- Level of Success: Maybe your site isn’t meeting WCAG 2.0 AAA standards, but that could be OK. If all you’re required is to meet AA standards, failing at an aspirational level isn’t a bad thing.
- User Experience: Are you gathering data or anecdotal feedback from actual users? What are their satisfaction levels? Do you have a baseline upon which you can measure improvement? What are they telling you anecdotally about the experience?
- Progress: Has something been on six consecutive reports? Why isn’t it fixed? What’s your expectation for remediation action? And is your in-house team or your development partner meeting those expectations?
And When Should I Run a Report?
As often as you need to, would be the simple answer. In most cases, we recommend an annual audit, just to make sure everything’s compliant. However, if you’ve undertaken a significant content change or update, you may want to do an ad hoc scan to make sure nothing’s been negatively impacted.
Even after hiring new content curators or working with third-parties, it may be worthwhile performing a spot check to make sure that everyone understands the requirements and is meeting them. If there’s a gap, it’s the perfect opportunity to do some training!
And if you’re looking at building a new website, don’t wait until just before launch to test for accessibility – it should be tested at regular touchpoints throughout the build, to ensure that the foundation of the site is accessible.
Get Help if You Need It
Accessibility can be very confusing, so it’s often worthwhile to call in some additional support. There are a number of experts in this field that can help answer any questions or support your efforts. We encourage you to reach out to us and discuss what options may be available to you and how Northern can help you reach your compliance goals.