After going through the Udacity course, reading the Professional Web Accessibility Auditing Made Easy, watching through A11y casts and more, it’s time to do the first full (albeit smallish in size) accessibility audit. That audit will be, of course, on this site itself - pragmaticaccess.com and first only on its homepage. Before this audit, I didn’t think about accessibility on the site so I’m expecting some errors.
The idea is to run the audit and get the results first in part 1, fix the errors in part 2, and then run the audit again in part 3 to make sure everything is working as expected. Fixes and follow-up audit (parts 2 and 3) will be covered in the next posts.
As there is quite a lot of material regarding the accessibility and multiple areas it covers - from screen readers, contrast checks, markup, and ARIA checks, etc. it can be a bit overwhelming to know where to start.
Roughly, the audit stages will be as follows:
- Tab key navigation test & keyboard operable test
- Automated accessibility checker
- Markup validator
- Screen reader test
- Color contrast analysis
- WCAG guidelines
Tab key navigation
One of the simplest one to test is the tab key test where we tab through the interface to figure out if all actionable elements (eg. buttons, input fields) are focusable (read keyboard accessible) as well as see if the focus order makes sense (left-to-right, top-to-bottom).
As the site is quite simple there were no major issues - everything was keyboard accessible, however, focus on the buttons is a bit hard to see.
Also, not sure if a problem, but because the article area is clickable as a whole it makes a bit of a weird outline when focused.
One thing that should be added is the missing bypass/skip link when the first item in the page receives focus (first Tab press).
The menu could be reached via a tab, however, I’m not sure if when within the flat menu should it be operated via arrow keys (which is a must when having nested menus) or a tab or it doesn’t matter?
Landmarks are HTML5 elements (also available as ARIA equivalents) and when used they enable quicker and easier navigation throughout the page. Some common landmarks are header, main, footer, etc.
For easier check, I used the Landmark Navigation via Keyboard or Pop-up Chrome extension by Matthew Tylee Atkinson which lists and optionally outlines all the available landmarks in the page.
There are only 2 landmarks on the page -
main should be further split into
article items, and
Do note that there is a difference between HTML5 landmarks and ARIA landmarks eg. HTML5
footer should have
contentinfo ARIA role.
Headings are also used for quicker and easier navigation. They span from
h1 all the way to
For easier check, I used the headingsMap Chrome extension by Rumoroso which lists all the headings in the form of a tree.
The homepage is simple so there are only 4 headings in-total. What I’m not sure is if
h2 should also be
h1 in this case or should I keep it as is.
There is a whole article about The Truth About Multiple H1 Tags in the HTML5 Era which discusses the use of headings. Steve Faulkner from TPG has advice regarding the usage of headings and their corresponding ranks in his The HTML5 Document Outline blog post.
Automated accessibility checker
There are several free accessibility checkers out there, namely Accessibility audit via Chrome (Lighthouse) which uses axe-core in the background, WAVE Web Accessibility Evaluation Tool, ACheker and more.
However, primarily I used the paid one Tenon.io by Karl Groves, as it has the pay-as-you-go option for $0.05 per API call (which equals one-page check) and has the most modern look & feel which makes it easy to scan & track through the reports.
It found 29 errors and 20 warnings.
Mostly it’s about basic error such as images without
alt tags, links without text, low contrast, and actionable elements having smaller than the minimum size.
Chrome accessibility audit rendered the score of 67 and found pretty much the same issues.
HTML validator helps to catch some issues if the HTML is not properly formatted. In this case, I used the W3 HTML Checker which found the missing
alt tags and had one warning.
Warning stated that
The main role is unnecessary for element main.
which some are arguing is not really true if we are targeting older browsers which don’t support new HTML5 landmark elements. However, HTML5 Landmarks Exposed by Scott O’Hara tests that hypothesis with newest browsers and screen readers and concludes that
As far as banner, navigation and main landmarks are concerned, outside of a single navigation failure with Edge + NVDA, these landmarks are exposed consistently and you may be able to safely ignore providing them roles.
Screen reader test
As I’m using the Mac I used the VoiceOver as my screen-reader of choice.
Some errors are repeated, namely, images are missing
alt tags and confusing/missing landmarks. Some new things also popped-up, like announcing the menu as a list of 4 items instead of a menu.
Horizontal splitters are also announced and I’m not sure if that’s needed as they are purely decorative (visual) to separate the particular sections of the page. I’m considering marking them with
Color contrast analysis
Then I tried the Color Contrast Analyzer Chrome extension by accessibility.oit.ncsu.edu but it was all zoomed-in constantly and I couldn’t figure out what is the issue (maybe due to using the MacBook Pro with retina screen?).
Then I tried Kontrast - WCAG Contrast Checker Chrome extension but it was constantly offsetting the cursor position so I couldn’t reliably pick anything, which is a shame as it has a quite simple & intuitive interface.
Finally, I settled on the axe Chrome extension as it had better UI than the Lighthouse (Accessibility tab) within Chrome.
The test failed at the low-contrast green color used on article-related content and on date-time text which is light gray. The best remediation I found is via Tanaguru Contrast-Finder or Chrome’s own color picker which clearly marks the boundary between AA/AAA contrast levels. The goal was to find the best next color match which would satisfy the required contrast levels.
In the end, I passed through the WCAG 2.1 guidelines to determine the overall “score”. I included all A and AA guidelines and was primarily guided via WebAIM’s checklist as the official WCAG spec is way too broad for this initial audit.
I also tried to find some tool to aid me with generating this report, but couldn’t really find any so here is text bare-bones version.
1.1 Text alternatives
- 1.1.1 Non-text content - FAIL - checked via Tenon.io; images are missing
1.2 Time-based Media
- 1.2.1 Audio-only and Video-only (Prerecorded) - N/A
- 1.2.2 Captions (Prerecorded) - N/A
- 1.2.3 Audio Description or Media Alternative (Prerecorded) - N/A
- 1.2.4 Captions (Live) - N/A
- 1.2.5 Audio Description (Prerecorded) - N/A
- 1.3.1 Info and Relationships - PASS - checked via Tenon.io
- 1.3.2 Meaningful Sequence - PASS - checked via Tenon.io and using this scriptlet
- 1.3.3 Sensory Characteristics - PASS
- 1.3.4 Orientation - PASS
- 1.3.5 Identify Input Purpose - N/A
- 1.4.1 Use of Color - PASS
- 1.4.2 Audio Control - N/A
- 1.4.3 Contrast (Minimum) - FAIL - failing contrast levels on date text
- 1.4.4 Resize text - PASS
- 1.4.5 Images of Text - N/A - checked via Tenon.io
- 1.4.10 Reflow - PASS
- 1.4.11 Non-text Contrast - FAIL - failing contrast levels on buttons
- 1.4.12 Text Spacing - PASS
- 1.4.13 Content on Hover or Focus - N/A
2.1 Keyboard Accessible
- 2.1.1 Keyboard - PASS? - checked via Tenon.io - not sure if menu should be operated via arrow keys instead of a tab
- 2.1.2 No Keyboard Trap - PASS - checked via Tenon.io
- 2.1.4 Character Key Shortcuts - N/A
2.2 Enough Time
- 2.2.1 Timing Adjustable - N/A - checked via Tenon.io
- 2.2.2 Pause, Stop, Hide - N/A
2.3 Seizures and Physical Reactions
- 2.3.1 Three Flashes or Below Threshold - N/A - checked via Tenon.io
- 2.4.1 Bypass Blocks - FAIL
- 2.4.2 Page Titled - PASS - checked via Tenon.io
- 2.4.3 Focus Order - PASS - checked via Tenon.io
- 2.4.4 Link Purpose (In Context) - PASS? - checked via Tenon.io; not sure if article links are too big (whole article summary is a link text)
- 2.4.5 Multiple Ways - FAIL? - don’t have any table of contents or a site map, however articles can be found via Articles on top or via See all articles button on the bottom
- 2.4.6 Headings and Labels - PASS - checked via Tenon.io
- 2.4.7 Focus Visible - FAIL - focus on buttons is not sufficiently visible
2.5 Input Modalities
- 2.5.1 Pointer Gestures - N/A
- 2.5.2 Pointer Cancellation - N/A
- 2.5.3 Label in Name - PASS
- 2.5.4 Motion Actuation - N/A
- 3.1.1 Language of Page - PASS - checked via Tenon.io
- 3.1.2 Language of Parts - N/A
- 3.2.1 On Focus - PASS
- 3.2.2 On Input - PASS
- 3.2.3 Consistent Navigation - PASS
- 3.2.4 Consistent Identification - PASS
3.3 Input Assistance
- 3.3.1 Error Identification - N/A
- 3.3.2 Labels or Instructions - N/A - checked via Tenon.io
- 3.3.3 Error Suggestion - N/A
- 3.3.4 Error Prevention (Legal, Financial, Data) - N/A
- 4.1.1 Parsing - PASS? - checked via Tenon.io; I guess missing
alttags are not significant HTML errors
- 4.1.2 Name, Role, Value - PASS - checked via Tenon.io
- 4.1.3 Status Messages - N/A
The goal was to go through the whole audit process from start to finish and get familiar with tools and processes. Although the audited page is quite simple there were several errors that need fixing and there are some things that I’m not so sure about yet (eg. should I include the site map?).
Luckily, errors are quite basic so fixing them should be quick & easy. Stay tuned for the part 2 where I’ll go through the errors and fix them and then in part 3 do the follow-up audit which should verify that the site is Level AA accessible.
For convenience, here is the list of tools I used for each stage:
- Tab key navigation test & keyboard operable test - none (keyboard only)
- Landmarks - Landmark Navigation via Keyboard or Pop-up Chrome extension
- Headings - headingsMap Chrome extension
- Automated accessibility checker - Tenon.io
- Markup validator - W3 HTML Checker
- Screen reader test - VoiceOver on Mac
- Color contrast analysis - axe Chrome extension, Tanaguru Contrast-Finder, Chrome color picker
- WCAG guidelines - WebAIM’s checklist