Skip to main content

Part 1: First accessibility audit

Published on March 11, 2019 in Audit category

Screenshot of the page being audited via tool that outlines HTML5 landmarks

After going through the Udacity course, reading the Professional Web Accessibility Auditing Made Easy, watching through A11y casts and more, it’s time to do the first full (albeit smallish in size) accessibility audit. That audit will be, of course, on this site itself - pragmaticaccess.com and first only on its homepage. Before this audit, I didn’t think about accessibility on the site so I’m expecting some errors.

The idea is to run the audit and get the results first in part 1, fix the errors in part 2, and then run the audit again in part 3 to make sure everything is working as expected. Fixes and follow-up audit (parts 2 and 3) will be covered in the next posts.

Guidelines

As there is quite a lot of material regarding the accessibility and multiple areas it covers - from screen readers, contrast checks, markup, and ARIA checks, etc. it can be a bit overwhelming to know where to start.

To keep things simple, at least for the first one, this audit is primarily guided with Template Audits and How I do an accessibility check – A11ycasts #11.

Roughly, the audit stages will be as follows:

  • Tab key navigation test & keyboard operable test
  • Landmarks
  • Headings
  • Automated accessibility checker
  • Markup validator
  • Screen reader test
  • Color contrast analysis
  • WCAG guidelines

Audit stages

Tab key navigation

One of the simplest one to test is the tab key test where we tab through the interface to figure out if all actionable elements (eg. buttons, input fields) are focusable (read keyboard accessible) as well as see if the focus order makes sense (left-to-right, top-to-bottom).

As the site is quite simple there were no major issues - everything was keyboard accessible, however, focus on the buttons is a bit hard to see.

Hard to see a focus on buttons

Also, not sure if a problem, but because the article area is clickable as a whole it makes a bit of a weird outline when focused.

Weird focus outline

One thing that should be added is the missing bypass/skip link when the first item in the page receives focus (first Tab press).

The menu could be reached via a tab, however, I’m not sure if when within the flat menu should it be operated via arrow keys (which is a must when having nested menus) or a tab or it doesn’t matter?

Landmarks

Landmarks are HTML5 elements (also available as ARIA equivalents) and when used they enable quicker and easier navigation throughout the page. Some common landmarks are header, main, footer, etc.

For easier check, I used the Landmark Navigation via Keyboard or Pop-up Chrome extension by Matthew Tylee Atkinson which lists and optionally outlines all the available landmarks in the page.

Landmarks on the page

There are only 2 landmarks on the page - navigation and main. main should be further split into header, main, multiple article items, and footer.

Do note that there is a difference between HTML5 landmarks and ARIA landmarks eg. HTML5 footer should have contentinfo ARIA role.

Headings

Headings are also used for quicker and easier navigation. They span from h1 all the way to h6. For easier check, I used the headingsMap Chrome extension by Rumoroso which lists all the headings in the form of a tree.

Headings outline

The homepage is simple so there are only 4 headings in-total. What I’m not sure is if h2 should also be h1 in this case or should I keep it as is.

There is a whole article about The Truth About Multiple H1 Tags in the HTML5 Era which discusses the use of headings. Steve Faulkner from TPG has advice regarding the usage of headings and their corresponding ranks in his The HTML5 Document Outline blog post.

Automated accessibility checker

There are several free accessibility checkers out there, namely Accessibility audit via Chrome (Lighthouse) which uses axe-core in the background, WAVE Web Accessibility Evaluation Tool, ACheker and more.

However, primarily I used the paid one Tenon.io by Karl Groves, as it has the pay-as-you-go option for $0.05 per API call (which equals one-page check) and has the most modern look & feel which makes it easy to scan & track through the reports.

It found 29 errors and 20 warnings.

Tenon.io results

Mostly it’s about basic error such as images without alt tags, links without text, low contrast, and actionable elements having smaller than the minimum size.

Chrome accessibility audit rendered the score of 67 and found pretty much the same issues.

Markup validator

HTML validator helps to catch some issues if the HTML is not properly formatted. In this case, I used the W3 HTML Checker which found the missing alt tags and had one warning.

Results of the W3 validator

Warning stated that

The main role is unnecessary for element main.

which some are arguing is not really true if we are targeting older browsers which don’t support new HTML5 landmark elements. However, HTML5 Landmarks Exposed by Scott O’Hara tests that hypothesis with newest browsers and screen readers and concludes that

As far as banner, navigation and main landmarks are concerned, outside of a single navigation failure with Edge + NVDA, these landmarks are exposed consistently and you may be able to safely ignore providing them roles.

Screen reader test

As I’m using the Mac I used the VoiceOver as my screen-reader of choice.

Some errors are repeated, namely, images are missing alt tags and confusing/missing landmarks. Some new things also popped-up, like announcing the menu as a list of 4 items instead of a menu.

Horizontal splitters are also announced and I’m not sure if that’s needed as they are purely decorative (visual) to separate the particular sections of the page. I’m considering marking them with aria-hidden.

Color contrast analysis

For color contrast, I tried to use the Colour Contrast Analyser by TPG but it kept spewing out errors (update: raised GitHub issue).

Colour Contrast Analyser error

Then I tried the Color Contrast Analyzer Chrome extension by accessibility.oit.ncsu.edu but it was all zoomed-in constantly and I couldn’t figure out what is the issue (maybe due to using the MacBook Pro with retina screen?).

Zoomed-in bug on Color Contrast Analyzer Chrome extension

Then I tried Kontrast - WCAG Contrast Checker Chrome extension but it was constantly offsetting the cursor position so I couldn’t reliably pick anything, which is a shame as it has a quite simple & intuitive interface.

Offset bug on Kontrast Chrome extension

Finally, I settled on the axe Chrome extension as it had better UI than the Lighthouse (Accessibility tab) within Chrome.

The test failed at the low-contrast green color used on article-related content and on date-time text which is light gray. The best remediation I found is via Tanaguru Contrast-Finder or Chrome’s own color picker which clearly marks the boundary between AA/AAA contrast levels. The goal was to find the best next color match which would satisfy the required contrast levels.

Chrome color picker

WCAG guidelines

In the end, I passed through the WCAG 2.1 guidelines to determine the overall “score”. I included all A and AA guidelines and was primarily guided via WebAIM’s checklist as the official WCAG spec is way too broad for this initial audit.

I also tried to find some tool to aid me with generating this report, but couldn’t really find any so here is text bare-bones version.

1. Perceivable

1.1 Text alternatives
  • 1.1.1 Non-text content - FAIL - checked via Tenon.io; images are missing alt tags
1.2 Time-based Media
  • 1.2.1 Audio-only and Video-only (Prerecorded) - N/A
  • 1.2.2 Captions (Prerecorded) - N/A
  • 1.2.3 Audio Description or Media Alternative (Prerecorded) - N/A
  • 1.2.4 Captions (Live) - N/A
  • 1.2.5 Audio Description (Prerecorded) - N/A
1.3 Adaptable
  • 1.3.1 Info and Relationships - PASS - checked via Tenon.io
  • 1.3.2 Meaningful Sequence - PASS - checked via Tenon.io and using this scriptlet
  • 1.3.3 Sensory Characteristics - PASS
  • 1.3.4 Orientation - PASS
  • 1.3.5 Identify Input Purpose - N/A
1.4 Distinguishable
  • 1.4.1 Use of Color - PASS
  • 1.4.2 Audio Control - N/A
  • 1.4.3 Contrast (Minimum) - FAIL - failing contrast levels on date text
  • 1.4.4 Resize text - PASS
  • 1.4.5 Images of Text - N/A - checked via Tenon.io
  • 1.4.10 Reflow - PASS
  • 1.4.11 Non-text Contrast - FAIL - failing contrast levels on buttons
  • 1.4.12 Text Spacing - PASS
  • 1.4.13 Content on Hover or Focus - N/A

2. Operable

2.1 Keyboard Accessible
  • 2.1.1 Keyboard - PASS? - checked via Tenon.io - not sure if menu should be operated via arrow keys instead of a tab
  • 2.1.2 No Keyboard Trap - PASS - checked via Tenon.io
  • 2.1.4 Character Key Shortcuts - N/A
2.2 Enough Time
  • 2.2.1 Timing Adjustable - N/A - checked via Tenon.io
  • 2.2.2 Pause, Stop, Hide - N/A
2.3 Seizures and Physical Reactions
  • 2.3.1 Three Flashes or Below Threshold - N/A - checked via Tenon.io
2.4 Navigable
  • 2.4.1 Bypass Blocks - FAIL
  • 2.4.2 Page Titled - PASS - checked via Tenon.io
  • 2.4.3 Focus Order - PASS - checked via Tenon.io
  • 2.4.4 Link Purpose (In Context) - PASS? - checked via Tenon.io; not sure if article links are too big (whole article summary is a link text)
  • 2.4.5 Multiple Ways - FAIL? - don’t have any table of contents or a site map, however articles can be found via Articles on top or via See all articles button on the bottom
  • 2.4.6 Headings and Labels - PASS - checked via Tenon.io
  • 2.4.7 Focus Visible - FAIL - focus on buttons is not sufficiently visible
2.5 Input Modalities
  • 2.5.1 Pointer Gestures - N/A
  • 2.5.2 Pointer Cancellation - N/A
  • 2.5.3 Label in Name - PASS
  • 2.5.4 Motion Actuation - N/A

3. Understandable

3.1 Readable
  • 3.1.1 Language of Page - PASS - checked via Tenon.io
  • 3.1.2 Language of Parts - N/A
3.2 Predictable
  • 3.2.1 On Focus - PASS
  • 3.2.2 On Input - PASS
  • 3.2.3 Consistent Navigation - PASS
  • 3.2.4 Consistent Identification - PASS
3.3 Input Assistance
  • 3.3.1 Error Identification - N/A
  • 3.3.2 Labels or Instructions - N/A - checked via Tenon.io
  • 3.3.3 Error Suggestion - N/A
  • 3.3.4 Error Prevention (Legal, Financial, Data) - N/A

4. Robust

4.1 Compatible
  • 4.1.1 Parsing - PASS? - checked via Tenon.io; I guess missing alt tags are not significant HTML errors
  • 4.1.2 Name, Role, Value - PASS - checked via Tenon.io
  • 4.1.3 Status Messages - N/A

Summary

The goal was to go through the whole audit process from start to finish and get familiar with tools and processes. Although the audited page is quite simple there were several errors that need fixing and there are some things that I’m not so sure about yet (eg. should I include the site map?).

Luckily, errors are quite basic so fixing them should be quick & easy. Stay tuned for the part 2 where I’ll go through the errors and fix them and then in part 3 do the follow-up audit which should verify that the site is Level AA accessible.

Tools used

For convenience, here is the list of tools I used for each stage:

Sven Kapuđija profile image
Sven Kapuđija
Author