Testing web content for accessibility can be a difficult task, but fortunately there’s some great guidance from W3C in the form of the Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0. However, many people get stuck at Step 1 – defining the evaluation scope. While setting a conformance target is generally straightforward such as WCAG 2.0 Level AA, it’s much harder to decide baseline-related issues such as which assistive technologies should be used for testing against the WCAG 2.0 Success Criteria and in which browsers the content should be displayed. Recently encouraged by a student question in the course I teach, I’ve put together some information on the topic for you to consider.
Before outlining my thoughts though there’s two important things to keep in mind: firstly, if you are doing a web audit for a client, the answer to ‘what browser and screen readers should I use to form a baseline?’ should generally be answered as ‘whatever the client wants’. You’re certainly welcome to point them to this post if it helps to explain your point of view about accuracy of testing, but I appreciate there may be very specific reasons why the client wants to test out, say, JAWS with Chrome. While some pairings in industry may be unusual, there’s often a method to the madness especially in locked-down enterprise-level systems where the Standard Operating Environment (SOE) gets changed for no one. Secondly, keep in mind that this assessment is a point in time and is purely my opinion based on working in the industry and many anecdotal conversations with screen reader users, so this can change quickly.
With that in mind, here are my recommended pairings for screen readers and web browsers.
1. NVDA and Mozilla Firefox
If you want to test in a traditional desktop environment on a Windows platform, it’s hard to go past NVDA with Firefox. NVDA is a fantastic screen reader with the developers at NV Access working hard to ensure the screen reader is up-to-date with great support across a number of recent Windows versions. In addition, updates tend to come out very quickly ensuring that it caters for changes to web standards and best of all, it’s free.
The benefits of NVDA are also in many ways the benefits of Firefox. The browser focuses heavily on standards-compliance with a legacy of effective support in this area and it plays very well with NVDA. As Firefox is also updated regularly you’re well placed to use these two together to maximise your accessibility testing with a degree of certainty for the results and at no charge to your organisation.
2. JAWS and Microsoft Internet Explorer
While NVDA and Firefox is arguably the most accurate paring for testing, there’s no denying that JAWS remains the king of screen readers and it’s likely that organisations will want to know how things go for JAWS users as a result. In my opinion JAWS testing should remain with Microsoft Internet Explorer despite the browser being so old that its existence in Windows 10 is barely acknowledged to the point that you probably didn’t realise it’s still there. The reality is that there’s still a lot of screen reader users that rely on old versions of JAWS due to the cost of upgrading, and its slow upgrade cycle has caused issues for it with other more modern web browsers. People testing websites often get frustrated with this combination as Internet Explorer is not the most standards-compliant browser, and JAWS certainly has its own quirks so the two combined may show up errors that you don’t believe most users would experience, yet this pairing is still reflective of a large number of desktop users and you may need to consider this if your organisation has JAWS users.
3. VoiceOver (iOS) and Safari (iOS)
With WCAG 2.1 on the way a time is coming when testing on a mobile will be an essential part of WCAG conformance, and with that in mind it’s important to know which pairing is best for mobile devices. On iOS devices such as an iPhone or iPad, it’s a pretty easy choice – use the built-in VoiceOver screen reader with the built-in Safari browser. Both work great together and in my experience there’s no other iOS browser that comes close to VoiceOver support.
4. Android TalkBack and Google Chrome
As noted above, with WCAG 2.1 coming it’s likely you will need to test on mobile devices, and the best Android option is to test using the TalkBack screen reader with the Chrome browser which works very well, better than any other web browser I’ve tested. If you have to choose only one mobile platform for testing though I’d go iOS at this point as while Android dominates the market in the general population, Apple iOS is far more popular among people who are blind or vision impaired – and I’m acknowledging this despite being primarily an Android user myself. That said, there’s nothing wrong in using Android for testing and if you do, go for TalkBack with Chrome.
5. ChromeVox and Google Chrome (desktop)
From a testing perspective, you may not be aware that there’ is a little-known screen reader tucked away for users of Google Chrome called ChromeVox, and this screen reader is also found in Chromebooks. While its not commonly used by people with vision disabilities, it can be a useful pairing for testing purposes Just be mindful that while it can help pick up issues, the overall experience is not going to be reflective of most screen reader users hence its further down this list. That said, Chrome is an excellent browser and both Chrome and ChromeVox are free. Furthermore, given the popularity of Chrome as a desktop web browser there’s a good chance it’s already on your computer.
6. Windows 10 Narrator and Edge – in the near future
At this point, screen reader purists are likely to start questioning my sanity by including Windows Narrator on the list paired with anything. I certainly won’t argue that Narrator in Windows 2000, XP, Vista and 7 was a terrible screen reader. However, updates in Windows 8 and significant improvements in Windows 10 including the Braille support coming soon highlight its significant improvements. I’ve used Narrator for testing and while it should certainly be no higher than six on this list, it has the advantage of being built-in so people are likely to have it, and Edge is a reasonably standards-compliant browser – well, significantly more so than Internet Explorer anyway! At the moment Microsoft still recommends Narrator users in Windows 10 use Internet Explorer, but with updates to both Narrator and Edge in the insider preview of Windows 10 likely to be released later in the year. This will become an option for testing. Not the best option, but good enough to be an option if your machine at work is locked down like a fortress and you have no other choice.
7. VoiceOver (Mac) and Safari (Mac)
I’ll be the first to admit that VoiceOver and Safari on iOS are fantastic, but VoiceOver on Mac OS and Safari on the desktop is in a desperate need for a massive overhaul. It’s unfortunate that VoiceOver on a Mac is good enough that you don’t really need another screen reader, but not good enough when its functionality is compared to other screen readers on other platforms. If you specifically have a blind user in your organisation that only uses VoiceOver on a Mac, then your best option is to test the content in Safari. I should stress that my opinion here is not because I particularly dislike VoiceOver – indeed it was the introduction of VoiceOver in Mac OS 10.4 Tiger that is credited for having the first fully-fledged screen reader in a desktop OS and that deserves respect, but its way overdue for an update.
So that’s a bit of an overview of my favoured pairings of screen readers with web browsers from a testing perspective. Hope it helps.
Screen readers and web browsers – what’s the best pairing for testing?
Published by Scott Hollier on 28 March 2017Testing web content for accessibility can be a difficult task, but fortunately there’s some great guidance from W3C in the form of the Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0. However, many people get stuck at Step 1 – defining the evaluation scope. While setting a conformance target is generally straightforward such as WCAG 2.0 Level AA, it’s much harder to decide baseline-related issues such as which assistive technologies should be used for testing against the WCAG 2.0 Success Criteria and in which browsers the content should be displayed. Recently encouraged by a student question in the course I teach, I’ve put together some information on the topic for you to consider.
Before outlining my thoughts though there’s two important things to keep in mind: firstly, if you are doing a web audit for a client, the answer to ‘what browser and screen readers should I use to form a baseline?’ should generally be answered as ‘whatever the client wants’. You’re certainly welcome to point them to this post if it helps to explain your point of view about accuracy of testing, but I appreciate there may be very specific reasons why the client wants to test out, say, JAWS with Chrome. While some pairings in industry may be unusual, there’s often a method to the madness especially in locked-down enterprise-level systems where the Standard Operating Environment (SOE) gets changed for no one. Secondly, keep in mind that this assessment is a point in time and is purely my opinion based on working in the industry and many anecdotal conversations with screen reader users, so this can change quickly.
With that in mind, here are my recommended pairings for screen readers and web browsers.
1. NVDA and Mozilla Firefox
If you want to test in a traditional desktop environment on a Windows platform, it’s hard to go past NVDA with Firefox. NVDA is a fantastic screen reader with the developers at NV Access working hard to ensure the screen reader is up-to-date with great support across a number of recent Windows versions. In addition, updates tend to come out very quickly ensuring that it caters for changes to web standards and best of all, it’s free.
The benefits of NVDA are also in many ways the benefits of Firefox. The browser focuses heavily on standards-compliance with a legacy of effective support in this area and it plays very well with NVDA. As Firefox is also updated regularly you’re well placed to use these two together to maximise your accessibility testing with a degree of certainty for the results and at no charge to your organisation.
2. JAWS and Microsoft Internet Explorer
While NVDA and Firefox is arguably the most accurate paring for testing, there’s no denying that JAWS remains the king of screen readers and it’s likely that organisations will want to know how things go for JAWS users as a result. In my opinion JAWS testing should remain with Microsoft Internet Explorer despite the browser being so old that its existence in Windows 10 is barely acknowledged to the point that you probably didn’t realise it’s still there. The reality is that there’s still a lot of screen reader users that rely on old versions of JAWS due to the cost of upgrading, and its slow upgrade cycle has caused issues for it with other more modern web browsers. People testing websites often get frustrated with this combination as Internet Explorer is not the most standards-compliant browser, and JAWS certainly has its own quirks so the two combined may show up errors that you don’t believe most users would experience, yet this pairing is still reflective of a large number of desktop users and you may need to consider this if your organisation has JAWS users.
3. VoiceOver (iOS) and Safari (iOS)
With WCAG 2.1 on the way a time is coming when testing on a mobile will be an essential part of WCAG conformance, and with that in mind it’s important to know which pairing is best for mobile devices. On iOS devices such as an iPhone or iPad, it’s a pretty easy choice – use the built-in VoiceOver screen reader with the built-in Safari browser. Both work great together and in my experience there’s no other iOS browser that comes close to VoiceOver support.
4. Android TalkBack and Google Chrome
As noted above, with WCAG 2.1 coming it’s likely you will need to test on mobile devices, and the best Android option is to test using the TalkBack screen reader with the Chrome browser which works very well, better than any other web browser I’ve tested. If you have to choose only one mobile platform for testing though I’d go iOS at this point as while Android dominates the market in the general population, Apple iOS is far more popular among people who are blind or vision impaired – and I’m acknowledging this despite being primarily an Android user myself. That said, there’s nothing wrong in using Android for testing and if you do, go for TalkBack with Chrome.
5. ChromeVox and Google Chrome (desktop)
From a testing perspective, you may not be aware that there’ is a little-known screen reader tucked away for users of Google Chrome called ChromeVox, and this screen reader is also found in Chromebooks. While its not commonly used by people with vision disabilities, it can be a useful pairing for testing purposes Just be mindful that while it can help pick up issues, the overall experience is not going to be reflective of most screen reader users hence its further down this list. That said, Chrome is an excellent browser and both Chrome and ChromeVox are free. Furthermore, given the popularity of Chrome as a desktop web browser there’s a good chance it’s already on your computer.
6. Windows 10 Narrator and Edge – in the near future
At this point, screen reader purists are likely to start questioning my sanity by including Windows Narrator on the list paired with anything. I certainly won’t argue that Narrator in Windows 2000, XP, Vista and 7 was a terrible screen reader. However, updates in Windows 8 and significant improvements in Windows 10 including the Braille support coming soon highlight its significant improvements. I’ve used Narrator for testing and while it should certainly be no higher than six on this list, it has the advantage of being built-in so people are likely to have it, and Edge is a reasonably standards-compliant browser – well, significantly more so than Internet Explorer anyway! At the moment Microsoft still recommends Narrator users in Windows 10 use Internet Explorer, but with updates to both Narrator and Edge in the insider preview of Windows 10 likely to be released later in the year. This will become an option for testing. Not the best option, but good enough to be an option if your machine at work is locked down like a fortress and you have no other choice.
7. VoiceOver (Mac) and Safari (Mac)
I’ll be the first to admit that VoiceOver and Safari on iOS are fantastic, but VoiceOver on Mac OS and Safari on the desktop is in a desperate need for a massive overhaul. It’s unfortunate that VoiceOver on a Mac is good enough that you don’t really need another screen reader, but not good enough when its functionality is compared to other screen readers on other platforms. If you specifically have a blind user in your organisation that only uses VoiceOver on a Mac, then your best option is to test the content in Safari. I should stress that my opinion here is not because I particularly dislike VoiceOver – indeed it was the introduction of VoiceOver in Mac OS 10.4 Tiger that is credited for having the first fully-fledged screen reader in a desktop OS and that deserves respect, but its way overdue for an update.
So that’s a bit of an overview of my favoured pairings of screen readers with web browsers from a testing perspective. Hope it helps.