Skip to main content

https://gds.blog.gov.uk/2012/01/20/user-testing-accessibility/

User testing accessibility

Posted by: , Posted on: - Categories: Accessibility, GOV.UK, User research

My name is Joshua Marshall, and I'm a front-end developer and the Accessibility Lead for the team working on the beta of GOV.UK.

Most of my focus has been on making sure that both the public facing website and the internal tools we're building to support the site are as accessible and usable as we can make them. Now that we're well into the development of the GOV.UK project I wanted to give an update on the work we've been doing.

What have we been doing?

From the outset, we knew that the more expert help we could get, the better the final product we could deliver. To that end we asked Léonie Watson from Nomensa to help us. She is a recognised expert in the accessibility field, and we've been enormously lucky to have her expertise along the way.

We've been regularly developing both the administration tools we've created for our editorial teams and the public facing templates that visitors to the beta site will interact with.

Lots of my time has been spent using screen readers and a multitude of different browsers, platforms and tools to make sure that what we're building isn't excluding anyone from the content we're providing. Constant iteration has been, well, a constant. I know my limits, as a sighted, able-bodied developer, and knew that getting people with a wide diversity of requirements was a necessity.

 User Testing

Testing our assumptions with actual users – both able-bodied and disabled – has focussed our efforts on not just whether our code validates or the colour contrast is good enough but that formats work for all users and don't exclude anyone from taking part. Building this means more than just making sure we’re checking the boxes for WCAG AA compliance. It needs to work in the real world for real people too.

It's great that a technically savvy user can use the site unaided, but knowing that an elderly person with limited computing experience or a user with Aspergers was able to do so means we're much more confident that we're presenting our information in a way that works for as many users as possible.

Early December saw us run a series of disability user testing sessions so we could observe and understand how visitors with different abilities interacted with the site.

Our respondents all had a variety of abilities and were chosen to try to represent as many as possible of the user-groups that will use the beta site. They included:

  • blind screen-reader users
  • screen magnification users
  • deaf British Sign Language users
  • keyboard-only users
  • speech-recognition software users
  • Dyslexic users
  • Aspergers or autistic users

Testing Results

The feedback we received was generally positive. Users responded well to the structure of the formats and the stripped-down content.

There was some work to do to clarify language and interactions around certain parts of the site that we, as a team of mostly able-bodied developers, wouldn't necessarily consider during day to day use, but we were generally pleased that people were able to complete the tasks required without too many problems.

We have another round of testing in the coming months so it will be interesting to see how the formats test as we finalise the visual design and include all of the follow-up development tasks.

British Sign Language

Visiting the last eAccessibility Forum this summer it was clear that users whose first language is British Sign Language rather than English often felt left out in terms of the content they are provided with. The assumption being that because the content is presented in text and in English, that should be enough.

We think we can do better.

I'm currently working on having some of our content translated into British Sign Language and recorded as a video-based alternative to certain text-heavy guides.

It's not always enough to show the exact translation of the text - sometimes roleplaying what might happen around the content is more appropriate, along with the literal translation of content pertaining to the laws of the land so there’s a fuller understanding of both what will happen and why.

 Future Work

There's always work to do to make our tools more useful. Accessibility is like everything else on the beta of GOV.UK: a constantly evolving project. There will always be more work to do to deliver a truly world-class website for the citizens of the UK and to keep delivering on the promise to make it as open and usable to all as we can. We’re listening, we’ll be testing our assumptions over the upcoming months, and we’ll really value feedback from the community once the beta site is live.

Sharing and comments

Share this page

41 comments

  1. Comment by Scott Woods posted on

    I cannot find anything about ordinal numbers (1st, 2nd, etc.) as opposed to cardinal numbers, but always advise editors to us cardinal numbers:

    OPTION 1 (this is what I suggest)
    "letting you choose from 3 monthly payment dates (1, 15 or 28)."
    OPTION 2 (this is what the department wants)
    "letting you choose from 3 monthly payment dates (1st, 15th or 28th)."

    We also do this with all dates, but I cannot find anything to support/contradict me on this, do you have any information or advice on this?

  2. Comment by Peter Bell posted on

    Can I ask if any consideration has been given to making the output available other than on screen (i.e. html only format)?

    I am able bodied but struggle to read for a long time on screen - so like to search and then print out the item to read later away from the screen (thus saving strain on my eyes).

    Where is the gov.uk PRINT button?

    • Replies to Peter Bell>

      Comment by Carrie Barclay posted on

      Hi Peter

      Thanks for your comment. If you choose a blog post to print, your computer should automatically remove the right hand navigation and simply print out the content of the blog post for you to take away and read at your leisure.

  3. Comment by Léonie Watson posted on

    @Paul

    Thanks for your comments.

    One drawback to providing a website widget is that they're all a little bit different. A text resize widget might use links, or radio buttons, or the AAA format, or something else. Instead of asking people to cope with that, asking them to use Control +/- to zoom in/out of content feels like an easier option. It has the added benefit of making all content easier to see, not just text of course.

    With colours it's often the case that if someone needs to change things, they're already using software to do it. If someone needs a particular colour scheme (high contrast for example), they're likely to need it everywhere on their computer not just a particular website. We can't hope to provide enough choice to be a viable substitute for software like this either.

    Another consideration is the impact these widgets have on other people. Placed prominently on the page they add cognitive load which may be difficult for people with certain cognitive disabilities. They also add additional steps for keyboard-only users, and although a skip link can minimise this it isn't always a viable option for people to use..

  4. Comment by Paul posted on

    I'm struggling to see how suggesting that users should be educated in how to use the accessibility features in their browser of choice (or their organization's choice), rather than provide an accessibility header as in the suggestion at https://gds.blog.gov.uk/2012/01/20/user-testing-accessibility/#comment-979 ?

    With this accessibility header I can change the contrast in a single click, whereas I can only do it in 4 in IE11 if I know where to look. With an accessibility header I don't need to know where to look and I don't have to learn for each browser that I might need to use.

    Do people really expect that non IT savvy citizens will take the time to learn the accessibility features of a browser before reading online what a public service has to offer? There used to be a rule of thumb that said that if a user had to scroll more than a page and a half then they would bale out on the web site and try another, or perhaps pick up a phone.

    Text size is probably easier, but still a big ask to expect somebody like my Grandparents and possibly parents to figure out.

    I notice that Disability Rights UK and http://www.equalityadvisoryservice.com/ still have an accessibility header.

    Am I missing something here?

    • Replies to Paul>

      Comment by John O donnell posted on

      Paul

      Fully agree with you

  5. Comment by Liz Nichols posted on

    I am trying to renew tax disc, however system not available and when I tried to phone I was told by recorded message to call back later. It is now 5 30 pm and I have been trying all day to renew tax. Not a good service. Soooooo what happens now - its due today 1 October ????

  6. Comment by Whittington W Hughes posted on

    Have been trying to phone the Department for Works & Pensions from my home in Canada. The telephone number I have been dialling is what is shown on your correspondence letters - that is + 44 191 21 87777, each time the phone operator cuts in and states the number is no longer in service. Please provide the proper telephone number so that I or my family may contact the Pensions Department. Many thanks Whittington W Hughes

    • Replies to Whittington W Hughes>

      Comment by Carrie Barclay posted on

      Hi Whittington - thank you for your message. I have checked the number you provided and it is working when you call from the UK. However I have done some research and some companies do not let you call +44 191 numbers from abroad - so it may be worth getting in touch with your telecoms service to confirm this. In the meantime, you can contact the International Pensions Centre using this online form: https://www2.dwp.gov.uk/tps-directgov/en/contact-tps/ipc.asp

      I hope that helps. Carrie

  7. Comment by jeff posted on

    i would like a contact / help part on the logging on page. if possible?

  8. Comment by Jim Simpson posted on

    Why does your site continually drop out, refresh does not work, there are no connectivity issues, pages lock up, job pages cannot be viewed. Very frustrating.

    • Replies to Jim Simpson>

      Comment by Carrie Barclay posted on

      Hi Jim. Could you let me know which pages are proving a problem for you? Thanks so much. Carrie.

  9. Comment by SANNAH FATTY posted on

    I APPLY A VISA THIS IS MY APPLICATION REFERENCE NOGWF031170458.I DONT HOW TO GET ACCESS ON IT

    • Replies to SANNAH FATTY>

      Comment by Carrie Barclay posted on

      Hi Sannah. Could you let me know what kind of visa you have applied for so I can try to help? Thank you.

  10. Comment by Stuart Carrington posted on

    Hi Joshua, I am the web manager for the Royal Armouries Museum and I have been following the progress of the European Accessibility Act with some interest. While I was reading this government document: http://www.publications.parliament.uk/pa/cm201213/cmselect/cmeuleg/86-xxxii/8603.htm#note13 it states "When the European Accessibility Act comes into force; HMG would expect that all UK public sector websites will need to comply with the legislation and — since in the UK public sector websites should be compliant with WCAG 2.0"
    I recall that on the old COI website it stated the required standard for all public sector websites as being WCAG 1.0 AA. Can you direct me to the policy where is states that all governement / public sector websites should conform to the WCAG 2.0 AA standard?

    • Replies to Stuart Carrington>

      Comment by Joshua Marshall posted on

      Hi Stuart.

      We noted in the Service Manual that the standard to aim for is WAG 2.0 at AA - https://www.gov.uk/service-manual/user-centered-design/accessibility.html

      It's perhaps not as direct as it could be given the guidance was written before our Service Standard went into effect yesterday, but I'm happy to reword it.

      Our expectation now the Service Standard is live is that any site operating on a gov.uk domain should be accessible by default so WCAG 2.0 AA would be the way to go.

      Hope that helps.

  11. Comment by Sarah posted on

    Hi Joshua, I am responsible for accessibility at the Office for National Statistics (ONS). I found your article very interesting and wondered whether I could ask you some questions about it?

  12. Comment by glenn posted on

    I am having trouble accessing the VisaUk site apparently because it doesn't support MAC OS 10.5.8 users. I don't have or wish to download IE versions. Moreover, my OS does not support Firefox 25 (which is mentioned by VisaUk for accessibility) or Chrome 29 (again mentioned by VisaUK).

    How can one thus submit an online application?

  13. Comment by Richard posted on

    There has been a lot of good work done in terms of accessibilty for this project so well done.

    However (there is always a however) I found out today that the video on the https://www.gov.uk/tour page has no captions. The same applies if opening the video in YouTube. There is a transcript but that does not meet the requirements for WCAG 2.0 level A http://www.w3.org/WAI/WCAG20/quickref/#qr-media-equiv-captions

    This is a particularly important video to be captioned because it is an introduction to the site. Is there a policy that all videos on the site should be captioned? If not, why not? Presumably the site is intended to at least meet level A of the checkpoints?

    Thanks.

  14. Comment by sm posted on

    Hi Joshua
    Thanks for the interesting post. Did you consider using an accessibility header across the site as per http://webacc.shaw-trust.org.uk/ ? What was your reasoning behind any decisions relating to this?Many thanks,
    Sam

    • Replies to sm>

      Comment by Joshua Marshall posted on

      Hi Sam.

      We've looked at a number of different solutions, but to date we've yet to come to the point where we think implementing one solves enough of a user need.

      I don't mean that to sound arrogant, and I know that to some it could be a sticking point that we currently don't, but it isn't because we don't care about accessibility. I work full time on accessibility across multiple projects at the GDS and it's at the heart of everything we do.

      We've tried to avoid using the typical font resizing or contrast widgets that you tend to see on sites. Including them by default because "that's how we've always built things before" leads to the possibility that a user visits a site without them, they perhaps won't know that their browser can do that for them already. Educating users that they can change the text size or contrast themselves without relying on external software or assistance felt like a better choice to us.

      I'd rather we didn't build a separate "accessible" site for users who may need a little extra assistance, I'd much prefer that we try and make our core experience as useful and usable as we can, for as many users as we possibly can.

      Like everything else on GOV.UK though, this is subject to change based on user feedback, so if we get an increasing number of people requesting it as a site-wide feature then we'll take another look and prioritise it appropriately.

      Hope that helps.

      • Replies to Joshua Marshall>

        Comment by sm posted on

        Thanks for the detail Joshua, I think that sounds like a sensible approach, I agree educating people not to rely on inbuilt accessibility controls is probably more useful for them in the long term.

  15. Comment by Evan posted on

    Brilliant. Particularly the recognition of dyslexia. Be great to include the simple guideline "stop limiting the maximum length of passwords" so we can use phrases. I'm locked out of so many government sites by short password requirements, no idea why, I can usually use passphrases on commercial websites.

  16. Comment by jellybelly123 posted on

    Hi Joshua,

    Really amazed with this type of effort for the people having certain disability.

    I am an Aspergers expert in Ireland, and i feel that this type of efforts are not just the need of UK, Ireland or any other specific country, but need for the whole world..

    It's great that you have initiated it, and people like me are really looking forward for the best results.

    All the best to you, as my Aspergers patients are waiting for easy accessibility.

    Reference: http://cluas.ie/children/aspergers-syndrome/

  17. Comment by Dale posted on

    As the COI website has been decommissioned, do you know if there will be any replacement for their "Delivering inclusive websites" guidance (perhaps on GOV.UK)?
    I only ask because public sector websites used to be held to a minimum accessibility standard of WCAG1.0 AA - see http://webarchive.nationalarchives.gov.uk/20120406035308/http://www.coi.gov.uk/guidance.php?page=131 .
    As a web developer for a local government website, I always used to refer to this as our benchmark when building new web services or writing our requirements when tendering for third-party services. However, with this guidance dying with the COI website, does it mean that public sector sites need only to reach single-A on WCAG2.0 to scrape by the Equality Act 2010? (I really hope not - we need the bar to be higher considering our target audience!)

    • Replies to Dale>

      Comment by Joshua Marshall posted on

      Hi Dale

      Yes, we absolutely should be publishing a replacement for the COI guidance.

      Not sure who would officially "own" it but as the Accessibility Lead on GOV.UK I'm happy to take the lead in making sure we have some current documentation to share.

      I'd guess we'd publish something as part of our Design Principles and style guides later on this year. At the very least I should add something to our Github styleguide repository (https://github.com/alphagov/styleguides), so thanks for the reminder.

      On GOV.UK we've been aiming for WCAG 2.0 AA where we can pragmatically reach it. We've put a lot of effort into making it work for all, so we should definitely share the knowledge we've gained along the way.

      Thanks for the comment, hope that helps!

      • Replies to Joshua Marshall>

        Comment by Craig posted on

        What testing tool do you recommend for developers to ensure we meet a minimum of Single A for Double A WCAG rating?
        Many sites claim to be A or AA compliant, but when I've tested them with free online tools, they usally fail.
        Is there a standard software install we should all be using?

        (Sorry for joining in so late)

  18. Comment by mrflicks posted on

    here even (excuse typo)

  19. Comment by mrflicks posted on

    Alt Tags

    I personally feel that nobody really writes all encompassing Alt tags either using short descriptions with a long description link or better short descriptions that bear in mind not just say "the deaf" like the alt tag hear might but also those who may not know the meaning of acronyms (most especially industry specific ones), as well as other aspects to.

    Now let us take the description of the hand signing alphabet. On the surface to those not visually impaired this looks the bees knees and most admirable. However if the description is not equally as admirable the essence of accessibility is lost in many translations

    IMHO

    But then I have and have designed cartoons and am having to think of such things

  20. Comment by Engaging With The Hard To Reach | Government Digital Service posted on

    [...] We recognise that if we are to succeed in driving channel shift to digital then services and transactions need to be developed with a relentless focus on users. We want to make use of the most innovative and versatile technology to deliver products that match industry leaders while ensuring that no-one is left behind. [...]

  21. Comment by Charlotte Daly posted on

    Well done Josh and Naomi, this will make a big difference to a lot of people with a variety of accessibility needs. I look forward to hearing more on the alternative video facility as a University friend of mine has accessbility problems accessing lecture handouts etc. Once Beta is launched, it hopefully will inspire other institutions to improve their accessibilty services.

  22. Comment by Pete Collier posted on

    Interesting article. Especially interested in the part about BSL and your comment about using video to enhance the service. Surely in it's current form this isn't a very sustainable solution - any consideration being given to using computer animation (a la Xbox) combined with BSL to give a more automated (and therefore more achieveable) route to provide additional support?

    • Replies to Pete Collier>

      Comment by Joshua Marshall posted on

      Thanks the for comment Pete.

      Personally I'm not sure that animating the content would be more sustainable over the long run. We can hire professionals to translate, shoot and package the content for us as video, and the turnaround time isn't particularly onerous. I'm sure we could commission someone to animate that content in much the same way but I'm not sure it'd necessarily save time or money.

      Obviously there would be production costs either way, but I'd be really interested in seeing any research that an animated clip vs video of a native-speaking signer provided as much or even more clarity or understanding for the viewer.

    • Replies to Pete Collier>

      Comment by Tim Paul posted on

      There are systems out that that will generate animated signing automatically for you - no need to get someone to manually create it. Here's an example: http://www.w3.org/WAI/RD/2004/06/sims-mov.htm

  23. Comment by Hosting the beta of GOV.UK | Government Digital Service posted on

    [...] iterative approach that’s been discussed here before, and that applies to writing content and designing user experiences right through to setting up the servers that power it all. When iterating so quickly we needed a [...]

  24. Comment by Tim Paul posted on

    Great to hear that accessibility is still being taken seriously. Over at TfL we've just completed our latest round of accessible user testing on the Journey Planner service (we're trying to improve the step-free access information for wheelchair users). As always it was an enlightening experience. It helps to be reminded occasionally of the huge gap between a product that's 'technically' accessible (ie. WCAG2.0 'AA') and one that's actually usable and useful for people with access requirements.

    • Replies to Tim Paul>

      Comment by Phil Day posted on

      Good to see accessibility being approached both in terms of following a standard, and also testing with consumers with different needs and abilities.

      Here at NCR we recently conducted user tests in partnership with the RNIB in the UK, and then in the US with the Center (sic) for the Visually Impaired & DisabilityLink.

      There is no substitute for evaluating with consumers.

      • Replies to Phil Day>

        Comment by Joshua Marshall posted on

        "There is no substitute for evaluating with consumers."

        Couldn't agree more Phil! I'm really looking forward to seeing how our next group of test participants find the site and how the results differ from the last round of testing we carried out.