On Tuesday, Sarah Prag wrote about some changes being made to GOV.UK based on feedback from usability testing. We wanted to share a few more details about what we’ve been doing; the methods we’re using, and the people we’ve been talking to in the run up to release.
“Putting user needs at the centre of everything we do” is one of our guiding principles at GDS, so it’s our job to make sure that we test GOV.UK with real end users, of all abilities, from across the UK.
Usability testing was carried out on Alphagov, and in the Beta site earlier in the year. Since then, more and more content has been added to the site and this has enabled us, in the last few weeks, to test much more realistic user journeys. Of course we’re finding a few issues… and fixing them – there’s nothing like seeing a member of the public struggling to find something on your website to help convince you that it needs to be different!
How we’re testing
Our approach has been to use several research methods to help give us the big picture – by this we mean in depth, qualitative lab-based testing, larger scale quantitative remote usability testing, ‘guerilla’ testing small parts of the site with “volunteers” and specialist interviews with stakeholders from professional groups and trade organisations. We’ve also been capturing Key Performance Indicators (KPIs) from Directgov and Business Link that will allow us to make a fair comparison with GOV.UK once it’s fully launched.
Quantitative testing gives us a lot of data to work with, so we can see how people move through GOV.UK, and Nick will tell you more about that method soon. Smaller, one-to-one methods like lab based testing gives us real, qualitative information about how people respond to the website as well as how they use it.
How lab-based testing works
In the last 2 months we’ve conducted 4 rounds of lab-based testing, in London, Manchester, Birmingham and Glasgow. In each case we recruited 12 people (a standard number for this type of testing) who represented end users of particular areas of GOV.UK.
For example, we tested business content with small business owners, and detailed international trade content with people working for companies who regularly import and export goods. We tested citizen content with people with young families, older people approaching retirement, people who are working, people who are unemployed and low income workers on state benefits.
In every round of testing we’ve aimed to include at least 2 people with disabilities, but we’ve also tested the website with people with a much wider range of access needs.
This testing has enabled us to identify several key usability issues which are being fixed before we launch GOV.UK, as Sarah mentioned in her post. All of the testing is cross-referenced, so we make sure we aren't just taking a few people's word for it - often, if we find issues in the one-to-one interviews, we'll see there's data that backs that up.
We don’t expect to stop testing once the site is launched and millions of people are using it every month – in fact we’re expecting to be working closely with our colleagues in analytics and data insight.
Once the website data starts telling us what the users are doing, we’re pretty sure our colleagues will want to know why – and that’s where the Insight and Usability Team can help.
11 comments
Comment by mkwebman posted on
I'd like to know if something as basic as log in to the government gateway has really been tested. I just went to try and renew my driving licence as I'm coming up to 70 (that's to indicate where the log in is) and it wouldn't accept my gateway ID and password. Then I tried to reset it - it wanted memorable date, name and place saying I would have set these up when registering - er no never done that. So I went to the HMRC self assessment login and lo and behold it accepted my stored details quite happily. I'd just like to point out that the ID and password were copied and pasted from stored info so were identical.
Can someone tell me how to resolve the issue because I can't actually find a page for that. I don't want to create a new ID - I shouldn't have to.
Comment by mysquarewheel posted on
I understand there are rather a lot of pages on the GOV websites, but is every single page tested?
I found some info on the National Careers Service website that is 2 years of out date...!!!
Comment by paulclarke posted on
Can you provide the URL and we'll have a look for you?
Comment by mysquarewheel posted on
https://nationalcareersservice.direct.gov.uk/advice/planning/jobprofiles/Pages/networkengineer.aspx
Thanks for the reply, everyone else i have spoken too has passed the buck.
Someone must be responsible for the content, and someone would have needed to test it before it went live surely?
Comment by paulclarke posted on
I've confirmed that the content team at the National Careers Service are looking at this now. Thanks for raising it.
Comment by mysquarewheel posted on
has this been resolved yet? i need to know if the correct info is on the website yet. what a shoddy service offered to those in need....
Comment by Aaron posted on
10 days along now, has this been resolved?
who is looking into it? can i have their contact details, maybe they can give me an insight into making a website live without testing it, when that website is for people who are out of work or looking to change careers.
Another waste of tax payers money??
Comment by paulclarke posted on
As we're in direct contact in other channels about resolving those content issues, we'll use those channels rather than this thread for further discussion, as it's getting off the original post topic of testing.
Comment by Researching Inside Government | Government Digital Service posted on
[...] don’t rely solely on any one method of user testing. In November we conducted formal lab-based user testing in London and Leeds. We did this to get a better understanding of how our audience was interacting [...]
Comment by What we know about the users of Inside Government | Government Digital Service posted on
[...] most recent user testing last week involved one-to-one qualitative testing with a 12 people who were all regular or frequent users of the DCLG and DFT websites. We asked them [...]
Comment by Not so special after all | Government Digital Service posted on
[...] were our goals and assumptions, and we set about validating them using lab-based testing [...]