So then, how did the experiment go? First, let's remind ourselves of the prototype's primary objectives
- To test, in public, a prototype of a new, single UK Government website.
- To design & build a UK Government website using open, agile, multi-disciplinary product development techniques and technologies, shaped by an obsession with meeting user needs.
The prototype was developed in 12 weeks for £261k. It launched 1 day late, but given the need to recruit and gel a suitably skilled project team from inside and outside government, Objective 2 can reasonably claim to have been delivered. A boundary-pushing experimental prototype (aka a Minimum Viable Product ) was delivered by an in-house team working in an open, agile way, placing user needs at the core of design process.
This isn't a new approach, but it's one that still all too rare across government. Crucially, all but a handful of the team who developed alpha.gov.uk are now civil servants working full time for the Government Digital Service.
But what about Objective 1? The reaction to the prototype itself?
Most commentary was very encouraging. It's nice that a UK Government prototype garnered plenty of positive press, it's nice to field queries from other Governments around the world and it's pleasing that the Chancellor highlighted the prototype in his keynote speech at Google Zeitgeist.
But the reaction that really matters came from real users. Actively asking people what they think about a new product is always chastening yet ultimately rewarding, akin to a visit to the stern dentist. And we were thrilled with the volume and quality of user feedback garnered. People are so keen to help Government improve our products. We just have to ask for help, listen and respond.
In the two months following launch on 10th May we saw over 100,000 visits to Alpha.gov.uk, with nearly 1,000 people leaving structured feedback and a further 3,000 passing comment via Twitter. If you've the time, it's worthwhile reading the comments and discussion on Get Satisfaction, helpfully sifted into common Problems, Ideas and Praise.
Inevitably it's the problems users report which are of most immediate interest - here's the top 10 with my comments in italics.
Top 10 problems
- Location not working / ambiguous / not working overseas (Big spike of bug reports at launch, most fixed quickly)
- Don't use 3rd party logins for feedback (Lack of clarity wrt feedback options. Fixed after a few days.)
- No browse navigation (Clearly vital, both for navigation and orientation. Time to mine much of the good work done as part of Project Austin.)
- Too much below the 'fold' / cannot see it on my screen (Not enough testing at smaller screeen resolutions. Mea culpa.)
- Not accessible (For which see a pre-launch mea culpa )
- Not audience specific enough (Need much clearer design delineation between sections aimed at specific audiences, be they citizen, business or professional)
- Links to transactions / local stuff broken (Not enough link testing plus link rot for local council transactions etc.)
- Search not accurate / does not use location (Although it's a cop-out to say search is never good enough, search is never good enough...)
- Browser compatibility (Mainly lack of IE6 support from those trapped on this browser in the public sector)
- Colour contrast (Sorry)
Top 10 praise
- Loved the space of the design / clear layout / great design / modern
- A lot achieved with time and money / keep it up
- It's not orange (although structured user testing showed some people rely on the orange for reassurance that they're in the 'right' place)
- Love the approach / open-ness / public feedback
- Love the location stuff
- Love pre-emptive search
- Content is easy to read / clear and to the point
- Strategic - more efficient, easier to do
- "Thank you!" / "Finally!" / "Brilliant work"
- Love the council alerts
Top 10 ideas
- Local alerts
- Spell check on search
- Remove the home page image / remove government news
- Topics
- Filtering by personal context (age, car user, specialism / location)
- Location by place name
- Better search (clearer sign posting, filter by content types / areas of interest, more accurate, better synonyms)
- Find my nearest (school, hospital, polling station)
- Make it mobile
- Provide official feedback channels
However, there's a problem. The reactions of users of alpha.gov.uk, while incredibly useful, are far from a complete picture. For a start, users were rarely using the site in anger, although it did help me deal with my neighbour's errant burglar alarm. But, even more importantly, the 100,000 users of the prototype were simply not representative of the diverse audiences who need gov.uk to work for them.
Alpha.gov.uk users were disproportionately younger, early adopters, and while people coming from the BBC and more mainstream press balanced this out to some degree, we would have been naive to not also conduct more structured, demographically-balanced user testing. This was lead by the excellent user insight team within the Government Digital Service.
The panel of young and old, experienced and novice users gave us some very clear steers, albeit in the inevitably artificial context of a user testing lab. Amongst much else, the research stresses the importance of:
- browse/category/related links, as much for orientation as navigation
- clearer design differentiation between audience types, again to help orientation
- ensuring we don't edit down the content *too* much (some felt it was too basic to be 'the government')
- internal search not being noticed, with people invariably return to the search engine from which they came
- more thinking needed to understand how to make location work well for everyone
Orientation is clearly critical, especially when users are arriving deep in a very large site. Does the user think they're in the 'right' place? What clues can we offer to help them? How can we help them understand when they're in the wrong place, as well as the right one?
I could claim we were deliberately provocative with alpha.gov.uk in not implementing any browse/category navigation, and only scratching the surface of related links. But truth be told we ran out of time, as we prioritised designing landing pages for external search queries before implementing any browse/category navigation. Happily, there's a lot of very good research within GDS on citizen's browsing behaviour from a earlier initiative (Project Austin). I'm hoping that the GDS insight team will blog in more detail on the findings from their research.
Digital specialists within other government departments have generally welcomed the prototype. They appreciate the user centric approach, the consistent UX and the desire for de-duplication of platforms. They raise valid concerns about how a single domain will work in practice, wondered in particular how a focus on user tasks can allow govt to tell people things they don't know they don't know, or to affect behaviour change. All valid, taken on board, solvable.
Enough for now, bar a final thanks those members of the public who found time to give such useful feedback to help gov.uk make a step change. The really hard work starts now. Of which, more soon.
24 comments
Comment by John Wills posted on
Please please don't leave the organisation and arrangements for content management until the end and a massive pleading for a consistent look and feel across the whole government structure. Mandatory of course!
Comment by Gov.uk, la nueva apuesta del Reino Unido! posted on
[...] de analizar el resultado final del portal de prueba Alpha.gov.uk, se detectaron algunas de sus principales falencias, los elementos rescatables y algunas [...]
Comment by Finding your way around GOV.UK | Government Digital Service posted on
[...] of not supporting a browse experience from the alpha prototype of GOV.UK, where number 3 of our Top 10 Problems via user feedback was “No browse navigation”. Once we tested our assumptions about search and the homepage [...]
Comment by Why we’ve changed the homepage | Government Digital Service posted on
[...] Google. We compounded this problem by making the homepage look like Google. Partly this is because the alphagov team ran out of time, but it’s also because we were thought that search would be the easiest way for people to find [...]
Comment by Dr. Jacobs posted on
I guess I'm too late. Would have loved to see how the site worked..since I'm considering buying one of ScraperWiki's services.
Comment by Tech weekly roundup | Cogapp posted on
[...] an extensive public Alpha testing phase last summer, the Cabinet Office’s Government Digital Service department have been carrying [...]
Comment by Pulling the servers’ strings | Government Digital Service posted on
[...] The beta of GOV.UK is most definitely a beta; the requirements for handling scaling and resilience are looser than they’d be in a full “production” system. But we still need to have started down a path of understanding how we’re going to run this thing for real and build on what we learned in the alpha phase. [...]
Comment by shugss posted on
The prototype was developed in 12 weeks for £261k !
shocking cost !
Comment by Karen Lewis posted on
Agree with much of the positive comment but what about the content? it doeen't matter how clever and whizzy a site is if the information isn't well organised, clearly labelled, current and easy to find (even if you don't quite know what you are looking for). A meta site as this would be is going to have to cater for multiple audiences looking for superficially simlar but actually quite different things. More attention to the less "sexy" aspects of web design; applying metadata standards, a robust information architecture and design (a government website isn't just about engagement but in supplying routes to advice, guidance, the evidence base etc) Principles around inclusivity and accessibility should apply across the board.
Comment by Gov.uk – From Alpha to Beta | Government Digital Service posted on
[...] digesting feedback from the Alpha.gov.uk prototype of a single domain for government, there’s now a clear next step towards the gov.uk [...]
Comment by JohnDunn posted on
Integrating local and national government levels always going to be difficult. Clean clear design certainly helps usability. Forward plan may be to template this to council / local government orgnaisations - might never be total uniformity but would be a bit closer.
Comment by Alex Stobart posted on
Tom and team
Very impressive achievement, and it leaves one asking :-
What happens next, and how do users stay involved, and able to help ?
Alex
Comment by Young Rewired State 2011: Fresh blood, fresh data and fresh hacks | dailywebday.com posted on
[...] of work that the Alpha Gov team has been doing with government websites, though that first phase has now come to a close.)How the NHS might look?The second team were building MyNHS using a combination of current NHS data [...]
Comment by Young Rewired State 2011: Fresh blood, fresh data and fresh hacks | Kanak Shah posted on
[...] of work that the Alpha Gov team has been doing with government websites, though that first phase has now come to a close.)How the NHS might [...]
Comment by Alpha.gov.uk: an obsession with meeting user needs « WhatUsersDo Usability and User Experience Insights posted on
[...] his wrap up report, Tom Loosemore (Deputy Director, Single Government Website, Cabinet Office) openly and honestly [...]
Comment by Tim Fanshawe posted on
Dear Tom, you mention that critical catch-22 of enusuring customers who do not know what they need to know find the right stuff and dealing with behaviour change. You boldy state it is "solvable" - I hope so, but how? Thanks, Tim.
Comment by Mark Steven posted on
"erupted" is predictive text for "stilted"
Comment by Mark Steven posted on
Great roundup... Looking forward to the next phase. It's nice to see gov embrace a less erupted, more immediate style of communication! Bravo!
Comment by William from Talk About Local posted on
good stuff Tom and crew.
Always interesting to see a call for really local stuff from a national site. It's very hard to do and keep up to date. Local gov gets so little traffic (in proportion) from national government sites that you end up with an external benefits problem.
In the spirit of the Big Society would you consider linking to a local volunteer-run non-govenrment site that is more useful on a local topic than a government one?
BTW - How many press officers, ministers and SPADs did you have to clear this piece through? Without their guidance...
Comment by Project WIP posted on
"ensuring we don’t edit down the content *too* much (some felt it was too basic to be ‘the government’)"
The expectation that it should be complicated because it is government terrifies us.
Surely this is an expectation you shouldn't mind breaking?
Great round up by the way. We love it!
Comment by Tom Loosemore posted on
Expectation gov is complicated is pretty deep rooted, and some people need to read all the detail to feel fully informed about what are often big life decisions. However, that doesn't mean *everyone* has to suffer all the detail all the time. Challenge ahoy!
Comment by The Emperor’s new clothes (and why it’s now OK to point) | Helpful Technology posted on
[...] I suggested to @pubstrat that the fact he could blog about such issues, and make such critiques, was a sign of how far things had come. That Ross could write a thinkpiece about privatising a civil service function — quite possibly fodder for a trade magazine story — is fascinating, and encouraging, too. And a few minutes later, up popped a ping from the new Government Digital Service blog, where the GDS team will be blogging about their projects, warts and all. [...]
Comment by links for 2011-07-31 :: Blog :: Headshift posted on
[...] Alpha.gov.uk wrap-up | Government Digital Service So then, how did the experiment go? (tags: government2.0 uk agile web2.0 webdevelopment) [...]
Comment by William at idealGov posted on
Good, thorough, open, honest feedback. Far closer to Ideal 🙂