So then, how did the experiment go? First, let's remind ourselves of the prototype's primary objectives
- To test, in public, a prototype of a new, single UK Government website.
- To design & build a UK Government website using open, agile, multi-disciplinary product development techniques and technologies, shaped by an obsession with meeting user needs.
The prototype was developed in 12 weeks for £261k. It launched 1 day late, but given the need to recruit and gel a suitably skilled project team from inside and outside government, Objective 2 can reasonably claim to have been delivered. A boundary-pushing experimental prototype (aka a Minimum Viable Product ) was delivered by an in-house team working in an open, agile way, placing user needs at the core of design process.
This isn't a new approach, but it's one that still all too rare across government. Crucially, all but a handful of the team who developed alpha.gov.uk are now civil servants working full time for the Government Digital Service.
But what about Objective 1? The reaction to the prototype itself?
Most commentary was very encouraging. It's nice that a UK Government prototype garnered plenty of positive press, it's nice to field queries from other Governments around the world and it's pleasing that the Chancellor highlighted the prototype in his keynote speech at Google Zeitgeist.
But the reaction that really matters came from real users. Actively asking people what they think about a new product is always chastening yet ultimately rewarding, akin to a visit to the stern dentist. And we were thrilled with the volume and quality of user feedback garnered. People are so keen to help Government improve our products. We just have to ask for help, listen and respond.
In the two months following launch on 10th May we saw over 100,000 visits to Alpha.gov.uk, with nearly 1,000 people leaving structured feedback and a further 3,000 passing comment via Twitter. If you've the time, it's worthwhile reading the comments and discussion on Get Satisfaction, helpfully sifted into common Problems, Ideas and Praise.
Inevitably it's the problems users report which are of most immediate interest - here's the top 10 with my comments in italics.
Top 10 problems
- Location not working / ambiguous / not working overseas (Big spike of bug reports at launch, most fixed quickly)
- Don't use 3rd party logins for feedback (Lack of clarity wrt feedback options. Fixed after a few days.)
- No browse navigation (Clearly vital, both for navigation and orientation. Time to mine much of the good work done as part of Project Austin.)
- Too much below the 'fold' / cannot see it on my screen (Not enough testing at smaller screeen resolutions. Mea culpa.)
- Not accessible (For which see a pre-launch mea culpa )
- Not audience specific enough (Need much clearer design delineation between sections aimed at specific audiences, be they citizen, business or professional)
- Links to transactions / local stuff broken (Not enough link testing plus link rot for local council transactions etc.)
- Search not accurate / does not use location (Although it's a cop-out to say search is never good enough, search is never good enough...)
- Browser compatibility (Mainly lack of IE6 support from those trapped on this browser in the public sector)
- Colour contrast (Sorry)
Top 10 praise
- Loved the space of the design / clear layout / great design / modern
- A lot achieved with time and money / keep it up
- It's not orange (although structured user testing showed some people rely on the orange for reassurance that they're in the 'right' place)
- Love the approach / open-ness / public feedback
- Love the location stuff
- Love pre-emptive search
- Content is easy to read / clear and to the point
- Strategic - more efficient, easier to do
- "Thank you!" / "Finally!" / "Brilliant work"
- Love the council alerts
Top 10 ideas
- Local alerts
- Spell check on search
- Remove the home page image / remove government news
- Filtering by personal context (age, car user, specialism / location)
- Location by place name
- Better search (clearer sign posting, filter by content types / areas of interest, more accurate, better synonyms)
- Find my nearest (school, hospital, polling station)
- Make it mobile
- Provide official feedback channels
However, there's a problem. The reactions of users of alpha.gov.uk, while incredibly useful, are far from a complete picture. For a start, users were rarely using the site in anger, although it did help me deal with my neighbour's errant burglar alarm. But, even more importantly, the 100,000 users of the prototype were simply not representative of the diverse audiences who need gov.uk to work for them.
Alpha.gov.uk users were disproportionately younger, early adopters, and while people coming from the BBC and more mainstream press balanced this out to some degree, we would have been naive to not also conduct more structured, demographically-balanced user testing. This was lead by the excellent user insight team within the Government Digital Service.
The panel of young and old, experienced and novice users gave us some very clear steers, albeit in the inevitably artificial context of a user testing lab. Amongst much else, the research stresses the importance of:
- browse/category/related links, as much for orientation as navigation
- clearer design differentiation between audience types, again to help orientation
- ensuring we don't edit down the content *too* much (some felt it was too basic to be 'the government')
- internal search not being noticed, with people invariably return to the search engine from which they came
- more thinking needed to understand how to make location work well for everyone
Orientation is clearly critical, especially when users are arriving deep in a very large site. Does the user think they're in the 'right' place? What clues can we offer to help them? How can we help them understand when they're in the wrong place, as well as the right one?
I could claim we were deliberately provocative with alpha.gov.uk in not implementing any browse/category navigation, and only scratching the surface of related links. But truth be told we ran out of time, as we prioritised designing landing pages for external search queries before implementing any browse/category navigation. Happily, there's a lot of very good research within GDS on citizen's browsing behaviour from a earlier initiative (Project Austin). I'm hoping that the GDS insight team will blog in more detail on the findings from their research.
Digital specialists within other government departments have generally welcomed the prototype. They appreciate the user centric approach, the consistent UX and the desire for de-duplication of platforms. They raise valid concerns about how a single domain will work in practice, wondered in particular how a focus on user tasks can allow govt to tell people things they don't know they don't know, or to affect behaviour change. All valid, taken on board, solvable.
Enough for now, bar a final thanks those members of the public who found time to give such useful feedback to help gov.uk make a step change. The really hard work starts now. Of which, more soon.