At GDS we iterate. One of the key contributions to that iteration is the post change checking to make sure that the users needs truly have been met and that the rules of the game have not changed. Peter Jordan, Product Analytics Lead unravels the Bank Holiday story.
GDS talks a lot about meeting user needs as the defining principle behind GOV.UK, and, of course, one of GDS’s design principles is to design with data. We already have data from many existing sources, such as usage from existing sites, A/B testing, search data and user feedback.
But as the GOV.UK beta gradually gains more visibility and traffic, we are starting to see useful digital analytics data from actual users even though total numbers are still quite small at the moment.
One of GOV.UK’s content formats is the ‘Quick Answer’. We identified a number of needs where people just want a quick, accurate, answer and don’t want or need to interact further with GOV.UK at that time.
Design input drove the decision to highlight the next Bank Holiday in a super clear big box, but the remaining information is still available below it.
What does the data tell us?
To look at some real data, we drilled down to the Quick Answer which has, so far, had the most interest, Bank Holidays. The data is for an eight-week period from mid-May 2012.
Google is the homepage
During the eight week period, there were 10,742 visits to the page. The overwhelming proportion (89%) came from search compared to referrals from other websites or typing in the URL.
People used 972 different search phrases to get to this page - here’s the top 15:
Notice the number of queries that include next, which is a nice vindication of our designers’ decision to highlight the next bank holiday date.
Timely information
As you’d expect, people mostly searched for bank holiday information just before the weekend the holiday would fall on and then on the Monday of the weekend itself.
Behaviour of “Bank Holiday” visit intent
We wanted to know if visits with a “Bank Holiday” intent behaved differently to other visits to GOV.UK. To gauge this, we filtered all visits to GOV.UK to select a group that:
- came from search keywords that contained 'bank holiday' or
- links that landed on the 'UK bank holidays' page
- entered the URL for the page directly
We then compared the behaviour of this group of visits with that of all visits to the site for the same period.
Compared to all visits, “bank holiday intent” visits showed different behaviour. On average, they:
- visited half as many pages (1.19 compared to 2.72)
- spent 5 times less time on GOV.UK (22 seconds compared to 118 seconds)
- had a bounce rate twice as high (nearly 90% compared to 40%). A bounce occurs when a web site visitor only views a single page on a website.
A simple reading of this data might indicate a poor level of engagement with the Bank Holiday page, but in the context of what we are trying to achieve with Quick Answers, we believe this is an excellent result. People came from Google or another search engine, could quickly scan the key information thanks to the design of the page and then leave with a definitive answer.
It also shows the value of segmenting analytics data to focus on specific user journeys and formats.
We will be doing more of this work in the future, to analyse other needs and formats. No doubt some of our learning will show less successful outcomes - but that’s all part of the journey!
16 comments
Comment by Ed posted on
Is it possible to have a web-service deliver up an (e.g.) xml file of future public holidays in a standardised format? Thanks. Ed
Comment by GOV.UK – One day in | Government Digital Service posted on
[...] the help of the Data Insight team we’ll be looking at this data, and more, every day to make sure we’re focusing on improving the [...]
Comment by richard Davey posted on
Those statistics are interesting results.......I wonder if you would get the same oh Yahoo search?
Comment by peterjordan posted on
The data is for referring traffic from *all* search engines, not just Google. Though, of course, Google has by far the largest market share.
Comment by peter posted on
No images? or is it me and my PC
Comment by Louise Kidney posted on
Hi Peter,
Is it possible for you to check on a different PC or on your phone to see if they appear? No one else is reporting problems.
Louise
Comment by peter cardenPeter posted on
Thats interesting, I am on a Gov network which normally wont let me see Blogs or Social etc (including WordPress sites), however I can read your site but not see images. I checked on my home PC last night and your images are there.
Comment by David Bennett posted on
It's not likely given the metric for the search term, but it's vaguely possible that the high bounce rate occurred because people did not find what they were looking for.
You could put a 'rate this page' on the page to help to clear up any doubts?
Where you say - "Notice the number of queries that include next, which is a nice vindication of our designers’ decision to highlight the next bank holiday date"
On what did the designer base his decision to highlight the 'next' date? Was it intuition, anecdotal evidence, market research, or something else?
Comment by peterjordan posted on
This post focused on using data of actual users, but when we are developing new content or products, part of the process is to research up front user needs around the product. A really good way of getting data is to look at search data to identify not only the core need, but also different facets around it. This older post explains more about this: http://digital.cabinetoffice.gov.uk/2012/01/27/search-data-user-needs/
Comment by Philip Hale posted on
Did you develop analytics software in-house or do you use commercially available tools? Probably a mix of both...
Comment by peterjordan posted on
There are, of course, lots of commercial analytics products on the market, one of which we are using. But the two things we are working on at GDS are:
1. Embed a culture of using evidence and insight in product development and improvement
2. Make it easy for product owners and other stakeholders to have actionable information by integrating data from a variety of sources (digital analytics, web ops, insight, cost) into a data insight platform that we are building in-house.
Comment by Philip Hale posted on
This post is great, thanks for showing us what it looks like to develop software with heavy emphasis on user needs.
Comment by alan mather posted on
But isn't a better question "should we (government) be offering the bank holiday service at all?" ... when I typed "UK bank holidays" into google, I got 321k results (47m if I take out the quotes) and 189k from Bing. There's a lot of competition for that. Is it government's job to be the definitive answer? More to the point given they come and go so fast, is it not a job best left to others? Not all user needs need be filled by government.
Comment by Ben Terrett (@benterrett) posted on
Hi Alan, thanks for your question. One of our key design principles is "Government should only do what only government can do." https://www.gov.uk/designprinciples#second To answer your question, yes we should be offering up this information. Government sets the bank holidays, as some of them move around, and so we are the official source of that information.
Comment by Andy posted on
Doesn't the government also set which diseases in honey bees have to be notified to Defra?
Comment by peterjordan posted on
Alan, yes I think governemnt should. It's important that there is a canonical version of the information -and as government makes the dates, by Royal Proclamation, then I think GOV.UK should carry this information.