At GDS we iterate. One of the key contributions to that iteration is the post change checking to make sure that the users needs truly have been met and that the rules of the game have not changed. Peter Jordan, Product Analytics Lead unravels the Bank Holiday story.
GDS talks a lot about meeting user needs as the defining principle behind GOV.UK, and, of course, one of GDS’s design principles is to design with data. We already have data from many existing sources, such as usage from existing sites, A/B testing, search data and user feedback.
But as the GOV.UK beta gradually gains more visibility and traffic, we are starting to see useful digital analytics data from actual users even though total numbers are still quite small at the moment.
One of GOV.UK’s content formats is the ‘Quick Answer’. We identified a number of needs where people just want a quick, accurate, answer and don’t want or need to interact further with GOV.UK at that time.
Design input drove the decision to highlight the next Bank Holiday in a super clear big box, but the remaining information is still available below it.
What does the data tell us?
To look at some real data, we drilled down to the Quick Answer which has, so far, had the most interest, Bank Holidays. The data is for an eight-week period from mid-May 2012.
Google is the homepage
During the eight week period, there were 10,742 visits to the page. The overwhelming proportion (89%) came from search compared to referrals from other websites or typing in the URL.
People used 972 different search phrases to get to this page - here’s the top 15:
As you’d expect, people mostly searched for bank holiday information just before the weekend the holiday would fall on and then on the Monday of the weekend itself.
Behaviour of “Bank Holiday” visit intent
We wanted to know if visits with a “Bank Holiday” intent behaved differently to other visits to GOV.UK. To gauge this, we filtered all visits to GOV.UK to select a group that:
- came from search keywords that contained 'bank holiday' or
- links that landed on the 'UK bank holidays' page
- entered the URL for the page directly
We then compared the behaviour of this group of visits with that of all visits to the site for the same period.
- visited half as many pages (1.19 compared to 2.72)
- spent 5 times less time on GOV.UK (22 seconds compared to 118 seconds)
- had a bounce rate twice as high (nearly 90% compared to 40%). A bounce occurs when a web site visitor only views a single page on a website.
A simple reading of this data might indicate a poor level of engagement with the Bank Holiday page, but in the context of what we are trying to achieve with Quick Answers, we believe this is an excellent result. People came from Google or another search engine, could quickly scan the key information thanks to the design of the page and then leave with a definitive answer.
It also shows the value of segmenting analytics data to focus on specific user journeys and formats.
We will be doing more of this work in the future, to analyse other needs and formats. No doubt some of our learning will show less successful outcomes - but that’s all part of the journey!