In the content team we’ve developed new theme dashboards that give us a page-level indication of GOV.UK mainstream content performance.
These dashboards are useful to everyone in GDS, as GOV.UK user data can reveal how people interact with government services, and how they’d like to. Data is the voice of our users - we need to interpret this language to give you what you need.
Trying to find a couple of data sources that would give us an insight into the performance of GOV.UK pages wasn’t easy. Our pages have different functions according to the user need they’re meeting, and Google Analytics can quickly turn into a rabbit warren without a clear idea about how a page should be working. Add to that the fact that the Google Analytics data we use is sampled, and you’ve got some pretty big challenges to overcome.
After lots of trial and error, we stripped our analysis back to show us whether we’re meeting the original aims of GOV.UK, namely:
Using the Google Analytics API, which John will blog about separately, we’re able to isolate the most useful Google Analytics metrics in our own customised spreadsheet. This allows us to include data from other sources, such as user comments from Zendesk.
Are we optimising for the common case?
Unique pageviews
GOV.UK is probably just as successful for what we’ve left out, as what we’ve included. We need to see at a glance what content is getting the most traffic so that we can ensure that the most popular content is really good, and is prioritised within the site. Likewise if content isn’t being used, we need to question why. The first column shows how much traffic each page is getting, and the pages are ordered by popularity.
For example in the Education dashboard, the most popular page Student finance login is getting about 200K more unique pageviews than the second most popular - the Student finance guide. A bit digging reveals that this is because these students are returning visitors - they’ve already created their student finance accounts, and now just need to log in to these accounts. Even those unfamiliar with student finance can see at a glance what the top user need within the Education section is.
How is the common case changing?
Percentage changes in unique pageviews, month to month
Which brings us to the second column, which shows month-to-month changes in unique pageviews. We need to monitor changes in user needs and reflect these on the site.
For example, the Winter Fuel payment benefit got about 160% more traffic from October to November, and is now the fourth most popular benefit. We prioritised this content in the browse page, and made sure people were getting what they needed from related links. It’s pretty obvious that winter-related content will get more popular in winter, but with thousands of pages on the site we need an over-arching view of demand cycles.
Here you can see how demand for the Winter Fuel payment benefit has varied from July to mid-December. See a post on GOV.UK traffic from last winter if you’re interested in seasonal cycles.
Of course not all changes in popularity are related to seasonal demand. Sometimes traffic will change in response to changes we make to the content, like if we’ve optimised content to appear higher in Google, or added a link that generates a lot of traffic to a page. For example, pageviews to our Make a SORN page increased by about 70K from October to November because we added a link to this page from the car tax related links section. Likewise a change we make could inadvertently reduce traffic - we need to keep an eye on this.
Are people finding our content in Google?
Entrances
The next column shows how many people are coming to a page from outside GOV.UK. This figure covers people coming from a link on another site, typing the URL in the address bar, or using a bookmarked link, but generally our external traffic comes from Google. So this column gives us an indication of whether Google is acting as the home page for GOV.UK. People looking to get something done online have a specific task in mind, and often and don’t know or care whether government provides the service. They’ll search for their need in Google and choose the best result.
For example the Jobseeker’s Allowance browse page is only getting about 6% entrances, which seems alarming. But we need to consider the context. This is a browse page, we’d rather people got straight to the actual content page that fulfilled their need. In this case it’s the Jobseeker’s Allowance guide, which as you can see below gets a higher percentage of entrances (though this could still be improved).
How can we be simpler, clearer and faster?
Searches on pages
The fourth column reveals how many people are searching on each page. This metric allows us to make GOV.UK simpler, clearer and faster by giving people what they want on a specific page so they don’t have to search. See our blog post The search is over… almost! to find out what we’ve done with the Search pages report in Google Analytics.
This is our starting point for page-level performance analysis. We also have a user comments column waiting in the wings for when we can plug Zendesk data into the spreadsheets. User comments are probably the most useful bit of data we have, as people can tell us exactly what they want on every content page. Data is the voice of our users, and we’re doing our best to interpret this language with both the dashboards, and our deeper content analysis with Google Analytics.
Note: sampled data means that Google Analytics has only given us data based on a proportion of visits to GOV.UK, and this proportion varies. This will be less of an issue when Google increase the amount of unsampled data we can get, but in the meantime we’ve designed the dashboards to give content insight despite this limitation. Anyone using the dashboard needs to refer to corresponding reports on Google Analytics before using the data.
Follow Lana on Twitter, join the @GDSteam conversation and don't forget to sign up to email alerts. Need more than 140 characters? See our contact and email information here.
You may also be interested in:
Dashboard for the Lasting Power of Attorney transaction released
6 comments
Comment by simonfj posted on
Nice byline Lana.
'Research is a team sport'. It's so true. Now if only we (the users trying to stay across the various inter-departmental teams) knew who, and where, all the teams were. So it's good that you're looking at "theme" services dashboards, because every theme must have a(n inter-departmental) team, which it is responsible for. Got a category list? Want one?
"Data is the voice of our users". No. It is an indicator of a service's usage = how well it's designed; nothing more. e.g. How many voice-activated services are being designed? Not on the agenda just yet.
Can we start to get some focus on the comms side - The voice of users - now - because it's just so important. This GDService blog now acts as the central page for a bunch of teams who have evolved from the gds.digitalcabinet office. This blog now sits in this list. https://blog.gov.uk/ where many other teams are being exposed, particularly the departmental-centric ones.
The only thing which is marketing to them is an individual departmental subscription, except the GDS which is a combination of service (inter-departmental) teams. So you've retained the gds silo as a brand for your teams in exactly the same way that the gcn has.
If one wanted some indication of the demand for "a voice", just look at the steam Sir Bob 's discussion started. https://bobkerslake.blog.gov.uk/2014/01/23/share-your-views-civil-service-leaders-to-discuss-progress-on-reform/
Boy could he use some help with something more than just a personal blog; threaded discussions at least. Top marks for leadership. But where are his teams? Imagine if Mike Bracken was the only contributor to the GDS blog.
BTW. You might want to update the link on the bottom of this page. http://my.civilservice.gov.uk/head-of-the-civil-service/ "provided by GDS".
We've even got some NZ teams popping in to the GDService to compare notes, which shows blogs are used for two way global communication, not just a way of informing National outsiders what's going on in any one department, or the GDService's hidden teams. e.g. Looking at the "Topics" and metadata you've used for this post has my head spinning about what team you could possibly be in.
So don't sweat the design of any one particular service too much. You know how it works. One can spend 10% of the time getting a service 90% right, and then get bogged down spending 90% of the time trying to get the last 10% "perfect", which no designer can ever do, unless they offer a user the ability to personalize a service. And you're delivering, not co-designing.
You guys are doing a great job. But the gds is working to the design philosophy of "deliver" "a service", to, "a user". That's OK. Get the automation on each (transaction) service and interface design, slick, and let's total up the savings (as we go, if we really want to show the accountants some 'results'). Low hanging fruit first.
Over the next 200 days we should see some real savings; the flip side being that many clerical workers.gov won't have a (reason for a ) job. That is, after all, the only way savings can be recognized (apart from the technology-spend reductions).
The tradeoff here is in developing the communication's hubs for Inter-departmental teams/groups/taskforces (from your perspective) from two levels of government, and including their common users, cause, as you say, that's the way (open) research AND development is done these days. These team's comms hubs are developing into global online universities. Hey! The GDS teams are good. Why hide them? Need a bit of education in designing networks though. Where are the DIO teams?
You might like this. The US national sport is a silent, short game of poker where, if one run out of chips, they must leave the table. The English seem to prefer a quiet chat over a never-ending game of scrabble. But we better have all the tiles on the board.
BTW, I did notice the recruitment going on. https://gds.blog.gov.uk/jobs/#transformation-lead Not that you need more people. You just need to think in teams. The talents already there.
Excuse this earful. Just saves me saying the same thing to some NREN managers. Difference is, they can design networks but can't write, or design services.
Comment by Dan Craddock posted on
Very interesting approach to measuring content effectiveness, Lana.
I'm keen to hear why you chose unique pageviews over pageviews.
Thanks,
Dan.
Comment by Lana Gibson posted on
Hi Dan,
Here’s the official Google description of each:
Pageviews vs. Unique Pageviews
A pageview is defined as a view of a page on your site that is being tracked by the Analytics tracking code. If a visitor clicks reload after reaching the page, this is counted as an additional pageview. If a user navigates to a different page and then returns to the original page, a second pageview is recorded as well.
A unique pageview, as seen in the Content Overview report, aggregates pageviews that are generated by the same user during the same session. A unique pageview represents the number of sessions during which that page was viewed one or more times.
So we treat unique pageviews as the closest metric to visits we can get to visits (we can’t apply the ‘Visit’ metric at the page level). We don’t want to count multiple times they’ve looked at a page within one session, rather know that they’ve visited it within their session.
Cheers, Lana
Comment by Dan Craddock posted on
Thanks Lana, that makes sense.
Comment by Nathan Wall posted on
Hi Lana
Any chance when John does his blog post that you guys will share your spreadsheet? Sounds like it's got some pretty useful API calls hooked in to it. Would save me from having to build my own 😉
Understanding the overall demand cycle is such an important thing. Over the past couple of months on our site we saw a jump in people looking for information about holidays, school terms, and benefit payment dates. We also saw a jump in people looking for information about the rules for taking time off, entitlements to holidays, impact on shift workers etc - we'll know next time to make this supporting info more obviously linked from the page that shows the holiday dates.
Getting sampled data is a bit of pain isn't it! I'm wondering though, in your closing note you say that people need to look at the GA reports before using the data - but doesn't the sampling stay in effect? What additional information are they getting from the reports - is it just because the spreadsheet doesn't cover everything in enough detail?
Comment by Lana Gibson posted on
Hi Nathan!
We can’t share the data yet, but I’ll send you the magic script used to run it.
Thanks for sharing about your seasonal content cycles - would be interesting to see if NZ and UK trends match!
The dashboards are designed to give an indication of page performance. They show what is happening, but deeper analysis is needed to find out why its happening and whether it’s good/bad/expected. In the standard Google Analytics reports our data is unsampled, but it gets sampled through the API. And because we calculate two metrics together (e.g. entrances as a percentage of unique pageview we’re calculating two sampled data sets.
Google intend to increase the level before sampling takes place for Premium customers, and once that happens sampling won’t be such an issue.
Cheers, Lana