I’m James Stewart and I’m one of the Technical Architects working at GDS. Over the past few months I’ve been mainly focussed on the publishing tools and platform that underpin GOV.UK, and in particular on the APIs they use and provide.
Way back in the early days of this crazy ride I wrote about "Building APIs, building on APIs". As I said back then, APIs, or application programming interfaces, are a vital feature of the modern web and are really important to GOV.UK.
They'll help other people use the information on GOV.UK to build tools and services, which is one of the recommendations of Martha Lane Fox's report. They also help us to build a site that can be iterated rapidly, with components that focus on their core specialism, and;
It also means that the APIs the rest of the world uses will already have been put through their paces in a real production system, refined through implementation, stable and constantly monitored.
That approach is as central to our approach today as it was back then. It's become more and more crucial as we've grown as a team and begun to get further into the detail of what it means to build the robust, secure system that GOV.UK has to be.
Clarifying the roles
We've moved fast to get GOV.UK ready to take the place of DirectGov and Business Link. Initially we built our main APIs into our publishing tools so that we could quickly iterate the two in tandem. That was helpful when we started out but it wasn't really allowing each component to focus on its core responsibilities. Dependencies between parts of the system became complex.
As time's gone on we've split those responsibilities apart, partly to keep each part of the code tight and focussed, and partly so that we can more clearly understand what each piece is doing at any given time.
Administrative and editorial tools update data, public-facing applications read and present that data. Small, focussed applications provide the APIs to join them all together. They talk to each other using the same technologies that we all use every day when browsing the web.
Even though we’ve changed the exact details of how they work, the APIs we’re building now remain very similar to those you may have seen if you dug into the beta: driven by clear URLs and returning easily parsed JSON.
Listening to the chatter
One huge advantage of separating out components and having them talk via clear APIs is that it helps us make GOV.UK secure and robust. Key to building a secure system is being able to quickly tell if a given component is doing something unusual. Are there an unusually large number of requests for new passwords coming through? Is the interface that's just meant to read information out of the database actually trying to update or delete that information? Information like that can give you an early warning that something's under attack so you can respond accordingly (and, ideally, automatically).
By separating out our code and having the components talk via clear APIs it becomes much easier to set up automatic monitoring that will tell us when something unexpected happens. It also becomes easier to work out which parts of the system are working hardest and might need a few more servers to run on, or an extra cache to reduce their workload.
Joining in
APIs that are built for the web and of the web really are central to everything we’re building here at GDS. That's why we've worked hard to clarify their roles and to find ways of monitoring chatter.
In the next few weeks we’ll explain a bit more about how those same APIs we rely on internally can be used by other developers building other sites and applications.
9 comments
Comment by Martin Godfrey posted on
We're about to spruce up a bespoke tax calculator tool on our website.
It's not a million miles away from HMRC's calculator http://stccalculator.hmrc.gov.uk/UserDetails.aspx
Before we do any development it would be great to know if there's an API yet to the HMRC tool ?
Or, if not yet, whether it will be available this year ?
Thanks
Martin
Comment by Tools over Content | Government Digital Service posted on
[...] talked before about various aspects of how we build software at GDS, and especially for GOV.UK, but I’d like to take a step back [...]
Comment by Technology behind the Next Generation WebApps for Government | GovInspire posted on
[...] for all data. Now the scope has also been expanded to include private data sources also and is made available in an API. Licensing is an factor to think about in this emerging area. The more open the better. Here is the [...]
Comment by APIs for all | Government Digital Service posted on
[...] week I talked about GOV.UK’s APIs and how we use them to build and monitor the site. Now it’s time to say a little more about [...]
Comment by Adrian posted on
We were told that "A list of government APIs will be published by September 2012". so where is the list?
Comment by Mike Thacker posted on
Thanks James. I'm looking forward to hearing more. Is the intention that third parties can not only access GOV.UK information but also write front ends that push data into GOV.UK? I'm also interested in the potential to extract transaction details in summary and detail format (subject to privacy considerations) along the lines of Open 311 and your forthcoming ELMS replacement.
Comment by James Stewart posted on
There aren't currently any plans for a writable API for third parties but if you have specific ideas we'd love to hear them.
We've also not yet got any plans to support Open311 but we are keeping a close eye on it (particularly in conversation with friends at Code for America), but again if you've got specific ways you'd like to use that data please do let us know.
Comment by Mike Thacker posted on
My ideas aren't well formed 🙂 but a document on changes to ELMS says "This new [GOV.UK] system will provide the foundations for extracting the application data in an electronic format, so that it can be input into each authority's CRM / case management systems." So if you can feed into a council's CRM, why can't a third party feed in the same format to you? My own interest is in being able to just read individual or aggregated data on service transactions - particularly against the Local Government Service List.
Comment by James Stewart posted on
Mike - there's no technical reason we couldn't allow that (though obviously we'd need to be very careful about security, data integrity, etc) - the key thing is to establish the specific user needs that that would serve. We'd definitely welcome any ideas along those lines, though they're perhaps best captured by emailing govuk-feedback@digital.cabinet-office.gov.uk rather than here.