Right now, hosting services is one of the most time-consuming barriers for new digital services, and usually involves duplicating work done elsewhere. On the Government Platform as a Service team we’re working on solving that.
Repetition, repetition, repetition
Every digital service that government runs needs to have a place to run from; it needs to be hosted somewhere so that it is accessible to users via the internet. The service doesn’t ‘just work’; there is a lot of effort involved in setting up all the components required to host a service.
These components don’t vary much between services. Every service needs an automated way to let developers know when something is wrong, or to alert them to something. So, in practice, these groups of components end up looking very similar across very different services. The picture below shows you an example:
As you can tell, there’s a lot of duplication. Teams all over government can end up duplicating work that’s already been done elsewhere. That means spending time on areas that aren’t their speciality, such as application monitoring or log aggregation, which stops teams from focusing on their areas of expertise.
It also leads to a lot of time searching for people with expertise in this area to hire. All of this takes time and money and leaves teams less time to focus on their users’ needs.
One way to address these issues is to provide a platform as a service (PaaS) that services could use for their cloud hosting. A shared PaaS would then change the diagram above into something more like the one below:
A Government PaaS wouldn’t just solve the issue of duplication, and where to focus your teams. One thing that takes a lot of time in government is procuring commercial services and making sure they are accredited. If we could do that once, for the PaaS, then that could save service teams a great deal of time, while making sure that those aspects are being handled in the correct way.
What a Government PaaS needs
From the user research we’ve been doing it’s clear that it’s important that our platform has a concept of multi-tenancy - applications that run on the platform should be isolated from each other and not be able to read or change each others’ code, data or logs. It wouldn’t be appropriate, for example, if the Digital Marketplace application was able to access the data of the GOV.UK publishing platform.
We’ve also learned from our experience supporting GOV.UK that a platform where the people developing applications also support the application out of hours leads to better software and a better user experience. We want a platform that supports this model right from the beginning.
Apart from multi-tenancy and the support model, there are some other things that we feel are important in a shared PaaS.
It needs to be self-service. It needs to be easy and quick for application teams to get started, and the teams using the platform need to be able to make frequent changes. That means we need to make sure applications can be deployed and managed by the application teams, but also that they can make other administrative changes to their applications, for example configuring DNS. Allowing teams complete control of their applications will remove any unnecessary delays for them, and means the platform team can focus exclusively on iterating and improving the platform itself.
It needs to run on multiple public clouds. This approach ensures that we avoid being locked into a single provider, so we encourage price competition, while also removing the risk of a single point of failure. Changing infrastructure providers is very difficult to do if you’ve built to a single provider’s specification so this needs to be built in from the beginning.
What we've been doing
We’ve spent a couple of months exploring what a Government PaaS might look like and how it could help teams running digital services across government. We’ve spoken to many potential users, and we’ve worked closely with our colleagues in other departments who are working to address similar problems, and we’ve found that no existing departmental solution meets all the needs we’ve identified.
We’ve evaluated several open source and commercial options, and we’ve built a prototype and shown it to potential users – developers, web operations engineers and services managers – both within GDS and in other departments. We’ve tested our prototype by seeing how it works with real applications (for example, we tested it using Digital Marketplace and GOV.UK’s Government Frontend).
We’ll write about all of this more in later blog posts.
What we're doing next
We expect to be in alpha until the end of November, by which time we will have completed a detailed comparison of two open source PaaS technologies and addressed issues around performance, security, and scalability, for example. We are really interested in talking to more potential users, so if you are interested in getting involved in our user research, or seeing a demo of what we’ve done so far, please get in touch.
If this sounds like the kind of problem you'd like to work on, we are currently hiring for developers and web operations engineers. We are always on the look out for talented people to join the team so take a look at our videos describing how we work, our vacancies page, or drop us a line: gds-recruitment@digital.cabinet-office.gov.uk
Follow Anna and Carl on Twitter, and don't forget to sign up for email alerts.
8 comments
Comment by Joel Smith posted on
Really interesting work. As a starter for 10, in your Alpha, will you be making the scripts available for pre-building the platforms, so others can tinker? Std adage of many eyes etc. making a better product.
Comment by Carl Massa posted on
We have plans to present all of our repos in a tidy way soon, but we'll almost certainly scrap everything we have now if we determine it's best to build something. We're using terraform and ansible for the prototypes, but even that could change when we start beta.
UPDATE: 21 December - We've just published another blog post (https://governmentasaplatform.blog.gov.uk/2015/12/17/choosing-cloudfoundry/), but you can now find our repos here: https://github.com/alphagov?utf8=%E2%9C%93&query=paas-
Carl
Comment by Lai posted on
I love the idea of government platform as a service. I guess many government departments are hosting their own services. While they have their own existing systems, I wonder how you promote the use of your platform. Do they resist changes?
Comment by Carl Massa posted on
All of the departments and agencies we've engaged with are working hard to deliver savings in a responsible way, so we've found that they're interested in any solution that would do that. We consistently get unanimous nods of agreement when describing the issues, so we know the problems we're trying to solve aren't unique to us.
We've not had to pitch any specific solution yet because we've not yet decided on the approach, but we intend to promote the use of whatever we decide on via example. I'm confident that if we demonstrate the value, it will sell itself.
Comment by Ross posted on
This sounds great!
One thing that's not clear to me though, the blog post mentions hosting, and then goes on to mention various components of the stack, Search, MQ etc. Surely choosing an existing PaaS won't give you all of those things, so is choosing a single stack (you *must* use this MQ, the preferred DB is X) also part of the work being undertaken with this?
Comment by Anna Shipman posted on
Thanks! And good question.
You're right, a PaaS wouldn’t give us the entire ecosystem, so we’ve already begun exploratory work around offering the additional components as services. We're approaching what components we'd offer based on user need. So if the majority of people we’ve engaged with are using a particular technology (Postgres, for instance) then delivering Postgres as a service would be the first database we'd target. We’d then prioritise adding other databases based upon user need. The same goes for all other services.
Comment by Tom Padgham posted on
Very interesting challenge you have ahead of you and I'm sure any successful solutions will be welcomed by many digital service teams in the departments.
Can you shed any light on how you might overcome the (as yet unresolved, I believe) issue of hosting sensitive (OFFICIAL-SENSITIVE or above) services/parts of services and associated data in commodity public cloud environments?
Comment by Carl Massa posted on
Thanks for the question, Tom.
The security requirements for systems with data classified at OFFICIAL-SENSITIVE (or even OFFICIAL) will vary depending on the data, but the Cloud Security Principles offer guidance on how to design and or build solutions that are appropriate for data classified at those levels.
A PaaS does mean that we have to consider the needs of many different systems, but I believe there are approaches we can take that would satisfy the requirements of the majority of applications we’ve profiled as part of user research. And for the outliers, where the system includes a dependency on components that will never be deployed in a public cloud (like a specific Hardware Security Module or something similar), I’m confident we can find solutions to integrate those systems with a PaaS too. Admittedly though, anything at SECRET or above is out of scope for this exploratory work.