https://gds.blog.gov.uk/2012/10/12/coding-in-the-open/

Coding in the open

We talk a lot about Open Source at GDS. GOV.UK is built on open source software and, to a degree, built as open source software. It’s a topic we care passionately about because it helps us maintain our focus on user needs by helping us to quickly test and iterate software and systems.

The software world has a rich and growing range of tools to build your own software on. From operating systems such as GNU/Linux, web servers such as Apache or nginx, databases like MySQL, Postgres or MongoDB, the tools are there so we don’t have to do it all ourselves.

The challenge is often in knowing which of those tools are right for the job you’re focussed on.

Solving problems with the right tools

By choosing open source we can quickly try the tools that seem right, test them out, and make a decision about whether this is the right solution and start using the software straight away. A good example of that is the Elasticsearch search engine that Rob wrote about a few months back. From what we were reading it seemed like it’d be simpler for us to work with than solr, and we were able to dive in and try it out without a lengthy pre-sales process or costly and complex licensing arrangements. Instead, we test, we decide, we get on with tailoring our choice to best serve the user needs we’re focussed on.

There are open source tools that cover most of what we’re doing. We can analyse which are going to fit together in the ways we want, and where we should write our own code to make sure we’re building the best product we can in a way we can maintain. One of my fellow GDS architects, Paul Downey, has often likened our approach to that advocated by JP Rangaswami in his “open source pyramid”:

For common problems use Opensource.
For rare problems use Buy.
For unique problems use Build.

Make things open: it makes things better

We’re also making our code available for others to use. We tend to talk about that as Coding In The Open rather than Open Source. Partly because we’re not currently in a position to build and support a community around that code, and partly because most of it’s so tailored to our system that we’re not sure how useful it will be to others right now. But rest assured, if you see something useful you can use it – it’s published under an MIT licence.

We don’t want to jump to conclusions about what people will want to use, but we’re eager to hear from other people who might have overlapping needs. Together we can work out which components would be good to extract and package up in a way that makes them easier to share.

There are, of course, a few pieces that were so clearly standalone components that we think they’d be great for others to use. In his first week with us Nick Stenning put together unicorn_herder to help us manage the unicorn application server that our ruby applications rely on. It’s one of a few examples you can find in our github account.

13 comments

  1. Tim Manning

    This is the beauty of open source – the freedom to explore and experiment!

    Reply
  2. Vine

    Coding needs to be more simple to follow. The only people who consider coding is easy are people who work in the industry itself. Good blog.

    Reply
  3. Blogging for Queen and country « Matthew Sheret

    [...] worked. Since the start of October there have been 23 posts published, covering topics as varied as what it means to code in the open, redirections, how the Finance team work and a load more. For the writers, it takes just a little [...]

    Reply
  4. GOV.UK offers best practice insights » b.c.s.

    [...] Coding in the open, technical architect James Stewart said open source helped the team “focus on user needs by [...]

    Reply
  5. Jonathan

    If you need help with your MySQL databases for government projects, I can volunteer my time to help.

    http://www.jonathanlevin.co.uk/p/volunteering.html

    Reply
  6. Tools over Content | Government Digital Service

    [...] talked before about various aspects of how we build software at GDS, and especially for GOV.UK, but I’d like to take a step back and about one of our [...]

    Reply
  7. Making tools for makers | Government Digital Service

    [...] some pieces we can’t share in public yet, but in the coming months we’ll be working on opening up whatever we [...]

    Reply
  8. John Fraser

    We’re building a new dev team here at Registers of Scotland. (Thanks for this resource, which gives us good ideas.) We would like to use, develop – and therefore contribute – open source wherever appropriate. Do government guidelines encourage contributing open source? (I’m assuming so, as you have a github account.)

    Reply
  9. The UK government pays me to write open source all day | Quick People Blog

    […] point where I have perhaps exaggerated: as James Stewart, one of GDS’s technical architects, points out, what GDS does is actually “coding in the open”, rather than “open source” […]

    Reply
  10. Kevin Mitnick

    How are security concerns met where open source software isn’t widely distributed and subject to the ‘thousand eyes’ argument?

    How is provenance and assurance of the software used gleaned?

    Reply
    • James Stewart

      Hi Kevin – sorry for the exceedingly delayed response. Where software isn’t widely distributed the onus is very much on our development team to do due diligence, reviewing the code, making sure it’s subjected to testing and working out how to keep track of changes. That’s everything from tracking github repositories, to building relationships with the authors, to detailed penetration testing. Depending on how and where we’re using the code that’s an important consideration as we consider the cost of ownership.

      This is also one of the reasons we’re cautious about how we frame the release of our own code. There will be some pieces of code where we can commit to all the support that we would want ourselves, including defined ways of advertising new releases, disclosing vulnerabilities, etc.

      Reply
  11. MarkDalgarno (@MarkDalgarno)

    Wondering why you chose the MIT License rather than the Open Government License.

    Reply
    • James Stewart

      MIT felt more appropriate as a license for code as it’s designed for that purpose. It’s also more widely recognised in the software world and so makes it easier for other developers to make decisions about whether or not to use our code.

      Reply

Leave a comment