As you know, today we launched the Government Digital Strategy. I’m part of the team that was tasked with putting these documents online, and from the outset we wanted to do it in a much more comprehensive way than simply uploading a PDF or a Word document. I’m going to explain a bit about how we put it together, from the perspective of a developer.
Making documents useful
We wanted the documents to exist online in a form that was both useful for readers and simple for the authors to work on collaboratively. We quickly agreed that sharing Markdown files would be a good approach.
Our first port of call was Jekyll, which is used at GDS extensively for prototyping sites and products, but it wasn't quite right for what we needed here. While brilliant for turning Markdown files into a blog, Jekyll was going to struggle with our fairly complex documents. We turned instead to Kramdown, a Markdown parser written in Ruby which supports some useful extensions. Kramdown helped us as it:
- parses things Markdown doesn't, including tables
- automatically generates a table of contents from headings on the page
- automatically generates IDs for headings
- gives us the ability to mark up content and give it extra classes or attributes
Publish, don’t send
The next question was how to share and manage the source documents. As a developer, my obvious first thought for anything to do with version management was using Git, with Github for sharing in public. We didn't want everyone to have to learn how to use Git, but thankfully Github's online interface allows Markdown files to be edited and committed from the site, eliminating that concern.
It was an interesting experience to see so many different people using Github's interface to update the content, from hardened developers like myself to policy experts who were totally new to the service. There were a few bumps along the way, which was expected, but overall it was a really smooth experience.
Having these documents online and in the open is great, and the fact that Github keeps a history of these documents is good for transparency; over time, anyone can track the progress of the documents.
Assembling the strategy
Our build process is quite simple. Each document lives in its own folder as a series of Markdown files. We then build these by merging them into one Markdown file, running that through our very own Markdown pre-processor (which does some extra formatting), before they are finally run through the Kramdown compiler and outputted as HTML. I also built a very simple templating and partials system to avoid repeating content. All of this was integrated into a pretty straightforward shell script, meaning anyone who wants to see how it looks can simply run a small shell script and get everything generated for them. The script also handled our assets and produced a ‘built’ folder with everything neatly packaged up.
We used Sass (the "SCSS" variant) for styling, and RequireJS to manage our JavaScript. JavaScript was also used for the charts on the site, along with some clever CSS by Tim which turns regular tables into responsive bar charts.
This system was produced in just under three weeks. Whilst it’s very much geared to the documents published today, our next job is to extract the system into its own package that can then used by different teams for different documents and open-sourced onto Github.
There's a fair bit of refactoring and reworking to be done, as there is on any system written in a short amount of time, but we're looking forward to seeing what uses people find for it.
30 comments
Comment by Joe Walker posted on
I still don't get it, with this markdown hypes I am starting to feel back into 1997 flatfiles...
Comment by Changes to the strategy | Government Digital Service posted on
[...] few weeks ago I wrote a post detailing the technical side of the Government Digital Strategy. One of the things I spoke about was how we were keen to keep [...]
Comment by Boot up: Apple’s real problem, how GDS did it, Nexus 4 and 10 benchmarked and more | News & Specials posted on
[...] Shipping a digital plan Government Digital Service [...]
Comment by Designing for the long read on Inside Government | Government Digital Service posted on
[...] GDS we intend the web to be the primary platform for publishing things like policy, speeches and detailed guidance, with any print documents taking a supporting role. As [...]
Comment by A little blog on the side – Inside Inside Government | Government Digital Service posted on
[...] to keep these stakeholders informed about. We could be meeting this need by email, but we much prefer to publish than to send, [...]
Comment by This week at GDS | Government Digital Service posted on
[...] about their experiences producing it, but it’s been particularly good to see write-ups from developers and policy specialists, who worked together to make the strategy part of the web, not simply on the [...]
Comment by Publishing the Digital Strategy as a website, rather than on one | Government Digital Service posted on
[...] the traditional gulf between ‘policy’ civil servants and the designers and developers. As Jack said in his blog post, we had to work together across disciplines to ensure that the strategy would be presented in a way [...]
Comment by Tim Davies posted on
A few quick usability thoughts from reading the documents on the web and as PDF:
- Contact information - it would be good to have a clearer statement of who is responsible for the document; and what it's status is.
- Interactivity - the web format feels like it should invite interaction (comment thread etc.) but no feedback means is offered.
- Action perma-links: It would be useful to have clear 'link to this' links for each of the actions, as some of the diagram/tables have in the research report, so that it is easier to find the right permalink to point someone direct to them
Comment by Tim Paul (@timpaul) posted on
Thanks for this Tim. Making headers linkable is on our list. We've looked at two options so far; adding 'link to this' text for each header, or updating the URL automatically as you enter each section. There are pros and cons to both - unfortunately, deep linking into web documents is not something that people outside the blogosphere are familiar with, so making the feature obvious is a challenge.
Comment by Ryan posted on
Just for clarification, you mentioned Github "for sharing in public". I'm sure you had a paid account though, with an Organisation *private* to members, right?
Otherwise, what would you be agreeing to in terms of legal status / copyright of government documents by using Github's free service, usually reserved for open-source content?
Comment by Roo Reynolds (@rooreynolds) posted on
We do have a GitHub organisation account (https://github.com/alphagov/) which helps us manage teams, etc
While we were developing this strategy, we used Git (and GitHub) privately. Once we were ready to publish, there was no issue with it being a public open source project on GitHub.
Comment by Ryan posted on
I see - sounds like quite a commitment on the part of civil servants. I imagine there are lots of things in the early drafts of some types of document, even ones like the GDS which are clearly destined for public consumption, that could be inconvenient to have on the public record. Radical openness indeed!
Comment by Roo Reynolds posted on
Sorry Ryan, I should have been clearer.
Once we were ready to publish, there was no issue with the finished version being a public open source project on GitHub. All the updates and corrections since we published on Tuesday morning are public for all to see: https://github.com/alphagov/government-digital-strategy/commits/
(That in itself is quite a step forward! We're working on making the link between the documents themselves and the history in GitHub clearer, so people can track changes easily.)
However, the working drafts before we published were an internal work in progress and not something we'd automatically publish. The guidance around the FOI act puts it quite well when it talks about ensuring "there remains a safe space within which the formulation and development of government policy and government decision-making can proceed, balanced with proper public participation in policy debate" [http://www.justice.gov.uk/downloads/information-access-rights/foi/foi-exemption-s35.pdf#page=14]
Comment by ryje posted on
Ahh, I see now what you meant, thanks for the clarification. Still, a big step forward, as you say 🙂 I'm interested in showing how legal texts change over time; will be interested to see how you end up displaying that kind of longitudinal information for the public.
Comment by Boot up: Apple’s real problem, how GDS did it, Nexus 4 and 10 benchmarked and more | Nur, was da steht posted on
[...] Shipping the digital strategy >> Government Digital Service [...]
Comment by Iain posted on
Like John Smith# I'm wondering why not use one of the many CMS packages around? Very clever, but a case of reinventing the wheel?
Comment by Michael Allen (@MichaelDFAllen) posted on
I thought the post explained that pretty well:
" Having these documents online and in the open is great, and the fact that Github keeps a history of these documents is good for transparency; "
The text is in a easily accessible format that anyone can parse / link to and it's all versioned by GitHub so everyone can see any changes that are made.
A regular CMS wouldn't allow these types of openness.
Comment by S Moffitt posted on
That's not completely true. Versions can be exposed without too much difficulty on most CMS'. The DH/NHS Space for Health, for example, has versions of all its previously published guidance available, along with the current version.
Comment by Boot up: Apple’s real problem, how GDS did it, Nexus 4 and 10 benchmarked … – The Guardian (blog) posted on
[...] Shipping the digital strategy >> Government Digital Service [...]
Comment by Boot up: Apple’s real problem, how GDS did it, Nexus 4 and 10 benchmarked and more | IT Support London | SupportWizard.net posted on
[...] Shipping the digital strategy Government Digital Service [...]
Comment by Boot up: Apple’s real problem, how GDS did it, Nexus 4 and 10 benchmarked … – The Guardian (blog) | Hani Dalgamouni posted on
[...] Shipping the digital strategy >> Government Digital Service [...]
Comment by Karl posted on
"There were a few bumps along the way, which was expected, but overall it was a really smooth experience."
What were the bumps?
What were the solutions, strategies? put in place.
(if you could expand on these in another blog post that would be great. Thanks for sharing)
Comment by Jack Franklin posted on
Hi Karl,
Thanks for your comment. There were no serious bumps but more minor issues. For example, the Github interface for editing Markdown files would often not let you save if others had edited documents in the same folder. At times when 3-4 people were all editing the content, this caused some issues. We probably pushed the Github interface for the editing a bit further than it was ever designed for. However, it still saved us time with some of our internal users who had not previously used Git.
What we actually ended up doing is having a lot of people pair up for editing and doing things together, which was really successful. After the first week or so of working with Github, everyone had settled into working with it and we didn't have any problems after that. And whenever something did go wrong, we were able to revert back or rescue the old content because it was all stored in Git.
Hope that helps - if you have any more questions feel free to leave another comment.
Jack.
Comment by gilwoodcs posted on
It does look very impressive .. but a PDF or word doc would be handy so it can be saved on a reader and read offline. Have you made a design decision that overrides a practical user need?
Comment by James Stewart posted on
It's all one HTML document so should work well offline, but there is also a PDF version available at http://publications.cabinetoffice.gov.uk/digital/strategy/government-digital-strategy.pdf (the link's a bit subtle but is there at the end of the table of contents)
Comment by Jack Franklin posted on
Thanks for your comment! As James said there's a PDF link in the document but clearly it should be more clear - you're not the first to ask for the link after missing it. We'll be looking into making it more obvious.
Comment by gilwoodcs posted on
cheers
Comment by Jack Franklin posted on
We've updated the documents today and the PDF link is now a bit more obvious - thanks for your feedback.
Jack.
Comment by John Smith posted on
Brilliant. Now all you need for this tool is a name. How about Content Management System?
Comment by Ben posted on
Very clever, yes, but I think the clever point from Jack is this "This system was produced in just under three weeks. Whilst it’s very much geared to the documents published today, our next job is to extract the system into its own package that can then used by different teams for different documents and open-sourced onto Github."