https://gds.blog.gov.uk/2014/07/22/making-things-open-making-things-better/

Making things open, making things better

Every time you save a document on your computer, there’s a short string of characters at the end of the filename - the stuff you see suffixed after the dot.

Those letters perform an important task. They tell the computer what format the file is saved in, so that your chosen application can open it. So if your computer sees a file with a ".pdf" suffix, it knows it's a document, and opens it in an application that can read it when you double-click.

Today, we're making an announcement about the formats government uses by default. Government documents will use what are known as open standards for document formats. Word processor files will be saved with ".odt" suffixes, rather than ".doc". It's a different format, but it does a similar job. These formats are open in the sense that you don’t need any specialist software to use them. If your existing software doesn’t understand them, you can download software that does for free.

We're making this switch because we want:

  • users to have a choice about the software they use to read government documents
  • people working in government to be able to share their work more easily (we think sharing is a good thing: one of our design principles is "Make things open, it makes them better")
  • to make it easier and cheaper to do business with government (no-one should have to pay for specialist software just to send us some information)

This isn't a decision we've taken lightly. We've spent a lot of time recently asking for feedback from the people who are most likely to be affected. Responses on our Standards Hub (over 500 of them) were overwhelmingly positive. Thanks to everyone who took the time to comment.

This is a big step for government, and things won't change overnight. We have to make sure that the switch is managed properly. We shall work with departments to make the transition as smooth as possible, and ensure that the burden stays with government and not users.

You can read more details about:

And we’ll keep blogging about our progress in implementing open standards on the Government Technology blog.

Update: I made an error in this post, conflating the concepts of file suffixes and media types. My aim was to simplify things for the general reader, but the experts among you have (quite rightly) pulled me up on it. The purpose and behaviour of file types, MIME types and file extensions is very complicated and varies depending on the context and the computer you're using. For those readers who want to find out more, start at this Wikipedia page about Internet media types and go from there. Apologies - and I hope the error doesn't detract from the more important message about open standards.

Follow Mike on Twitter, and don't forget to sign up for email alerts

27 comments

  1. Mark

    Hooray! From such acorns may a mighty (open standards) forest bloom! (Do forests bloom? Perhaps the forestry commission can help)

    Link to this comment Reply
  2. Eduardo

    Congratulations from Zaragoza Spain. You are in the right way

    Link to this comment Reply
  3. Angus Marshall

    Errr. No. The file extension is a hint - it's actually the first few bytes of the file that identify the type. Try looking up "magic numbers" for more info.

    It's perfectly possible to save ANY file with a .odt extension - but that won't change the contents.

    I applaud the initiative, but please get your information right.

    Link to this comment Reply
    • Dr. Jerry Fishenden, Deputy Chief Technology Officer at GDS

      Thanks for your comment, Angus.

      Technically you're correct, of course, that a file's internal metadata can be used to determine the format used for that file. The file extension illustration is used in the blog since it is what most users will be most familiar with (e.g. .doc for binary Word documents, .pdf for PDF documents and so on) - and we're trying to use language that makes these changes easiest to understand for general users.

      Link to this comment Reply
    • Tom Davies

      Hi :)
      Using
      File - "Save As ..."
      is the best way to convert a file from one format to another in almost any program. In MS Office 2007 the "File" menu was replaced by a golden-globe but does the same job. I've not yet found any program that doesn't do it that way but inevitably there is always one that has to be different, hence my disclaimer!

      I have used and taught MS Office (almost all versions), LibreOffice/OpenOffice and simply used (not taught) Kingsoft, Caligra/KOffice, AbiWord/Gnumeric and many others.

      Angus seems to be over-complicating the issue by adding things that are not strictly correct anyway or has just enough truth but takes it sideways and spins it around until it becomes just plain wrong.

      For example files can be renamed but in Windows it is not easy to rename the file-extension. Users are protected from doing so accidentally. If the file-ending doesn't match with the identifier handshaking coding Angus points to then most platforms generally refuses to open the file and Windows just can't cope. So the hidden coding would also have to be edited and that is even tougher and there are incidental protections that a cracker could easily forget.

      The main attack-vector that is difficult to protect against and fairly easy for crackers (with a shockingly low level of skill) to use is macros written specifically for MS Office.

      Another route might be to hide coding within formats that are not strictly open. There are some formats which claim to be open but are made to hide binary blobs. A skilled attacker might be able to place their own blobs. Such formats are often called "transitional".

      ODF doesn't have any need for such transitional formats. Any program failing to implement ODF properly is just plain wrong and it is usually possible to post bug-reports and/or to complain about it to a neutral third-party international organisation, and to the ISO people.

      Regards from
      Tom Davies

      Link to this comment Reply
  4. Angus Marshall

    P.S. - misusing extensions is a common vector for malware, particularly by email. It should NOT be encouraged and extensions should not be relied upon.

    Link to this comment Reply
  5. Denny

    Please to hear about this. Although this bit caught my eye:

    "no-one should have to pay for specialist software just to send us some information"

    ... I wonder when that news will reach HMRC.

    Link to this comment Reply
  6. Gibbs

    About time in regards to the ODF's, still a great move.

    Outside of Windows, in most cases, file extensions are used by humans - not the computer (to add to what Angus pointed out). Its perfectly acceptable to have no file extension at all.

    Link to this comment Reply
    • Tom Davies

      Hi :)
      Windows needs file-extensions otherwise it easily gets confused. Also if you deliberately rename a file so that the file-extension doesn't match the clues in the first few (or last few) bytes of a file then again Windows can become confused. Luckily Windows helps guard against this by hiding the file-extension so that users don't get too much information and potential for confusion.

      Tablets, phones and other non-Windows platforms are the only ones able to cope with a lack of file-ending or a mismatch and even they can't always cope with it. However they are also far less likely to suffer from malware anyway.

      Apparently tablets, phones and other non-Windows platforms now accounts for over 80% of computer usage.

      It's difficult to be precise and easily understandable at the same time. I think the original article did a magnificent job. it didn't compromise as much on accuracy as other people have stated and i am not sure why they are trying to confuse the issue.
      Regards from
      Tom Davies

      Link to this comment Reply
  7. Steve

    Thanks for doing the right thing when others have not. This is massive. Excellent work. :)

    Link to this comment Reply
  8. Jon

    Good work, finally a sensible decision to move to open standards

    Link to this comment Reply
  9. Mark

    Perhaps Angus should try renaming an HTML file to .txt and seeing how his OS handles it. "Hint"? Not so much.

    Link to this comment Reply
    • Tom Davies

      Hi :)
      Nicely said Mark! Much more succinct than my rambles! :)

      Of course there is a difference in that html is meant to be able to be read as txt and web-designers often do edit the html as plain text as it helps see what is really going on. However the page does look very different! Different enough that people can probably see the fallacies in Angus' remarks.

      Regards from
      Tom Davies

      Link to this comment Reply
  10. Daniel Llewellyn

    Another particular point I want to highlight is HMRC currently have several "webcasts" available on their self-assessment programme which are recorded in WMV (Windows Media Video). This means that I on a Macintosh or occasionally on Linux or other Operating Systems than Windows cannot view these webcasts. Will the GDS be pushing for more open formats in the wider environment, such as video and audio, in addition to the "Office Documents" highlighted by this post? An important question, which I hope to receive an answer.

    Link to this comment Reply
    • Dr. Jerry Fishenden, Deputy Chief Technology Officer at GDS

      Dear Daniel, thanks for highlighting this issue.

      We have a process in place via our Open Standards Hub (http://standards.data.gov.uk/) where anyone can lodge a challenge. This open process is designed specifically to enable users of government services to flag challenges they face that open standards can help to fix and this would be a good place for you to flag the problems you are encountering.

      Link to this comment Reply
  11. Nick Aurelius-Haddock

    This is excellent news, and not before time. Getting people to pay a Microsoft tax just to read a document from our own government has always been ridiculous. Now, start on wasting money on all proprietary software that the government just doesn't need to be using. Oracle, Microsoft, VMware, Citrix etc etc etc.

    Link to this comment Reply
  12. Martin Lugton

    Very cool. Does this correspond to changes in software procurement too?

    Link to this comment Reply
  13. Paul

    Heartiest congratulations!

    I think a state in India Kerala recently took some bold steps in securing their documents and supporting ODF as well.

    Please do provide links to FOSS software (LibreOffice & OpenOffice) that easily opens & edits files in this format. This is necessary because most people may not be that tech savvy & your helpful info could help. :) Cheers!

    Link to this comment Reply
    • Tom Davies

      Hi :)
      There are too many programs to link to them all!

      That is part of the point! You can choose to pick, at random(!) any program that uses ODF or to follow procurement "best practice". Then any time in the future you could choose to change to using any other program that supports ODF and STILL be able to open any existing ODF document.

      Different departments could use different software or different versions of the same software and still be able to read each others documents.

      This is NOT about restricting choice of programs! It is NOT about FOSS vs proprietary. Nor is it about moving away from MS Office. Quite the opposite! MS Office has an option to use ODF and during the standard installing procedure can even be set to use ODF by default.

      Versions of MS Office prior to their MS Office 2013 seem to have trouble, particularly with spreadsheets, but it's easily possible to use a powerful, free program specifically for that (Gnumeric is one of many good ones) until MS Office can be upgraded to their 2013 (or replaced)
      Regards from
      Tom Davies

      Link to this comment Reply
  14. Steve Kidd

    For years I have been banging on about the slavish devotion to purchased products.

    I am not sure about the politics of such decision making, nevertheless kudos to those who have managed to push through such an eminently sensible measure

    Link to this comment Reply
  15. Benno

    Great move! I hope seeing other countries following your lead.

    Link to this comment Reply
  16. Brian Kelly

    In this policy statement the Government is doing more than mandating use of open standards: it is mandating specific open standards, namely ODF and PDF.

    However that statement in the what publishers to GOV.UK need to know document that "Once open publishing standards are adopted in full by your organisation, no documents should be published in proprietary formats" is disingenuous as it doesn't address other open standards, in particular ISO/IEC 29500. This ISO standard is better known as Office Open XML (OOXML) and is based on Microsoft's office formats which were ratified as an open standard by ISO, just as Adobe's PDF format also became an open standard.

    There may be legitimate criticisms of OOXML (it is complex. for example). But it would be wrong to describe it as proprietary. What would the response be to government departments which state that they intend to make their documents available in ISO/IEC 29500 format?

    Link to this comment Reply
  17. Lawson Davies

    It makes me wonder how the considerable investment in Visio and Visio files, templates and stencils to document technical views (particularly of government IT systems and services) is going to continue to be leveraged by colleagues constrained by this policy to use ..odg and those still using Visio given that I don't believe Visio supports .odg and I know of no drawing tool with .odg support that also supports saving to Visio.

    Link to this comment Reply
    • Jon Lane

      Reading Visio files is usually straighforward for non-Visio users who are able to convert them to .pdf files online (if security marking permits!). Libvisio is a (free) means of converting .vss to .odg, albeit not yet a mature one.
      Microsoft will support open standards, but only when pushed, and this decision represents an effective push. Perhaps it should focus on producing the best tools for those who don't mind paying - there's always a market for premium products - rather than trying to own the format of what they produce.

      Link to this comment Reply
    • Tom Davies

      Hi :)
      LibreOffice can handle Visio files and stuff. More will follow now that 1 can do it. MS Office can handle ODF and Visio files.

      The change-over is not expected across the board and over-night. Word processing, spreadsheets, presentations are likely to start using ODF sooner than certain other types of files.

      Also the same problem exists if you stay with Visio. The older templates and stuff will increasingly be found troublesome. Older Visio files my not even open in newer versions of MS Office and especially in future versions.
      Regards from
      Tom Davies

      Link to this comment Reply
  18. Benjamin Rusholme

    This is excellent news.

    Link to this comment Reply

Leave a comment