10 comments

  1. David

    As a content writer for another Government website this phrase chimed very much with me: “Another key point: we wrote the script within our digital communications team. Previous scripts have ended up being written by committee. It was inefficient, unproductive, and didn’t work for the customer.”

    I think it is a symptom of the civil service that everything be done ‘by committee’ often with terrible results. So to read that this is being challendged and successfully too is fantastic!

    Reply
  2. Keith Emmerson (@KeithEmmerson)

    Great video!
    Thinking about what the user is searching for is a great leap forward when providing these resources, kudos for that.
    However, would a more accurate title be something along the lines of ‘what happens at your theory test’? ‘How to pass your theory test’ suggests (to me) that it might contain tips on revision or basic principles to remember.

    Reply
    • jploughman

      Hi Keith

      Thanks for the great feedback. One of the key things we wanted to answer in the video was ‘how to pass’.

      One of the first things we say is that to pass, you need to be well prepared, by using the learning resources available. There are no tricks, as it were. Also to pass, you need to get the scores mentioned. At it’s most basic level, that’s how to pass.

      We’re planning on doing something separate around preparing for your theory test, where we’ll be optimising for those using ‘revise’ as a keyword. At the point, we’ll use YouTube annotations to link the 2 videos together.

      As I say, we’re still learning. We’re encouraged by the early results of the new video – but there’s definitely things I’d do differently if I were to make it again tomorrow. We’ll use that learning in our next video.

      John

      Reply
  3. ‘Harder, opaquer, slower’ | Digital by Default

    [...] recent post of the GDS blog has someone from the Driving Standards Agency talking about how they took that phrase and other lessons GDS have been sharing and rethought their [...]

    Reply
  4. Hannah Goraya

    Hi,

    Great to see the improvements being made. My comment is more about the video than the general learning within the DSA so if this isn’t the appropriate please delete it and ping me an email.

    Is there any indication that people are searching for ‘What happens in a theory test’ or ‘How to sit your theory test’ because that’s what the video is and might explain why 43% still aren’t watching to the end.
    There’s nothing about what it actually takes to pass the theory test, such as revision techniques or how to incorporate what you learn from the text book in your driving lessons with the instructor so that you can understand the theory test better. I think people will be looking for that as well as what actually happens, as it’s not much good sitting an exam if you haven’t prepared for it.

    Also, the point in the video where it describes the 15 minute practice test (1:06) didn’t seem that helpful, as you didn’t say why someone would bother doing that, will they learn what areas they need to improve? Has research shown that some people find this helps them to become less nervous in the real exam? etc. I know you were trying to keep it short but there must be a reason why you offer the practice test that could be explain in under 15 seconds. Was it in the original video and taken out?

    Final question, you mention that more viewers are watching the videos on their mobile devices, are the figures available publicly? Just about to go and have a look at data.gov etc but if you can point me at them, that would be much appreciated.

    Thanks,

    Hannah

    Reply
    • jploughman

      Hi Hannah

      Thanks for your really great feedback – and some challenging questions :)

      One of the first things we say in the video is that to pass, you need to be well prepared, by using the learning resources available. There are no tricks, as it were. Also to pass, you need to get the scores mentioned. At it’s most basic level, that’s how to pass.

      We know that everyone has a different learning style, so trying to cater for each of those with tips in the same video could get confusing. We’re planning on doing something separate around preparing for your theory test, where we’ll be optimising for those people using ‘revise’ and ‘prepare’ as keywords. At the point, we’ll use YouTube annotations to link the 2 videos together.

      One of the reasons we think that 43% still aren’t watching by the end is that a lot of our YouTube subscribers are driving instructors, who will already know the information. We expect that they started to look, and then lost interest. The demographics of those watching the video to date seem to match our profile of driving instructors. We’ll keep watching the retention rate and seeing how it changes as we get more people coming to the video ‘naturally’.

      The part about the practice test is something I’d probably leave out if I were to make the video again. The reason it’s included in the test is for those who might not be as familiar with computers, and need the practice using the mouse and moving between screens. It’s another case of where I’d go further with optimising for the common case – most people know how to do this, and if they’re watching the video on YouTube, they probably don’t need that support when they take the test.

      To date, 22.8% of all views of this new video has been on a mobile device. Looking back at the equivalent period for the video it replaces, that figure was 0.5% of all views. The previous video was published on YouTube in February 2010, but existed for some time before that as a DVD. We don’t make these stats publicly available routinely at the moment, but it’s something I’d love to open up.

      Reply
      • Hannah Goraya

        Hi John,

        Thanks for taking the time to respond and for providing some stats, they were really interesting. Good point re: what it takes to pass being addressed in the film. I still feel their could be an element of what people expect to find when they search for that and what they do find being mis-matched, however people (like driving instructors) who are already familiar with the content searching for it to show someone makes a lot of sense. Look forward to seeing how your department’s work develops!

        Hannah

        Reply
  5. Rich Watson

    Hi John
    Great improvements.
    I think there are still some optimisation you could do. Every second you can shave off will help. For example the intro is around 15 seconds long. Let’s think what does the user want from this? They need to understand is this video going to benefit them and fulfil their need, is it for me?

    The video needs to sell itself in the first few seconds just like a web page does. Is it snappy, to the point etc or is it bland, slow and boring?

    Seeing a trusted title up front is great but the animations and transitions are not required. Frame one could start with the crest and title, therefore instantly offering reassurance from the start of who is offering this info, it’s trusted and stating what is it about. Then get straight to the point of what’s in it. I would be really interested in the user drop off rate from 5-14 seconds.

    The transitions are actually slow, movement of still images. I think offering very little meaning. 18 to 25 seconds also could be a lot snappier to get the same message across. All this offers a message, including the voiceover and music speed sets the tempo of a ‘long 3 minutes and 53 seconds of my life’.

    58 – 1.02. I wonder how much benefit the transitioned text on screen helps users Are people actually reading this?

    1.44 – 2.35 is ’51 SECONDS’ of showing the user how the system works. I would imagine this will be the biggest drop out for several reasons.

    1. This is the time most users want the video to start wrapping up yet it’s going into heavy detail of how a computer system works, remember they came here to find out how to pass a driving test.
    2. They don’t want to know at this stage how the system works. Showing types of questions and the interface is great to give an idea but the interactions should be intuitive. You shouldn’t need to have to explain it, especially not now!

    Again on the hazard perception there is no need to go into to much detail on how the system works, e.g. Don’t need to know about clicking too many times etc. Tell them on the day.

    It would be interesting to actually get feedback from users on the top five things users want out of the video and then tailor it to that. It would be great to see a 30 second clip that gives everything, less is more.

    Well the above is food for thought and already gets the video to under 3 minutes.

    I hope this quick heuristic review helps

    Regards
    Rich

    Reply
    • jploughman

      Hi Rich

      Thanks for some really great feedback!

      I absolutely agree with you on the introduction. It’s too long, and slows down the flow of the video. Next time we need to get straight into the content as quickly as possible.

      Interestingly, the biggest drop off in the shortest space of time (14% in 14 seconds) is the part about resources available to help very early in the video. This was something we debated within the team about including. It suggests that most people must already know this (or view it as a sales pitch), and aren’t interested in it.

      There’s a 13% drop-out in the section about how the multiple-choice part works (1:06 – 2:34).

      In the hazard perception part of the video (2:37 – 3:40), it only drops by 2% at the moment. This reflects roughly what we’d expect to see, as we know there is some misunderstanding of how that part of the test works, and it tends to be the part most people are nervous and unsure about.

      I’d love to get it down to around 2:45 in total. There’s definitely greatest scope to shave more from the multiple-choice part and the opening.

      The point about the music and voiceover is an interesting one too. Getting the balance right between being calm and reassuring while being punchier and pacier is one of the hardest things to do. We have a couple of videos in the pipeline, and we’ll be taking the opportunity to experiment more on the audio with those.

      Thanks again – it’s really great to get this feedback on how it’s viewed with non-DSA eyes and from people who are equally passionate about making it better!

      Cheers
      John

      Reply
  6. Joseph Jackson

    Its common knowledge this is all a money spinner for the Government so I dont think wasting money on more how "to" videos are needed as the click method is just pot luck.
    Maybe being able to click on a hazard with your mouse rather than having to gamble on a multitude of hazards put before you.
    Of course this would up the pass rate and minimise Government profits,so I'd imagine would never be considered.

    Reply

Leave a comment