https://gds.blog.gov.uk/2013/10/09/testing-with-users-around-the-world/

Testing with users around the world

A couple of weeks ago, the team who worked on the visit visa alpha for the Home Office came in to GDS to talk about how it went. We asked Katy Arnold and Chris Atherton to explain a bit more about how they’ve found testing the service with users around the world.

Why did we go to the effort of testing our prototype with dozens of people all over the world? Basically, we did it because they are our users. Around 1.5 million visitor visas are issued every year and we want to make the best possible customer journey for people coming for a lovely holiday in the UK.

Testing all over the globe

We have a network of staff working on visa operations in embassies all over the world, and it was these people we turned to to help us find participants. The staff were really enthusiastic about the work we were doing and went out of their way to help us by talking to staff in overseas visa application centres.

We tested a lot in Kuwait, quite a bit in Turkey, and a little in Australia, Thailand, and the UAE. That often meant late nights, staying up into the small hours so we could be online when our colleagues started their days.

We’d use screen sharing software to start a meeting with our contact at the application centre, then ask them to approach applicants in the centre to see if anyone had any free time. We wanted to create a space in which users felt comfortable, and so asked staff if we could use a quiet room in the application centre and give people headphones and a microphone — this encouraged a sense of privacy. Each participant would then go through a short application using our prototype and responding to what they were seeing on screen.

We were really dependent on the goodwill of customers, and it sometimes took a while to find someone. The longest we waited was about forty minutes; that was during testing in Kuwait, during Eid. Not unreasonably, it took some time to find people happy to test with us who had been fasting all day in 50 degree heat.

Our research went really well, and we also had a few funny things happen. One participant walked out of the room in the middle of testing — we just heard her footsteps as she left. Another participant’s phone kept ringing, until she ended up having to leave too. Some things you can’t control from the other end of a screencast.

What users told us

Unsurprisingly, language proved one of the biggest areas for feedback. We already knew that we had to be very clear and specific in our instructions, but occasionally the feedback was more about cultural differences. A good example was a case where we’d described the priority service as ‘jumping to the front of the queue’ — one gentlemen in Kuwait told us, ‘I’m sorry, but here that description would be considered very rude’.

Feedback from testing

The whole team was surprised by the huge range of people and nationalities brought in to use the service. It seems obvious now that ‘testing in Turkey’ isn’t the same thing as ‘testing with Turkish users’, but we hadn’t really considered that at the outset, and yet it kept happening.

A gentleman from Iran took part in our research with Turkish users. English wasn’t his first language, and as an Arabic speaker, he was used to reading from right to left. At the time we were using a left-hand navigation that we weren’t sure about. His reaction was completely unexpected: he tried to read across from each navigation title to each field in the form, even though this wouldn’t have made any sense. This gave us the push we needed to completely redesign the navigation, which we might not have done as early otherwise.

Quick feedback on ideas

We could have paid a recruitment company to arrange formal testing, or flown to each location, and while each approach would have produced valuable results, it would also have been expensive and time consuming. What we really needed in our alpha was quick responses to the ideas we were testing.

We also repeatedly ran into technical obstacles. Sometimes the application centres would have firewalls set up which stopped us from using screensharing. Thankfully, we usually found ways around it. Some of the local staff came to the rescue by bringing their own laptops in for us to use in the testing, which we really appreciated. Huge thanks to everyone who helped us out.

As the beta starts we’ll have to pick our audiences carefully. Initially we’re testing closer to home — with the help of colleagues in public enquiry offices, where applicants already in the UK can come to apply for a visa — but we’ll definitely be talking to international users again. We’d also really like to see some testing in China, Russia, South America and Africa.

In the meantime, bring on the beta!

Follow Katy and Chris on Twitter: @katyarnie and @finiteattention

1 comment

  1. Bex Tindle

    Please do test in Africa! My husband is from Guinea and we had to apply for a visa via the Embassy in Sierra Leone. He had to travel around for about a month with almost £1000 in his rucksack due to having to pay for a visa in person and in cash. £1000 would be a muggable amount in this country - in Africa it's a small fortune. This may be an issue for the UKBA or equivalent rather than you guys but the current system fails in a huge way to the extent that it puts individuals in danger.

    Link to this comment