User research is a large part of what we do here at GDS, but we aren't the only ones hard at work doing it, as this guest post from Katy Arnold, Head of User Research and Design at the Home Office, explains:
Here in the Visas exemplar, we’ve been working hard to understand our users. This has involved lots of user research, including:
- remote testing with people overseas
- visits to language schools in the UK
- meeting high volume users, like legal representatives who fill out forms on behalf of their clients
- usability testing in the same day service centre.
But one size doesn’t fit all when it comes to visa applications. We were due to release a Visit Visa for Chinese tourists in June 2014. We currently receive about 300,000 Visit Visa applications from China, mostly through our centres in Beijing and Shanghai. In total we have 12 Visa Application Centres (VACs) across the country; they process work and study visas as well as the more popular tourist ones. Unfortunately, we hadn’t had much luck finding users through our usual channels. So, we decided that if we were really going to get to know our users in China, we were going to have to go and see them.
Planning and preparation - lots of it
I can’t overemphasise the amount of planning involved before we went on the research trip to China. We were about to embark on 7 days of research over 9 days in a completely new environment. With people we’d never worked with before. Jet-lagged. In Chinese. As a comparison, the GDS standard is 1-2 days of research every 10 days. We really needed to know what was coming, but we also needed the flexibility to be able to react to what we learnt while there.
At home, we spoke to researchers across government who had experience of working in China. We also contacted research professionals who operate in China - such as at Dalian University - and carefully worked out a strategy that prioritised what mattered most.
We designed a methodology which would allow us to speak to as many ordinary Chinese users as possible during our short trip. Nearly 70% of visa applications are received through the Beijing and Shanghai visa application centres (VACs); many users travel from the surrounding countryside to these huge Tier 1 cities to make their application. To avoid wasting time on travel, given China’s enormous size, we decided to target Beijing. We made a point of communicating with users in Chinese. This meant recruiting experienced interpreters for usability testing. We also enlisted local embassy staff to help out.
Finally, we took advice from our departmental security units about and how we might gather and store insights while we were in China.
Day-by-day planning - with flexibility
We drew up a day-by-day plan. Early on in the trip we had clearly defined activities planned, with more flexible plans for the last few days of the trip, so we could respond to what we’d found on the scene. In preparation for the first day in the usability lab, we wrote scripts and shared them with the Chinese moderators and interpreters. We also agreed on how we would approach our first day of research in China with caseworkers (known overseas as Entry Clearance Officers or ECOs) and arranged for many of them to come along to observe our days in the lab.
What we did when we were there
We learnt so much in the time we were there. It’s impossible to cover everything in this post, but here are some of the highlights:
Inviting local staff to the lab days was a brilliant way of getting more out of the day. Having them on site meant we could involve them in debriefs between sessions. We could also immediately sense-check any assumptions we might have made about solutions. It also helped them to understand what we do.
Usability testing gave us solid evidence that some key elements in the design needed to change. We’d had our doubts about some aspects of the form and the way we were asking for some of the information, but what we saw over there was comprehensive. As a result we are about to release a significant change to the user interface - especially the review and summary parts.
Being in China in person let us recruit users who hadn’t been in contact with UK Immigration before, and let us observe them go through the Visit Visa application service in a natural way. Compared to what we’d seen remotely, user behaviour was much more representative of how they might behave with our service for real. Making sure we did the research in Chinese was a big part of this. We had a native Chinese speaking moderator, users typed into the form using pinyin and most checked dictionaries on their phones, or used Baidu Translate to give their answers in English.
Focussing on a site where high volumes of users came into contact with our service was a great way of gathering lots of insight from real users, very quickly. We spoke to 61 people. They were applying to visit the UK either as tourists, to visit family here, or to do business for a short time.
We used one page paper interview sheets which were portable, didn’t arouse suspicion - in either the participants or the security staff - and allowed us to go wildly off script if that was needed. We down noted down everything participants told us - verbatim - but still within the same broad areas. This gave us a brilliant data set, which we could use to build truly representative personas of the people who most often use our service.
Spending time with local staff in the visa section of the embassy was invaluable. It gave us our most significant finding of the trip. Seeing the problems our colleagues face daily during the busy summer season helped us understand just how important it is that we deal with the lack of clarity around what users should send in with their application. Despite being highly organised, the team in the embassy offices in Beijing are quite literally, being weighed down by paper supporting documents - originals and copies - endless piles and piles of them.
Third party users have a lot to teach us. We found out that the biggest local travel agents send send all their clients a list of documents which they think should be submitted with applications. It’s likely that lots of smaller agents get copies of this list and pass it off as their own - which feeds the vacuum we’ve created. Supporting documents are also their biggest cost.
What we did when we got back
Booking time with our service manager when we got back was a really good idea. It meant we were able to offload the important parts of what we learnt while it was still fresh. We included other relevant stakeholders, such as the customer insight team, policy and programmes working to reduce our reliance on paper. We are now all working together with the team in Beijing, to find a way to make the overall experience better for people applying from China.
Finally, something to say about the dignity, humanity and generosity of the people we met while we were there. Those who helped us run the research, who spared their time to take part, who showed us round the embassy and who shared ideas for how we can make things better. Thanks to all the staff at the embassy, to Xuejuan Yuan (Emma) our interpreter, and Haiyan, our moderator who were highly professional and flexible throughout. Also thanks to UK Visas and Immigration, for recognising that you can’t run a business with this impact without knowing who your customers are, what they need, and what makes them tick.
Follow Katy on Twitter, and don't forget to sign up to email alerts.
Transcript:
Apply for a UK tourist visa from China - user research case study
Home Office user researcher Katy Arnold talks about her team's recent visit to China to understand at first hand the experience of Chinese holiday visa applicants to the UK https://www.gov.uk/apply-uk-visa
Katy Arnold, User Researcher, Home Office:
The visa exemplar is a project to develop a new way to apply to come to the UK from China. This year they are on track to do between 300,000 and 500,000 applications.
We have feedback from our customer insight teams that people in China think applying for a visa is very difficult. We were really keen to start to understand why Chinese users supply enormous numbers of documents as evidence. When we walked around the embassy offices they’re literally drowning in paper.
We went to see the travel agents. Every time they take on a new client, they send them a list of documents. They have no incentive to limit that list. The difference is for users who want certainty around what documents to supply and how to negotiate the process of applying for an application, we’re not providing it, but travel agents are, but it’s just a very long list. They do at least say to their clients, “Send these in and you’ve got a pretty good chance of getting a visa.”
That was what was fascinating to us as well, to see a very clear reason for people using third parties to go through this process. The other big reason that they use them is that they don’t speak English. The difference with the beta service is it translates the entire service, users really appreciated that.
We actually have some very clear pointers from our trip that tell us which documents are being used in the decision-making process, so our recommendation will be to limit the list of documents to those and be very clear to Chinese users that they don’t need to send us the rest of them.
www.gov.uk/apply-uk-visa
