When we were designing and building our user research lab, we learned a lot about how best to build one. During the past 6 months of running our lab, we've learned even more.
In this post I’ll share our successes, what we’ve learned and where we could have done better.
Things to brag about
A very worthwhile investment
Within the first month of opening, our lab was being used to 70% capacity. Within 2 months, it was being used to 100% capacity.
In fact, because our lab is fully booked at least one month in advance, some GDS teams are once again using external labs. There’s no doubt this overflow could accommodate a second in-house lab space, even if much simpler and smaller.
The lab is an estimated 25% cheaper than going to market
By using an in-house lab, we’re spending an estimated 25% less than renting external labs. At 100% usage, our lab should have paid for itself in 3 years.
This includes all running costs, such as our full-time Lab Controller and things like stationery, biscuits and tea for guests.
User research is a team sport: our lab’s making this easier
Our lab is becoming more than just a lab space - it’s a space in which team members and senior managers (not just user researchers) come together to learn about users first hand. It’s also a place where people get to know one another.
GDS user researcher, Mark Branigan, says:
It's been considerably easier to get senior management involved when they only have to pop downstairs instead of across town. That's a major win. Making recommendations based on research is a whole lot easier when the decision makers have seen users interacting with a service first hand.
Also, it's far easier for observers to link up meaningfully with the rest of the team during research sessions, which has meant very rapid and effective fixes where there have been tech problems.
Liz Griffin, a GDS Content Designer:
The onsite lab is a great resource for anyone working on content (including new starters). It's also been helpful for members of the wider project team to be able to join us, as you learn things about your users from this kind of research that you can't get from web analytics, call centre feedback etc. You can learn a lot from dipping into a random session too.
Caroline Jarrett, a GDS user researcher, adds:
It’s thrilling to go in the lab to observe research and discover that it's already full of people; also a great way to meet people working on other aspects of projects.
Getting cross-government teams involved
Every Friday, we open our lab space to user researchers from across government who can use it free of charge. ‘X-Govt Fridays' are now fully booked almost 2 months in advance.
We’re starting to see cross-government teams drop in to observe research too, which is wonderful.
John Beale, a User Researcher, from Land Registry did just that:
...it was truly inspiring to see how hard the team are working to make every detail as intuitive as possible. This really is public service at its very best.
I found myself holding my breath like everyone else as the participant went through the screens, completing the revised photo upload section (bit of hesitation, notes scribbled) - would we get his full name? Yes! High fives all round.
A flexible space that accommodates many research needs
When we built our lab, one of our key goals was to create a flexible space - one that would suit a variety of needs; even those we’d not foreseen. For this reason, we designed the lab to run on a bespoke A/V setup and we don’t use proprietary research software. Because we understand the mechanics behind our lab, we can do whatever we want.
We’ve used the lab for various research methods such as forms studios, unmoderated usability research, cards sorting with paper cards, research with people working as a pair, research with people working individually, benchmarking and more.
We’ve dramatically improved the sound in the lab
The sound in our lab was ‘buzzy’ and ‘muddy’.
Jonathan Rodriguez, our Lab Controller (he’s also a qualified Sound Engineer), diagnosed our problem as “unwanted acoustic anomalies like flutter echoes”. These anomalies were treated with acoustic foam - seen below as large white canvases on the wall - adjusting of mics, and a frequency cut using the lab’s built-in EQ facilities.
Our sound is dramatically improved.
We’ve learned how to run our mobile testing more smoothly
Our mobile testing is set up to run on MirrorOp for Android and Apple TV for iOS mobiles. Our mobile testing capability could be improved still further but, for now, our setup feels as natural and uninterrupted as possible.
We’re now using Recordable for Android mobile testing to show interaction ‘snail trails’.
Although we’ve got really great ceiling cameras, we’ve learned that recording mobile usage with an external camera is tricky: hair gets in the way, reflections make the phone screens hard to see, and people don’t hold their phones still.
Even though we’ve got this very useful and team-building lab, it’s good to remember...
Yes, user research happens in the lab but no, that's not the only way to do it; it's only a small amount of the total user research. As we've got the facility it's great, but you shouldn't let lack of a lab hold you back. - Caroline Jarrett, GDS user researcher
Things we would do differently next time round
Involve the IT team from day one in the lab
We had meetings with our IT department when we first introduced the idea of the lab, but we didn’t follow through enough on their involvement.
If you want your lab’s long-term technical support to be delivered in-house, it’s important to get your IT department involved from day one, and keep them involved.
It’s usually possible to be supported by an external supplier, but it’s less convenient, may not be useful when you’ve got a last-minute problem, and it is more expensive.
Make sure you have an analog option for all your digital workflows
Digital is great when it works. Not so much when it doesn’t.
Our lab is primarily controlled using a digital console served via an iPad. The Lab Controller uses the console to remotely manoeuvre lights, sounds, cameras and what appears on the observation room screen. If the digital console won’t work - which happens rarely - the whole system comes to a standstill. Even if rare, we can’t afford any downtime in the lab.
Whatever your setup, it’s useful to make sure you have a readily available analog workflow as a backup to your digital one.
Sometimes bigger is better
Our observation room accommodates 12 observers and it’s regularly packed to the hilt. We could certainly do with more space.
It’s important to remember that an observation room is not just about cramming people in to watch a screen, it’s also about allowing people space to collaborate: write sticky notes, put things up on walls (or on boards which the teams can take away with them) and gather around things. The more space you have to play with the better.
When bigger is not always better
When we first set up the lab, we chose to store our videos as ProRes files, which are monstrously big, even if very good quality.
It was inevitable that storage of videos would soon become a problem, and it didn’t take long before it did. We now transcode our files to MP4 format using Handbrake, an open-source video transcoder. We’ve managed to maintain excellent quality, and cut our storage requirements from 7 TB per month to just 140 GB. That’s a huge difference.
In short, when setting up a lab space, it’s useful to think about how, when and where your videos will be used in the long term.
Don’t underestimate the need for good lighting
We’ve got bog-standard office lighting in our lab. It’s absolutely fine from a participant and moderator point of view but, as any camera buff will tell you, lighting is everything if you want to get the most out of your video.
We’ve got Lumens 1080p HD cameras in our lab, which record a very good picture. We’ve however noticed problems when the sun shines through the window - because the light is bright and uneven, the camera can’t hold focus.
If you’re building a lab, try and make your lighting as even as possible. We’re told that LEDs are a good bet. Also, make sure to have blinds or curtains so that you can block out glare when necessary.
On this point, when you furnish your lab, avoid white surfaces and glossy surfaces. You want to cut down visual glare as much as possible. We went for light grey and matt surfaces, which works well.
The equipment is hot
Our lab setup includes a lot of equipment, and it's hot. The bulk of the equipment is housed in a cooling rack, which might keep the equipment cool but gives off its own heat. The large monitors contribute their share. Add 12 tightly packed people to the mix and you’re bordering on the tropical.
We need to upgrade our standard air conditioning to something special. If I’d known this would be a problem, I would have planned an A/C upgrade into the original build more carefully.
Think carefully about your research participants door-to-door experience
There definitely is a transition when you start running your own lab - no more receptionist and no more drinks and sandwiches.
We quickly realised we’d have to think about how to get our research participants to and from the lab in a way that felt welcoming and relaxing. We now routinely include a host as part of recruitment. The host ensures participants are well looked after before and after their research session.
Make sure to consider a waiting area with as much care and attention as you do the actual lab space.
We’ve had several nice articles about the lab published
Economist: Digitising public services: Laboratory conditions
Digital by Default news: GDS unveils user research lab
Telegraph: Government bureaucracy 'insanely arrogant' says digital boss (actually a positive story)
Other fun stuff
See a spontaneous log of our lab’s story on this Twitter search. Read more blog posts about our GDS user research lab.
Keep in touch. Sign up to email updates from this blog. Follow Kate on Twitter.
