User testing is essential, but it shouldn’t have to be complicated. Sometimes keeping things simple can give the best results.
I’ll always remember the first time I tested in a ‘real’ lab.
It was for one of my first UX gigs, so I wasn’t long out of university. Despite my inexperience, I’d somehow convinced the client to rent a lab. An exciting prospect!
Lab-based testing was something I’d studied, but hadn’t been given the opportunity to do.
This lab had everything I could have wanted. Eye-tracking, recording equipment, and even an interrogation room style two-way mirror. I was like a kid in a candy store.
The lab was set up so for the moderator to sit in a hidden room, observing the session over cameras that were craftily hidden in fake office plants. A loudspeaker was also fitted. This was so the moderator’s disembodied voice could boom out, all ‘Wizard of Oz’ style.
It was awesome, and I couldn’t wait to put all these goodies to use.
The first couple of sessions went fine… I was still learning the ropes, but I worked through it.
And then the drama started.
The website I was testing was pretty old-school. It had some fairly obvious design problems, so it was inevitable that some big usability issues would crop up. One participant came across a real doozie, and simply couldn’t complete the task.
They started getting stressed. The borderline-Owellian setup of the lab had created a high-pressure atmosphere. This, combined with the stress of not being able to complete the task made the participant feel pretty rattled.
I did my best to reassure them, to explain that we were testing the site and not them. But it was all too late. They totally broke down into tears under the stress of it all.
I came out of my secret room and attempted to console the (now bawling) participant. You’ve got to remember that I was still pretty young, basically a student. This was a really awkward situation that I just didn’t know how to deal with.
My exciting usability test had turned into a nightmare.
Perhaps the participant was already having a bad day. Maybe they were just of a slightly nervous disposition. In my inexperience I’d also probably briefed them way too formally, acting as a ‘moderator’ rather than a friendly human being.
Whatever the case, the intimidating and impersonal setup had really distressed them. It spoiled the session, and worse still it had spoiled that person’s day.
Not exactly the high point of my career!
Chill out, keep it casual
Mistakes are made so that we can learn from them. After that embarrassing incident, I moved towards a more casual approach to user testing.
From that point forward, I conducted all of my sessions sitting next to the participant.
I couldn’t help them complete the tasks, or do anything to affect the study. But I’d be present. Smiling, friendly and supportive.
I ditched the microphone and spooky hidden cameras equipment in favour of just laptop running a screen recorder. My ‘lab’ became the cafe or a pub. When running testing in office environments, the staff canteen or kitchen were my go-tos.
I made sure to test in fairly busy public spaces, where people felt more at ease.
Testing in these kinds of environments had a much more positive impact on results. The whole affair was more informal and relaxed. That meant there was less pressure on the participant. When they encountered errors they didn’t feel like they had failed some kind of experiment.
What’s more, they gave more honest feedback. They were more chatty, too.
Work on your bedside manner
Getting good results in user testing depends so much on being friendly and approachable with your participants.
The first few times I tested, I always rushed through the initial briefing. I was eager to maximise on the time we had that participant for. I wanted to get straight into testing the product, covering as much ground as possible.
I didn’t realise how much the results could suffer if the participant didn’t feel fully comfortable.
I’ve since learned the importance of doing a really thorough briefing to the participant. Steve Krug provides a fanatic script in his awesome book, ‘rocket surgery made easy’ (the script is available for free here).
The part that I generally copy word-for-word is this :
“The first thing I want to make clear right away is that we’re testing the site, not you. You can’t do anything wrong here. In fact, this is probably the one place today where you don’t have to worry about making mistakes.”
- Steve Krug, Rocket Surgery made easy
As well as briefing the participant properly, make sure you’re generally being a super polite host. This might sound a bit condescending, but I know first-hand how easily it can be forgotten. Especially if you’re running several sessions in a single day. It’s bloody exuasting, and even those with the most patience can find it wearing thin.
Rather than getting straight into the purpose session, first take some time to talk about their day. Ask them how they got here, and how the journey was. Offer to make them a cup of tea or coffee.
By making yourself friendly and approachable, participants will feel much more comfortable opening up to you during the actual testing.
It doesn’t need to be expensive
I’ve already discussed some of the outcome-related benefits to testing in public spaces. It’s probably not suprising that it can also save a bunch of money. Instead of spending hundreds of pounds renting fancy equipment, all you need is the design running on a phone or laptop.
Sure an eye-tracker is cool, but is it really necessary? If your product has major usability issues, they’ll come out of the testing however you do it.
Let’s be clear. I’m not saying that usability labs are pointless.
There are plenty of scenarios where testing in a more formal, controlled environment is essential. For example, the product might involve us using very specific workspace setups that we simply can’t recreate in a public space.
For most of us making everyday digital products though, labs are just overkill.
Guerrilla user testing techniques are often put down as being a ‘quick and dirty’ alternatives to the real deal. I think that these methods can produce just as good (If not better) results. Better still, you can implement them for a fraction of the cost and effort.
As always, it depends
In our industry, there’s no right or wrong answer. The setup and equipment you’ll need are dependant on your project, and the outcomes you need.
Whatever you decide to do just remember : keep it casual!