Tips for Conducting User Testing
If you’ve been following this series, you already know why user testing is important to any digital project. And you’re familiar with a range of testing methods that help you gain insights into user preferences and measure user behaviors. Now the final element of your testing plan is to decide how you’re going to conduct your user testing.
Depending on the type of testing you plan to do, it might be performed either locally or remotely. And you may benefit from engaging a usability lab or an online service.
Stay close to home or take it on the road?
The difference between local and remote testing isn’t really the distance from your work station. It’s about whether it’s better to have a facilitator who is physically present to observe or interact.
With local usability testing, the test facilitator sits beside the tester and observes first-hand. The facilitator or a colleague notes comments from the tester, any non-verbal cues about their level of engagement, and the timing of the interaction (possibly including the number of mouse clicks). Sometimes these sessions are recorded.
Focus groups, stakeholder workshops, and some types of prototype testing are best performed locally, since they allow the facilitator to seek additional feedback for clarity.
Remote usability testing uses specialized software that allows the test facilitator to see users’ screens and hear their oral feedback as they complete a task remotely. This can happen in real-time or be recorded for the facilitator to review later.
A/B testing, heat mapping, online surveys and some types of task analysis are well suited to remote testing.
What about usability labs and online services?
Another major question about conducting user testing is when to go with a usability lab or an online testing service. The answer depends on the type of testing being done, the number of participants, the number of observers, and the testing budget and timeline.
Usability labs are useful when:
- local/in-person moderated testing is required
- sophisticated AV equipment is required
- you’re testing a large number of participants at one time
- you’re recruiting or testing a very specific audience (for example, a specific culture or language)
- multiple stakeholders want to observe (requiring a 2-way mirror)
- testing goals require participants to have a specific level of brand familiarity
Online testing services are useful when:
- your timeline is short
- your budget is limited
- you need remote access to crowd-sourced or pre-identified participants
- you’re conducting A/B testing, heat map testing, click testing, or design/brand testing
- you need to quickly resolve differences of opinion between stakeholders
Getting the most out of online testing
In addition to affordability, one of the biggest benefits of remote testing via an online service is the quick turnover: you can have results in a matter of hours. It’s also easy to recruit participants, and test results are compiled in an easy-to-share fashion.
But since you’re not present to facilitate the testing, it’s important to set up these tests and review them carefully for best results.
A few tips:
- Establish context for the testers. Make sure they know what they’re looking at and have any necessary background information.
- Clearly communicate the goals of your testing (a numbered list of steps to execute is helpful), but avoid leading the user.
- Always review more than the written responses. Written answers to your questions tell you what the tester thinks about the test he executed, but the webcam footage or screen captures tell a much bigger story about how well the tester understood the test and followed the instructions, and what other factors may have influenced the outcome of the test. (For example, a tester might answer a written question about a design with a simple “it’s not attractive” answer, and unless you watch the video you won’t know that as he viewed it, he commented, “I don’t think this design is very attractive, but that could have something to do with my color blindness.”)
Finally, a real-life example
Before we send you on your way to user testing nirvana, we’ll leave you with an example from our own experience.
One of our clients in the home building products industry wanted to improve the conversion rate on two key performance indicators on their website: ordering a product sample, and contacting a professional installer. We suspected that the placement of these calls to action might have something to do with the performance.
Hanson recommended user testing to gather unbiased feedback. We selected online testing for this initiative because we needed a quick turn around, and participants didn’t need to be familiar with the brand, products or website.
What we found was that testers looked for those calls to action on product comparison pages and in the photo gallery, but not on the product detail pages where the buttons were located. It turned out that users simply weren’t interested in ordering samples or contacting a professional until they had done more product research. So we moved the buttons to a more intuitive location, improved the user experience, and helped our client maintain engagement even during the typically slower off-season.
In summary
That brings our series on user testing to a close. The most important thought we can leave you with is that end users and their interpretations of a digital experience drive its success (or failure). User testing offers tangible insights that can make your project genuinely useful, and help you maintain the most relevant experiences for your users.
