How to Handle Difficult User Research Participants

User research and usability testing are familiar terms for many people but the way they work isn’t well understood. Although creatives involved in research hope for expressive, opinionated, and cooperative participants, that’s not always the case.

The goal of these usability tests is to see how real users interact with a design product. How the moderator reacts to participants who are hostile, combative, or non-participatory can greatly impact the testing environment and corrupt the attitudes of other participants. 

Keep reading to find out what UX researchers can do with difficult user research participants so the usability test can continue and realistic information can be gathered. 

Who’s Afraid of User Research?

When it comes to a target market, the possibilities seem endless. Particularly when the design product is an app or website with broad appeal, finding participants who mirror the demographics of the eventual users can be a challenge.

If you’re taking a risk or introducing new concepts, finding participants can be even more challenging. To make everything more anxiety-inducing, the reactions of the people chosen for the test are by nature a known unknown. 

Stakeholders and decision-makers at the company sponsoring the research may not fully understand the wide variety of potential results. And if they don’t understand the art of choosing research participants, they could insist on repeating the research or insist that it’s a waste of time altogether. 

Sometimes the wrong participants are chosen. Other times, a normally communicative and helpful person could have a stressful week or clash with fellow participants. 

But in the end, most UX researchers aren’t petrified of these possibilities because they can be anticipated. As long as you prepare for them, you can work around them if they arise. 

Defining a Difficult Participant

One important thing to note about our use of the word ‘difficult’ to describe user research participants is that we’re talking about people who don’t want to participate, are too shy or reserved, or exist on the opposite end of that spectrum and talk too much or talk over others. 

Any of these personality types can lead to less information gathering, corrupt the data you do get, or prevent other participants from making their views known. 

Many UX researchers might have an ideal research participant in mind whether they realize it or not. Anyone who doesn’t match the candidate profile they’ve imagined might be considered difficult. But like so many other steps in the design thinking framework, you can’t let your own expectations interfere with reality in user research. 

That doesn’t mean the selection process shouldn’t be selective so that your research can include people who match the target market of the project. But it does mean the criteria for selecting those participants should be broad enough to include people who don’t match the personas that were brainstormed before the research was conducted.

The point of this user research is to get away from that kind of conjecture and closer to the people who will use the final product. Here are a few criteria that help define what is and is not a difficult user research participant:

• Past Experience

Let’s face it: every idea comes from somewhere. Whatever product you’re developing, others preceded it in spirit or more literally as predecessors. 

You want to know if people have any experience with such things. Dividing user research participants and your end-users into different levels of exposure will help you see which features make sense to a layperson and which might need a bit more explanation.

UX and product designers who are working on projects with a particular goal in mind – say, editing photos and video, investment activity, car maintenance, etc. – need people who have experience in that field at a minimum. On the other hand, you might be more interested in finding out what people with no exposure to similar products think when they first interact with one.

• Communicative – Not Talkative

Moderators and facilitators frequently ask user research participants to think out loud and explain why they’re doing things a certain way. They could also be asked to state and explain their feelings on a given subject or idea. 

You want people who can communicate such ideas even if they’ve never put much thought into them before. Participants shouldn’t be talking your ear off or veering off the subject unpredictably. It’s frequently a balancing act to get people to talk without making them talk too much on irrelevant topics. 

User research being performed on groups of people can compound this problem. Some people who are timid by nature default to more boisterous or ‘type A’ personalities, resulting in results from only one type of person. 

Some louder types might talk over these people when allowed, so the moderator has to be able to step in and keep things on track. Pointing out the difference between the kind of discussion that is valuable to the research and chatter that isn’t can help the participants understand what the moderator is looking for.

• Familiarity With the Brand

Brand loyalty is always a great thing for the company, but too much of it can corrupt your user research. Typically, you won’t have anything to worry about if someone reliably chooses the brand sponsoring the research even if it’s for the same or a similar product. The danger comes when they feel some kind of pressure to say something positive about the design because of their brand loyalty. 

This is more common among people who actually work for the company that’s sponsoring the research, which is why UX researchers are always cautioning against using employees as participants even though it’s frequently much faster and cheaper.

In any case, the research could be and often is built to obscure the identity of the company (or companies) that are sponsoring the project and getting results from the user testing, particularly when the research is outsourced to a specialized company.

Your individual needs may necessitate more specific qualifications than these, but for general user research, these are a good set of criteria to start with. 

How to Handle Difficult User Research Participants

Participants don’t mesh with user research projects for a variety of reasons. The screener may have missed some crucial information and accepted someone who doesn’t fit the desired user profile. A misunderstanding could have caused this or it could have been a flawed screening process. In rare cases, people will purposefully leave out information to get the daily rate.

Defining criteria and expanding on certain ideas can help prevent this problem. For example, if the screener wants to know whether prospective participants have ever played the lottery, you’ll get more relevant participants by explaining what lottery you mean, what constitutes playing it, and how often they have done and continue to practice that habit. 

If a participant fits the desired profile but doesn’t contribute much to the conversation, the moderator will have to make a few decisions on the spot and use some strategies to try and get participants talking. 

Your quiet participant might just be soft-spoken, shy, or self-conscious. In that case, open-ended questions might be enough to elicit some kind of response. Even better, the moderator can acknowledge these types of feelings – chances are, the seemingly confident participants also have some feelings of self-consciousness. 

Relaxing the room with prompts is another winning tactic. Remember, not everything that happens during the interview has to contribute directly to the research. It’s better to waste a little time letting people get familiar with each other and the structure of user research than to spend an entire session hearing from the same couple of people while other participants sit quietly.

Perhaps more problematic for the moderator are the people who speak too much and far too often off-topic. Dealing with these sorts of participants is difficult because it can be awkward in a group setting. 

But the price you pay for being direct with overly talkative participants is worth giving the others space to express themselves. It can also prevent arguments from arising among the participants. 

One frequent issue with talkative or boisterous participants is that they contribute a lot at the beginning and then – despite continuing to talk – don’t add anything else significant for the rest of the session. For instance, they might have a strong opinion at the beginning and not budge from it no matter what kind of new information comes to light. Or, if it’s not about their opinion so much, they might have an idea for the product and not allow space for other ones. 

The worst thing you can do is alienate these people and transform them into silent participants for the duration of the session. Be direct and transparent about what kind of information you need and call attention to repeat information without being too aggressive about it. 

With both overly talkative and habitually quiet participants, focusing on some kind of task can be helpful because it allows quiet types to contribute to the research findings and it gives talkative participants new data that will hopefully lead to new ideas in the follow-up discussion. 

Asking direct questions to participants by name will get them talking. If you make this a recognizable pattern in the structure of the research, then talkative participants will accept that you want to move on to another person once they’ve said enough. 

Getting Participants to Speak Clearly

When participants are talking and sharing time fairly, the information they contribute can still be less than useful because they aren’t articulating well or have trouble explaining their ideas. 

Encouraging more detailed responses is sometimes as easy as pressing for more information. But if you’re tasking participants with explaining things they haven’t put much thought into before, they could understandably have trouble making themselves understood. 

Try some of these ideas to prompt clearer and more succinct explanations from your user research participants:

1. Do Multi-Day Studies

While stakeholders might prefer shorter research trials to save money, it might be worth it to give participants a break so they can process new information. 

If you’re presenting them with something like a TV commercial or an update to an app they’ve already seen, giving them the evening to think about their responses might not be that useful. 

But when the subject matter is more challenging, some rest time can help them think of better ways to explain their thinking on the design.

2. Ask About Priorities

Sometimes the opacity or vagueness of respondents’ answers is because they feel overwhelmed with thoughts on a topic. In that case, asking them to tell you only their most important feelings might help. 

It’s not that they need to prioritize their own feelings a certain way. Be transparent about the kind of information that could be useful to progress the design project and what cannot. You want the participants to think out loud, so think out loud with them as you try to reach conclusions that they feel express their thoughts while also supporting the goals of the research. 

3. Highlight Actions

Again, a profluence of thoughts could deter clear thinking. Sometimes, participants will be conflicted about what they really think. 

In either case, the moderator can focus on the actions participants took. It shouldn’t be a gotcha moment, but it will help clarify what the participants envision in their minds versus the way they demonstrably behave when they interact with a design product. 

An effective humanizing tactic to keep this action highlighting polite is for the moderator to contribute with their own behavior. Be careful, though, that you aren’t leading them anywhere with such information. You can also simply affirm participant thoughts and contributions with affirmations like “That makes sense,” or “I understand.”

4. Eliciting Opinions

Whether it’s due to personal taste or feelings of sheepishness, some participants don’t voice strong opinions one way or another. With design elements like buttons, many people don’t actively think about why they click on things or prefer designs that look a certain way. 

Ideally, the moderator will be able to get them to elucidate further and come to a realization that they can adequately express for the purposes of the research. However, that’s not always the case. 

Digging for more detail is a natural instinct for most UX researchers. One thing to consider is that you might have located a design element that people generally just don’t feel strongly about. If multiple participants feel that something is “just ok” or “fine” then the moderator should try to find out if anything could make them feel more strongly about it, or, if not, then the moderator should ask what these participants do think is important. 

5. Don’t Let Them Off Easily

Participants who try to avoid criticism or engagement with a design product could be doing so because they are conflict-avoidant or because they just don’t want to be there. While it is rare for people to go through the entire screening process when they don’t want to participate, it does happen. 

When either of these situations occurs, it’s important to keep at them. Without being negative or overly forceful, moderators can direct participant attention to the design product with prompts or further questions. 

For example, research participants who give up trying to make a design product work ought to be asked why they give up. Sometimes, they’ll say they would ‘just’ find an alternative. If that’s so, ask them to continue trying with the design product for the sake of the research. 

Conflict-avoidant participants should be encouraged (within reason) to be critical. Moderators can explain that they won’t take it personally, that the entire point of design thinking and the user research at hand is to find out what people think so the company can build a better product.

Conclusion:

The users designers dreamed up in brainstorming sessions are often starkly different from the ones selected to participate in user research. But that’s a good thing – it creates better-informed designers. 

Sometimes the moderator’s job is more challenging because of people who talk too much, talk too little, or communicate poorly. UX researchers can use the information and recommendations in this guide to handle difficult user research participants and ensure they get the best possible data to develop a successful product.