During my fifteen+ years of experience as a user researcher, I have trained and worked with many other researchers. I have seen some very good interview moderation and, unfortunately, some very bad. Moderating UX research and interviewing users aren't necessarily easy things to do, but fortunately there are some simple guidelines anyone can follow to be a lot better at it.
1. Shut up and listen
One of the biggest mistakes I see by moderators is simply talking too much and not listening enough. I think sometimes moderators feel uncomfortable and so they talk nervously, or they aren’t confident in their mod guide (see “It works if you work it” below), and some just like to talk, unfortunately about themselves!
The research participant should be the one talking at least 80% of the time. If this is not the case, something is wrong. One of the best tools a moderator can use is silence. If a moderator asks a question and doesn’t get as much data as she want from it, the best thing she can do is just sit in silence. Most of the time the participant will provide more detail without the moderator having to say anything.
2. Be as neutral as possible
Another big mistake I see in UX research is a moderator who unknowingly encourages or discourages certain responses. This is a form of bias and needs to be avoided. This happens in the moderator’s choice of words, tone of voice, and/or facial expressions. I am referring to those little confirming or non-confirming words used in response to the participant. For example, a moderator should almost never use words such as “great,” “awesome,” “excellent", or “wow.” These words tell the participant, whether it’s true or not, that the moderator likes their response. People naturally want to please, so this encourages participants to provide similar responses going forward. Moderators should give the impression that, while they are interested in and value what participants say, they don’t prefer any particular type of response Participants need to know there are no right or wrong answers.
Similarly, a moderator's face should never light up when a participant says something interesting. Moderators should act human and personable, but should also have poker faces. It’s a delicate balance. Words and phrases to use (and I use these over and over again, so don’t worry about seeming repetitive) are: “OK,” “makes sense,” “got it,” “I understand.” Even these phrases need to be said in a neutral and consistent tone throughout the session.
3. The devil’s in the details
Many times when asking a participant to answer a question or perform a task, the participant provides a very brief response and not as much information as a moderator would like. Many moderators either leave it at that or start stumbling around trying to figure out how to get more details and end up rephrasing questions in a biased way. Thankfully, in addition to the silence strategy, there are some simple fool-proof probing questions that can get to the participant’s motivations, thoughts, and other deeper details without bias. Here are some I use the most:
“Can you give me an example of that?”
“What are your thoughts on that?”
“Why is that?”
“Why do you say that?”
“Can you elaborate on that?”
“Tell me more about that.”
“Why did you do that?”
“Because..?”
“What makes you feel that way?”
“What are you thinking?”
4. It works if you work it
A great research session starts with a great moderator’s guide or script. If a moderator doesn't have that, he can’t hope to get what he needs out of the project. It should be carefully honed and crafted to flow like a conversation, reduce bias, and answer all of the research questions while keeping within the allotted time period. While there may be varying schools of thought on this, I try to stick to the same wording of questions with every participant. If a moderator asks each participant a question in a different way, for example “What do you like about this website?” vs. “Do you like anything about this website” those questions can actually be interpreted very differently and the answers can not be compared apples-to-apples. The data just won’t be good.
Once a moderator has a great mod guide, she should stick to it. I have seen moderators ask the same question five different ways to five different participants in the same study. I think this is for a number of reasons:
1. They are tired of hearing themselves say the same thing over and over (without remembering each participant has not heard it before.)
2. They stop paying attention to the mod guide and just “free-style” it thinking it doesn’t matter or it’s keeping things casual with the participant.
3. They were never really confident in the mod guide question to begin with.
4. They personally don’t like the answers they have been getting and are hoping for different ones.
While it is OK to adjust a question phrasing towards the beginning of the study or after pilot testing, it should be very deliberate, for a good reason, and should be used for all remaining participants. The best, and thankfully easiest, thing to do is stick with the guide. It was created for a reason. But you have to actually use it. It doesn't work if you don’t work it!
