My context is survey question development and usability testing. We'll have participants answers some questions or do a task and then debrief them to find out what issues they had. Some colleagues and I are debating the phrasing of the debriefing questions/probes.
In my mind, asking "What problems did you have?" is the best way to get people to open up because it presumes that there were problems, making it easier for a person to explain any problems they had, compared to asking "Did you have any problems?" I can't remember where I picked up this practice, but I've used it in other contexts, too, and it seems to work. Our natural tendency is to acquiesce, be polite, and not criticize, even in a debriefing setting, and I always worry that "Did you have..." promotes acquiescence.
Is there a social psych or communications phenomenon here? Some lit I can bring to the conversation?
Thanks!
Did you have any problems?" This is closed question requiring a Yes/No answer and then depending on response, it loops to another
"What problems did you have?" This is for open-ended question and best suited for in-depth interviews or a few qualitative answers from respondents
Yes, I agree with Albert that "What problems did you have?" will yield better responses. A better or follow up question may be "Could you describe the the problems you encountered/ faced"? The interviewer needs to probe at a deeper level. You could also apply Kelly's Repertory grid to understand usability.
Best wishes!
- Badges
- Science method
- Similar topics
- Mathematical Sciences
- Statistics
- Applied Statistics
- Survey Design