Colleagues,
I am in the planning phase of a project that will take the form of an online survey. My project consultant and I are thinking of ways to (1) minimize the incidence of fake or inconsistent responding as well as a (2) quick and (preferably) subtle way to identify these cases... does anyone have any ideas or suggestions from own experiences or the experiences of someone else?
Also, I'd like to add that I know that there are differing views regarding value and pros/cons of online survey methods, and that I am aware of these issues. For this Q&A post I'm primarily interested in gaining insights and ideas for identifying and reducing fake/inconsistent responding.
Thanks in advance.
Are you familiar with the Maniaci paper? It might be useful as well.
Best,
Ramzi
http://www.sciencedirect.com/science/article/pii/S009265661300127X
Thank you Jan and Ramzi for your insights and recommended resources! Very useful information!
I agree that faking and inconsistency are two different things, since the first requires a deliberate effort to create a false pattern in the data, while the later replaces any pattern that might exist with random error. The former is the more serious problem because it will bias the estimates in your analysis, while the second will leave you with larger standard errors but unbiased estimates.
One way to deal with severe faking is to do intense outlier examinations on your results, which requires setting careful standards so that you aren't "cherry picking" your data.
I'm curious why you are interested in this problem in your particular case. Are you working on topics where these issues are a particular concern?
Both answers are fairly complete and helpul... so nothing to add.
May be if you may tell more on the subject of your questionnaire... I may help with rules of thumb... I previouly used to track fake answers... that is ask specific questions within your questionnaire ... and propose answers including one that does not esist/ or make sence, providing that you have objective data on the subject.
Best,
Another idea: If you are using a web survey system that will record respondent timings, you can identify those who are just clicking through quickly without reading by the very short time they take to click through the web pages. Some survey web programs are better for this than others, though.
Also, Qualtrics has a nice online article on the topic of response quality that you might take a look at: https://www.qualtrics.com/blog/online-survey-valid-responses/
I think it depends on your design how any one find out the survey link. if you arrange it very carefully it reduce maximum fake response.
beyond that you may keep few questions that only can be answered for particularly group of people (your targeted population).
- Badges
- Science topic
- Similar topics
- Statistics
- Survey