Hello,
I'm new in this kind of analysis (applying surveys). So, I have some questions that I didn't find a proper answer yet.
First of all, here is my scenario. I'm going to apply an online survey for Uni students, which contains ten questions, some of them are just categorical (yes or no), and others are ordinal (negative, neutral, positive, NA). Considering that I'm going to broadcast this survey, I'm afraid that some students can answer it randomly.
So, my first question is: Do exist a strategy to avoid random answers in an online survey?
If there's no strategy, is it possible to detect it through the response analysis (like a statistical test)?
Thank you in advance for your attention.
Best regards,
Fábio
The people who answer randomly may show as outliers. However, it will be difficult to distinguish between a real outlier versus someone who randomly filled in the survey. You might be able to ask questions in such a way to catch these people.
You should use Cronbachs alpha reliability test on group of questions expressed in the same scale e.g. 5 point Likert
Have you considered keeping track of the time spent, by each participant, between their answer submissions?
@Antonios, I am not sure how this would help address the question. Some people read quick, others more slowly. Some will think about the answer, while others will mark the first thought. So there will be considerable variability. Of course if someone manages to fill in the survey in under five seconds then they are either exceptional or filled it in randomly. However, this information might be an interesting sub-project. Are people who answered in "this way" quicker or slower? It is additional data that might provide some unusual insights and raise some questions for further research. It should cost almost nothing to gather, and if it doesn't yield anything you have lost very little in the trying. I like the idea.
Dear Lobato,
You can design a questionnaire where the random answers can be detected. For example :
Repeat some questions in a way where the student should give the same answer for them.
(1) What is your favorite color?.....(7) What color you like ......?
If the answer of (1) is not matching the answer of (7), then you can ignore this one.
For a strong test, you can repeat some questions many times in different forms.
Best wishes
Hello all,
thank you everyone for your valuable contribution! I'm going to have in mind all strategies presented.
Please, if you have more ideas, post here!
Regards,
Fábio
Issam's answer brings up an interesting point. At least in English, there are often several ways to say the same thing. So it is possible to get slightly different answers for no other reason than the question or responses were phrased differently.
This could be as simple as leading questions that bias the results:
Considering now idiot president Trump's first year in office, would you view his miserable performance in office as ( Outstanding, fair, neutral, horrid, or abysmal ).
This could be in the categories: How would you rate the honesty of the police force in your neighborhood?
(Excellent, fair, neutral, poor, very bad)
(superb, good, average, bad, substandard)
(Outstanding, acceptible, average, deficient, they are all criminals, no opinion)
It might be that your results will change if you reverse the order of the responses.
Depending on what sort of questions are being asked, you might get different responses based on the order of the questions.
One solution is to have different versions of the survey. However, it may be difficult to keep focus on the survey as opposed to focusing on the role of survey methods in survey outcomes.
Certainly, many questionnaires do ask the same question more than once, but in a different way. While this can help examine bias/accuracy, too much of it might cause people (e.g., myself) to stop taking the survey. So it's a delicate balance.
-Bob Vadas, Jr.
An additional question, please. After detecting those random answers, how should they be treated?
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA