I have conducted two identical surveys, 4 weeks apart, containing Likert scales. Some questions use the scale of "strongly agree" through to "strongly disagree" whilst the others use "very useful" to "not at all useful". The responses have been assigned numerical values in Excel and SPSS (e.g. strongly agree = 5). Both surveys were given to the same cohort of participants (as mentioned, 4 weeks apart).
The results were anonymous and no ID number was assigned to respondents. Hence, when I come to compare the results from the surveys (i.e. how the responses to the questions changed), I will be unable to match each individual's results. I was originally going to do the Wilcoxen signed rank test for my statistical analysis. However, I understand that this requires matched results (i.e. the responses from each individual directly aligned/in the same row as one another) which is not possible in this situation.
Has anyone got any advice? The only option at the moment appears to be the Mann whitney u test.
Thank you in advance
Dear Rebecca,
As you say, it's a shame that your surveys were not matched. In view of this the Mann Whitney U test is an option, but it does not work very well when it only has a small number of ordinal values.
Have you considered an exploratory factor analysis or a scale reliability analysis to combine groups of your questions together into scales? If you could do this then the derived scales would have more values and could be treated as scale data (rather than ordinal), meaning that a Mann-Whitney U test would work better. It would also give you the option of an independent samples t-test if the assumptions were satisfied.
If all else fails you can still fall back on a chi squared test.
I agree with what Peter has suggested above.
If you can repeat the experiment, I would make sure to then keep track of each individual result. With what you have now, you can only test for changes in the overall responses for each question. There are many advantages to having matched the individual responses.
If repeating the experiment is not possible, I would form indices from several questions, if this is possible and meaningful. If you decide to use a series of Chi square tests, make sure to account for the multiple testing by using a Bonferroni adjustment or similar to obtain a reasonable Type I Error rate.
It is not possible for me to repeat the surveys. So yes, I am going to test for the changes in the overall response to each question.
Don't think chi squared test will be a possibility - as my study does not have a hypothesis. Instead, I have research questions that I am exploring.
I have changed the text responses into numbers e.g. "strongly agree" equates to 5 whilst "strongly disagree" equates to 1.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS