So I used a survey, where I had the participants answer 4 Likert-scale questions both before and after a presentation. Ended up with 7 participants total. Since I want to compare their responses pre and post, I had them enter in a 4-digit code before each survey.
I know my data is ordinal, so should I be using a paired Wilcoxon signed rank test?
Would I need to run a separate test for each of the questions, or is there a way to combine them with a test like this?
Thank you in advance.
You can use the Wilcoxon matched pairs test. You can run it separately. About combining them, it depends more on the interpretation and concept of the questions than statistics and mathematics. If all of them are of the same nature, you can use a binomial model with n=4 and p=1/2 for each of them as the null hypothesis.
The first step that the Wilcoxon signed rank test performs is to subtract the two paired observed values. So as far as I'm concerned, it treats the data as interval in nature, not ordinal. ... If this bothers you, you could use mixed effects ordinal regression. ... Whether you can combine the questions into a single index really just depends on whether you think it makes sense. If you average or sum responses from questions into an index, you are treating the data as interval in nature. ... With 7 respondents, you won't have much power to detect small effects.
I agree with Babak and Salvatore that the Wilcoxon signed rank is an acceptable analysis. I recommend that you analyze the questions separately unless they are measuring the same construct and you compute a reliability coefficient that is reasonable.
Since the sample size is only 7 and there are only 4 questions, you might consider qualitative interviews. You will probably gain much more insight in that way.
If you move forward with this analysis, in general, the Likert scale is used to understand an overall construct. It is important to identify this construct and understand how each of the items contribute to a better understanding of that construct. Then, in a LIkert-based analysis, the individual ordinal-level item scores are summed. Analysis should focus on this total score that is assumed to be interval (under proper conditions).
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size