We took two surveys, pre and post-pandemic, on the same group of people (n = 428). The surveys have 300+ questions for each respondent to answer. The surveys consist of mainly 4 or 5-point ordinal Likert-style questions and dichotomous questions (e.g., Yes/No) and a few continuous variables (BMI, age, family income, etc). I need to measure the difference between the surveys (p-value significance) for each variable PLUS measure the strength of association between them. So, here is my take so far.
1. Paired continuous variables --> Paired sample T-test (or Wilcoxon signed-rank test for non-normal distribution) for significance. Pearson or Spearman correlation for strength of association?
2. Paired dichotomous variables -> McNemars test for significance and Phi coeff for association.
3. Paired Likerts -> ????? (no clue) for significance and Spearman Rank Correlation (rho) or Kendall's Tau b or c for strength of association?
Again, please note that these are paired and not independent samples, so please do not list answers meant for independent samples.
Google searches, especially for the paired Likerts, have been frustrating and are all over the place. Also, my brain is tired :( ...
Some journals say you can use parametric tests for Likerts, even though they violate normality, for example. My dissertation professor says no...they are ordinals that require non-parametric tests, which I also agree with.
I have attached a StatsTest chart for your reference if it'll prove helpful.
Thank you!
David
Set a variable to identify the date and time of the survey point (before vs. post) and merge the pre and post-pandemic survey data into one data set, making sure that each participant's information is accurately matched.
Provided the assumptions are true, you can use the Wilcoxon signed-rank test (non-parametric) or paired t-test (parametric) to evaluate the difference in replies between the before and post surveys for each ordinal variable. Calculate effect size metrics like Cohen's d or r to express the degree to which the surveys' ordinal variables are correlated.
One can run a McNemar's test on dichotomous variables (such as Yes/No questions) to determine how the proportions changed between the pre-and-two post-surveys. For dichotomous variables, determine the odds ratio (OR) or relative risk (RR) to assess the degree of relationship between the surveys.
If assumptions are met, you can use paired t-tests to analyse differences in means or distributions among the pre-and post-surveys for continuous variables, or you can use non-parametric options such as the Wilcoxon signed-rank test.
If your Likert-scored variables can be combined into scales then this would allow you to compare those score via t-Tests of correlations. Use coefficient alpha to determine if the scales are sufficiently reliable.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python