I have two data sets to compare which are responses to statements using a Likert scale response anchors (7 in total, always untrue to always true, coded as 1-7) (so non-normally distributed, ordinal data). In the 2 groups (n=24 and n=34, responses collected a year apart), 5 cases appear in both data sets (responses at the different time points from the same people) but all other cases are from people who have only given responses at one of the two time-points.
If I am looking to hypothesis test for group differences, what is the best way to work with these groups? Split into independent and not? Analyse together ignoring the 5 cases where responses are from the same people? Previously, I have analysed other data sets (two and three groups) where all cases have been independent of each other both between and within groups using Mann-Whitney and Kruskal-Wallis tests, but as I have a mix of independent and not cases here, I am unsure on how best to treat the data set/the best hypothesis testing to run?
I had a long response to this question. Now deleted. To make a very long story short find the best consultant you can. Hope that there is a solution to this mess. This is not, not your fault because you were not at a planning meeting which most likely did not exist. If you are a PhD dissertation student in final semester GOOD LUCK if this is your dissertation problem. In this case, immediately find a new but kinder advisor. THIS should never have happened. IN the future remember: PROPER PRIOR PLANNING PREVENTS POOR PERFORMANCE. BTW ORDINAL DATA IS ORDINAL NOT CONTINUOUS NORMAL.. Both Ordinal and Normal data does not exist in the known universe. If your DV or DVs is ORDINAL my best guess is some sort of ordinal logistic regression The tests you mentioned are not going to help. I would normally suggest references. In this case there appear to be none. Having some of the same respondents in both groups pretty well kills things. These are NOT probability samples nor independent or nonindependent either so Who knows?. Maybe somebody on RG knows what to do but it is not I Good luck and God bless, David Booth.
Are these 5 "common" between the 2 surveys? If so, they might skew the second dataset because of their exposure to the first set of questions.
I suggest you try Cronbach alpha statistic on the 2 scenarios of including them and excluding them. You might get similar result.
Hi Hayley, Maybe this is a half full, half empty situation? How about trying ... 24 participants completed the survey and a year later (time 2) 29 different participants drawn from the same population completed the survey. Run your analyses as between-subject. Then say something like, "Additionally, 5 participants from time 1 repeated the survey at time 2. Though too few for statistical comparison, I (you) explored for additional insights and hypothesis generation. You can then say things like if the highest and lowest scoring participants remained the same. You can say if their survey responses uniformly increased or decreased across time like, or unlike, those in your between-subject study. Good luck with your project! ~ Katie
- Badges
- Science topic