I conducted a grammar test consisting of 36 questions divided into 8 categories. The categories form a cline, with category 1 being the prototype and category 8 being the furthest away from the prototype. The participants were 44 learners, divided into three proficiency groups.
I want to test if the mean score of each category is significantly different or not (within each proficiency group) . The distribution is not normal. Is LSD an appropriate test to use?
8 categories x 3 groups x 2 comparisons (within group) = 48 comparisons. You have only 44 learners. If you really want to do more than coffee grounds reading and keep that design, consider a total sample size of at least 500 learners.
8 categories x 3 groups x 2 comparisons (within group) = 48 comparisons. You have only 44 learners. If you really want to do more than coffee grounds reading and keep that design, consider a total sample size of at least 500 learners.
Hello Anubhuti,
Sorry, but I don't fully understand your query. Do you mean that:
1. Each question falls into one of 8 categories;
2. Each response is scored as being close or different from some preferred model answer (and the scoring is from 1 to 8);
3. Respondents give 8 answers to each question; or
4. Something else?
If it's #1, then you'll have 8 subscores for each respondent (as a within-subjects or repeated measures factor). In this context, LSD doesn't seem like a good option, regardless of distribution shape, since you'd be evaluating up to 28 pairwise comparisons overall, or up to 3X that, if done separately by proficiency level.
If it's #2, and if you also rate responses for correctness (as well as conformance to a model response), then you would also have up to 8 scores per case, but likely differing numbers across respondents. This is because not all participants may give answers that fall into all 8 categories of resemblance to model. So here you would have to deal with an incomplete repeated measures design.
If it's #3, then again you'll have 8 subscores for each person. As with #1, you'll wind up with a lot of comparisons. So, LSD isn't a great option.
If it's something else, then you'll have to help explain your data collection scheme in more detail.
At this point, I concur with Jochen that you're apparently trying to estimate a lot of parameters relative to the sample size. So, both statistical precision and statistical power will be concerns, regardless of what analytic method you apply.
Good luck with your work.
Thank you Jochen and David,
To answer David's question, it's #1.
The participants were asked to select the correct grammar item from 6 options.
If I may ask a follow up question - Would it be possible to conduct LSD if there were 3 categories (prototypical, intermediate and lower)?
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis