Is it worth using a paired t-test on pre and post survey data with 12 or less responses? Or should one rely more on the mean and standard deviation of the two data sets to evaluate the survey responses?
Hello Jamel Alexander Do you only have one subject group?
If your answer is yes, I have to say that I have not seen anywhere that if the number of samples is small, then the Dependent T-test cannot be used. In fact, as far as I know, using or not using the Dependent T-test has nothing to do with the number of samples. However, when the Assumptions of the Dependent T-test are not met (again, the size of the studied sample is not one of these Assumptions), one can think of non-parametric tests such as Wilcoxon Signed-Rank.
Jamel Alexander
The issue is not so much the sample size per se, rather whether the difference scores are roughly normally distributed. Having a larger sample size makes it easier to assess this assumption, but a smaller sample can satisfy the assumption.
Plot the difference scores for the 12 subjects to determine whether they are roughly normally distributed. If there are no major skews or asymmetries, do the t-test!
Just to be clear, for a paired t-test, the test is done on the differences of the pairs. The distributions of the pre- group or post- group don't come into play at all in the analysis.
Please tell us more about the survey. E.g., is it brand new, or is it an instrument that has been used in the past by other researchers? What is the nature of the items, and how many items are there? How is the total score computed? Is if sensible and defensible to use means & SDs as descriptive statistics? Etc. Thank you for clarifying.
PS- The question about whether the survey is new or not is motivated by this BMJ Stats Note:
Article Practice Statistics Notes Analysis of continuous data from s...
I'd modify the above comment to larger samples are more reliable if sampled at random.
A large biased sample is worse than a smaller non-biased sample.
- Badges
- Science topic
- Similar topics
- Biological Science
- Bioecology
- Systems Ecology
- Landscape Ecology
- Connectivity