If a trial was a single centre trial with a small number of people, and they found that a hypothetical intervention had a significant effect compared to the control group, but in-fact with a larger powered trial they would have found no significant difference. Is this described as inaccurate results or invalid results?
If the criterion for statistical significance was set at (say) 0.05, this means that there is a 5% chance of falsely rejecting the null hypothesis. If in a larger study, the difference is not statistically significant, then the most likely scenario is that in the first study, you did in fact make a Type 1 error. Type 1 errors are most likely caused by some type of bias, such as selection bias.
Small sample size can increase both alpha and beta errors. However, it usually tends to suppress the association. In your small sample size study, u found significant association (type 1error), which may be due to selection or info bias (as commneted by Adrian). However, your question was if that was ''invalid'' or inaccurate results? invalid is a synonym for inaccurate (but not a synonym for unreliable), and the common term used is ''invalid results''.
I think it is neither inaccurate nor invalid; I think that "inadequate" may be more appropriate.
A small sample size reduces the statistical power to detect a difference between intervention and control groups. Type II errors are very common with small sample sizes, although your study reports a Type I error. However increasing the sample size reduces the likelihood of both Type I and Type II errors.
Given the small sample size and that the results do not reflect the reality in the population, the results would be said to be 'inconclusive'.
Dear Peter,
Finding a significant result with a small sample size means that there was a large effect. Such a result is no less important than a similar significant result with a larger sample. The result with a small sample size should only be overlooked if other results have a much lower significance value. Alternatively, you could use a Bayesian technique to combine all the evidence together.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size