We don't determine sample size to see the differences between the two groups after random allocation in RCTs. So is it appropriate to apply statistical tests to see the difference between groups at baseline ?
Hmm, what would be a sufficient sample size to detect differences in baseline after randomisation? And what would you do, if you detect some?
When it comes to testing, Cochranes aphorism applies here
"Before ordering a test, decide what you
will do if it is (1) positive or (2) negative. If both answers
are the same, don't take the test."
Please check out
Senn, S. (1994). Testing for baseline balance in clinical trials. Statistics in medicine, 13(17), 1715–26. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7997705
Altman, D., & Dore, C. (1990). Randomisation and baseline comparisons in clinical trials. The Lancet, 335, 149–153. Retrieved from http://www.sciencedirect.com/science/article/pii/014067369090014V
Most trialsarchive an analysis plan that outlines what will be done if there are baseline differences, or sometimes state a priori plans to include certain adjustments in the primary analysis. This gets around the problem that emerges when people plan an analysis that does not adjust for baseline differences, then don't get the results they want and then start making post-hoc adjustments. This dilutes the integrity of the whole analysis. Personally, I don't think that statistical significance should be the key determinant for deciding whether adjustment for baseline differences should be made. A statistical test is based on identifying a difference that has a low probability of having occurred by chance but by definition baseline differences in an RCT all occur by chance. In my view, this is an issue of confounding, so the value of adjusting for differences is determined by the effect of those adjustments on the effect estimate.
Usually test of significance was applied to check for baseline comparability in RCT, but
Intention behind test of significance...
To see whether an observed difference is a real or important one.
The test actually assesses the probability (the 'p value') that the observed difference might have occurred by chance when in reality there was no difference.
whereas idea behind Random allocation is...
A difference of any sort between the two groups at the time of entry to the trial will necessarily be due to chance, since randomization prevents any external influences (biases) on which subjects receive which treatment.
Putting these two ideas together, performing a significance test to compare baseline variables is to assess the probability of something having occurred by chance when we know that it did occur by chance. Such a procedure is clearly absurd.
Douglas G. Altman
When randomization has been properly conducted the null hypothesis that treatment groups come from the same population is true.
In the usual framework of statistical inference rejection of the null hypothesis should lead to the conclusion that the groups are not properly randomized.
test of significance therefore be used to detect possible subversion of the allocation procedure.
Kennedy A, Grant A. Subversion of allocation in a randomized controlled trial. Control Clin Trial1997; 18(suppl 3): 778S.
Statistical significance is immaterial when considering whether any imbalance between the groups might have affected the results.
It is wrong to infer from the lack of statistical significance that the variable in question did not affect the outcome of the trial. Such an interpretation is unwarranted and potentially misleading.
Altman
- Badges
- Science topic
- Similar topics
- Medicine
- Epidemiology