We have conducted a study to compare the effects of 3 training methods on body composition and cardiovascular risk factors. A Log 10 transformation was applied for all non-normal distributions, however, the reviewer suggested considering a more conservative test (the Bonferroni correction factor). If we apply the Benforrini correction (p/6), all significant changes become non-significant because the p-values are all close to 0.05. What to do?
Dear Said,
The point made by David Morse makes sense. Indeed, the Benjamini-Hochberg procedure to control for false discovery rate at 5% (e.g. P < 0.05) is generally a helpful approach since it is not so conservative as the Bonferroni adjustment. I strongly suggest you go over the attached paper, that provides a beautiful explanation about multiple testing correction.
Hello Said,
The log(10) transform decision concerns a completely different issue than the decision to apply a Bonferroni adjustment (for multiple tests).
However, slightly more powerful alternatives to the Bonferroni do exist (especially as the number of comparisons increases). Have a look at the Benjamini-Hochberg (https://www.statisticshowto.com/benjamini-hochberg-procedure/) instead.
Good luck with your work.
Dear Said,
The point made by David Morse makes sense. Indeed, the Benjamini-Hochberg procedure to control for false discovery rate at 5% (e.g. P < 0.05) is generally a helpful approach since it is not so conservative as the Bonferroni adjustment. I strongly suggest you go over the attached paper, that provides a beautiful explanation about multiple testing correction.
I would like to thank David Morse and Giovane R Sousa. The solution remains unclear to me. A two−way (group × time) repeated-measures analysis of variance was used to identify significant within-group differences in all variables. A general linear model for repeated measures was used to identify between-group differences in changes. Knowing that some distributions are normal while others are non-normal, how to proceed? who is better, transforming non-normal distributions using Log 10 (what I have done) or acting on p (Benforrini or Benjamini-Hochberg)?
The transformation and the correction for multiple testing are entirely different things.
Using log-transformed variables is usually more reasonable than using the untransformed values for almost all biological variables (that are bounded on the positive real line). When the variance is small relative to the mean, the log transformation will simply not have any impact; it does not harm to transform such variables anyway, because then the statistical model asks for relative changes rather then for absolute changes - and this usually is the biologically more reasonable question. Exceptions are left-skewed variables (variables with an upper bound) or variables that are bounded at both sides (like age or proportions), that should better be analyzed using a beta-model.
Corrections for multiple testing are important when the experiment is some kind of screening, that is done to find or identify at least on factor out of a couple of several possible factors. If you screen 10 or so different cardiovascular risk factors, chances are that some of the data you collect will just accidentally scatter in a way that looks like an effect. If you let enough people ply lottery, someone will win - not much of an evidence that there is at least one person with psychic powers... so you should correct for that.
David proposed to control the FDR, what may not be what you want. Bonferroni controls the FWER, and yes, it's a very simple but also extremely conservative method. Holm (or aka. Bonferroni-Holm) is a more liberal modification that still is simple to perform.
Article A general introduction to adjustment for multiple comparisons
https://towardsdatascience.com/an-overview-of-methods-to-address-the-multiple-comparison-problem-310427b3ba92
- Badges
- Science topic
- Similar topics
- Correction