I have the revision of a paper, and a reviewer ask the next question regarding a correlation analysis:
Did you apply any correction (e.g., Bonferroni) for multiple comparisons?
I think that Bonferroni adjustment use for multiple comparisions, for example if I run several Anova. But I don´t know is there any adjustment o correction to made in a single correlation analysis.
These table present a correlation of 10 variables from 2 different questionnarie. So, I don´t understand the reviewer question. How can I answered it?
The adjustments are when you have multiple p values within the same family of tests, so in a correlation matrix it is likely most of these are in the same family. Some packages, like Systat, automatically adjust.
Easy to apply different adjustments. I'll use R since it is free. Suppose you have six p values: .02, .04, .06, .08, .10, .12. In R you would do
p.adjust(c(.02, .04, .06, .08, .10, .12))
and get
[1] 0.12 0.20 0.24 0.24 0.24 0.24
Daniel Wright would you say that one has also to adjust the p's in a multivariate analysis, such as structural equation modeling? I actually never thought about that but in a SEM there is also the problem that by chance you will get a certain number of false positives (when there are several endogeneous variables)....
Best,
Holger
@Holger. YEs SEM is like any other least squares regression. Best, D. Booth
Dear holger
whenever you Apply multiple test the same dataset whether univariate
Or multivariate analysis reserchers need to adjust p values order to capitalize on chance (prevent type 1 error) however this often not applied in psycology research or sometimes applied for mean comparasions such as t test or one way ANOVA some procedures developed to adjust p values a well known example is bonferonni but none completely effective http://www.biostathandbook.com/multiplecomparisons.html
Holger Steinmetz , whether you use adjustments depends on the costs of Type 1 and Type 2 errors, and how you are using the results. So the answer is, "it depends". The decision should not really depend on whether your multiple p values results from a correlation, Anova, SEM, or something else. All this said, you should question why you have this many p values and the limited value of these.
It is hard to answer this question without seeing the data. Technically you have to adjust for multiple tests. (Each correlation is a separate test. ) But most of the time, it is not done. One problem is that the correction methods over correct.You can try a different journal.
Was this a fishing expedition or were you hypothesis testing? You can try a sensitivity analysis. For example, randomly divide you data into 2 halves and look at the resulting correlations compared to the original correlations.
Best wishes.
Michael
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis