If I specified a priori the model for a logistic regression (based on previous research), and run a 10000 Monte-Carlo perumtation do I have to report the adjusted value? Can I just report the unadjusted considering that the model was specified a priori?
If you determine one p-value by Monte Carlo, that's one p-value.
I'm not sure by your question what adjustment you mean. If you mean adjust for multiple tests, then no, one p-value is one test, and shouldn't be adjusted.
Thank you I apologize for not being clear. Im referring to an output in stata.
In STATA after running a monte-carlo bootstrap the output reports an unadjusted p-value and an adjusted p-value. The adjusted is for multiple tests, however if the model was specified a priori (based on previous research) then the question is if it is necessary to consider the adjusted p-value for multiple tests? I would assume not, otherwise it is pointless pre-specifying the anlysis in a protocol/registry.
Thank you for your help and time.
With the question of adjusting for multiple tests, the real question is, How many tests are there?. The a priori / post-hoc distinction matters less. But if there are multiple tests for the significance of multiple coefficients, usually no one adjust the p-value on these. (Though in graduate school I was taught to do so.)
Being more reflective: Does the Stata documentation tell you how it's adjusting the p-values or why?
Multiple tests have to be adjusted but give both values and you are on the safe side of the road.
These links may help you
https://www.graphpad.com/support/faqid/1518/
https://www.biostars.org/p/130173/
http://www-stat.wharton.upenn.edu/~steele/Courses/956/Resource/MultipleComparision/Writght92.pdf
https://biology.stackexchange.com/questions/3066/what-is-the-difference-between-naive-and-adjusted-p-values-in-a-gwas-study
https://communities.sas.com/t5/SAS-Statistical-Procedures/Multiple-comparison-adjusted-and-unadjusted-p-values-are-the/td-p/324737
https://stat.ethz.ch/R-manual/R-devel/library/stats/html/p.adjust.html
https://brainder.org/2011/09/05/fdr-corrected-fdr-adjusted-p-values/
Article Three common misuses of P values
- Badges
- Science topic
- Similar topics
- Engineering
- Modeling