In my hypothesis driven SEM, I have in total 28 variables and 58 edges. Considering such amount, do you suggest me to report adjusted p-values with FDR for example or raw p-values?
Thank you in advance.
When dealing with a large number of statistical tests, as in your hypothesis-driven structural equation model (SEM) with 28 variables and 58 edges, it's important to consider the issue of multiple testing and how to control for the increased risk of false positives. Reporting adjusted p-values, such as those corrected using the False Discovery Rate (FDR), is generally a recommended approach in such situations.
Here are some reasons why reporting adjusted p-values, specifically using FDR correction, is beneficial:
Here's how you could proceed:
By reporting both raw and adjusted p-values, you provide readers with a complete picture of your findings while acknowledging the potential influence of multiple testing.
Remember that while FDR is a widely used correction method, there are other methods as well, such as the Bonferroni correction. The choice of correction method might depend on the specific context of your research and the level of stringency you require.
Ultimately, the goal is to strike a balance between controlling for false positives and not missing potentially important findings. Consulting with a statistician or a domain expert can help ensure that you choose an appropriate approach for adjusting p-values in your SEM analysis
Thank you very much for your reply Dr Abu Hammad. It was really helpful.
When conducting hypothesis-driven structural equation modeling (SEM) with a substantial number of variables and edges, it's important to address the issue of multiple comparisons to maintain appropriate control over the familywise error rate. Reporting adjusted p-values, such as those adjusted using the False Discovery Rate (FDR) correction, can be a prudent approach in this situation.
Here's why adjusted p-values are recommended:
By using FDR-adjusted p-values, you are mitigating the risk of making incorrect conclusions due to the increased chance of observing significant results purely by chance when conducting numerous tests.
Here's how you might approach this:
Remember that while adjusted p-values are important for controlling the false discovery rate, they are not a definitive solution to the multiple comparisons problem. Careful consideration of the theoretical framework, prior knowledge, and replication in independent samples is also crucial to drawing meaningful conclusions from your SEM analysis.
Lastly, if you're unsure about the specific procedures to follow or the choice of adjustments, consulting with a statistician or an expert in your field can provide valuable guidance tailored to your research context.
It depends on what error rate you want to use for inference. There is nothing wrong with performing inference based on per-comparison error rates using raw p-values. There is also nothing wrong with performing inference based on a family-wise error rate. Controlling a family-wise error rate is certainly seen as stronger evidence against the null when it is rejected, but this approach to testing can also lead to lower power.
- Badges
- Science topic
- Similar topics
- Philosophy
- Philosophy Of Science
- Reasoning