All the direct hypotheses of my doctoral dissertation are significant at 0.001 level of significance, now the reviewer has asked me to justify the same. now how to explain that?
I absolutely don't understand the question. Justify what ? ... But also, is this a review by your dissertation committee ? Like, it is a question from our committee to be sure you understand the statistical analysis ?
Well, I agree that I don't understand how anyone could ask this question. I suppose you could just say that all your probability values were produced by whatever software program you used (e.g., "All reported probability values were generated by the SPSS software program.")
It's not clear what kind of reply the reviewer is requesting. If you haven't, you might include e.g. "Analysis was conducted by multiple regression analysis. Residuals were checked for normal distribution and homoscedasticity. p values were based on a t-test for each included independent term." Or something like that.
I think the key word is "all" (p-values). You did not mention your scientific domain, but if it is not an exact science (and not even there), it is highly unlikely that all your hypotheses are confirmed, and even more suspicious at such a small 0.001 significance level.
In the social sciences, hypotheses are very difficult to test and most are never confirmed, so I actually agree with the reviewer.
Many consider a paper "successful" if the p-value is lower than the significance level, but I believe this is not the right way of doing (social) science. Unconfirmed hypotheses are just as valuable.
A factor that no one has mentioned yet is your sample size. If you are working with a very large N, then it is much easier to obtain p values in the .001 range.
Justifying a significant result means providing evidence or reasoning for why the result is meaningful and supports your research hypotheses. There are a few things you could consider when justifying a significant result with a p-value less than 0.001:
Strength of effect: Discuss the magnitude of the effect size, for example, effect size such as Cohen's d or odds ratios. High effect sizes can provide strong evidence for the significance of your results.
Replicability: Highlight the robustness of your findings by discussing the results of any sensitivity analyses or additional tests you performed to ensure the results were not due to chance.
Previous research: Cite previous studies that have found similar results and support your findings. Show how your study builds upon or extends previous research in the field.
Theoretical implications: Discuss the implications of your results for the theory or hypotheses you were testing. Show how your findings contribute to the advancement of knowledge in the field.
Practical implications: Discuss the practical implications of your findings. For example, if your study has implications for real-world applications, highlight how your results could be used to inform policy or practice.
Limitations: Be honest about the limitations of your study and how they could have impacted your results. Acknowledge any potential sources of bias and how they could have influenced your findings.
Overall, the goal is to provide a thorough and convincing explanation of why your results are significant (in your case 0.001 level of significance) and worthy of further consideration and investigation. Sehrish Munaf hope this is helpful, if you have any further questions, feel free to contact.