IMO, no one but you can answer these questions. But first, you must think carefully about the nature of your study, and what questions you are trying to answer. With that in mind, I recommend that you read these two 2005 Lancet articles by Schulz & Grimes. Despite being published nearly 20 years ago, they remain among my favourite articles on multiplicity. HTH.

https://pubmed.ncbi.nlm.nih.gov/15866314/

Article Multiplicity in randomised trials II: Subgroup and interim analyses

Thank you Dr for your answer. I will go through the papers.

Apologies for not being able to specific to your question! I had similar questions when I encountered multivariate situations, with multiple tests to be performed at the risk of increasing type 1 error. As a rule of thumb it is said if the number of tests is less than 6, we can use simple correction (alpha/ no of tests). If the number is large (I have not used the corrected Bonferroni test), I could look for, say, an overall F - test, i.e. ANOVA, MANOVA and then use, if the overall is significant at the desired alpha, an appropriate multiple comparison tests to check further hypotheses.

Darshi. Arachige Dr, does that mean if I have four independent t-test, 6 paired t-test and 2 ancova. Would I only use alpha value divide by the number of independent t-test? For paired t-test and ancova I remain my alpha value as it is. Would you shed some light on this Dr Darshi. Arachige ? Thank you. God bless.

Tharishni Anblagan If I comment on this without knowing the background and the data, I can be misleading or confusing you. The best approach is to carefully look at the study design and reflect on which research questions should be asked from the data. This in practice gives me the succinct number of tests that I should do to get the answers. Thus, I will still be discussing your question in general terms. The situation depends on whether all 4 independent t-tests are about the same outcome. For example, you took measurements on four variables of some study subjects and wish to compare them 2 by 2. In this situation you can do one F test at, say 5% and do comparisons, if significant, with a 'proper multiple comparison test' that accommodate, multiple testing. If you wish, you can go down Bonferroni path and use a reduced alpha. For simplicity, my preference would be the first approach. It is difficult to comment on paired t-tests. If the six pairs of measures are taken on the same subject and want to compare using paired t-tests, I would think I am not increasing my alpha as each test can be considered as standing on its own, i.e. not comparing pairs of different variables. Again in case of ANCOVAs, it depends. Once you have done ANCOVA with multiple variables and a covariate, you may still decide to look at pairs of means of variables. I would tend to use the same alpha for the overall test and only if significant, use other comparisons. Hope my answer helps in your quest in some way. All the best with your research! (I am not entitled to be called 'Dr' as I am not a PhD. But, had been a statistician and a consultant for a considerable time. Thank you very much for your kindness, anyway. When you do independent t- tests , if you wish, you can be aware of the concerns raised in the attached.Technical Report Some Practical Considerations about the t-Test for Two Indep...

Dear Darshi. Arachige, thank you so much for your input and advice. Totally appreciate it. I will also read your write up. God bless.