For example, if you have 4 samples A, B, C, D:
A = Control sample
B, C, D are samples at different conditions.
Assume all assumptions are met. Can you just simply use t-test
Can you just simply use t-test to compare between each conditioned sample with A i.e A with B, A with C, A with D?
As far as I understand, ANOVA is used for more than 2 samples, but the way the test works is that it compares every two samples together i.e A/B, B/C, C/D, D/A, etc. rather than with 1 sample. So even though there will be pairs between the control and the conditioned samples, i would argue that it's kinda unnecessary? Or you have to use ANOVA then use Dunnett's test to test with the control?
I have had this question for quite a while now and just want to know opinions from people who are more experienced in statistics.
Thank you very much
Given the assumptions are resonable, Dunnett's test is perfect. You don't need to perform ANOVA. Dunnett's test uses a pooled variance estimate and controls the family-wise error rate (FWER).
ANOVA (only) is plain wrong. It tests something entirely different.
ANOVA followed by Dunnett is nonsense, as ANOVA is already not the right tool, and performing Dunnett's test only after the ANOVA is considered significant will not control the FWER at the desired level.
You should think if controlling the FWER is desired at all, and if so, at what level (depending on the context, 0.05 might be a rather stupid choice; this is even mentioned shortly in the linked Wiki article, too).
https://en.wikipedia.org/wiki/Dunnett%27s_test
For mean comparisons we need to calculate the error variance. When the number of samples is more than 2 or more, it is recommended to conduct ANOVA and estimate a common error variance for all comparisons, instead of t-test. In t-test, however, you need to calculate the error variance for each pairwise comparison separately. This means that the error variance for some comparisons may be higher and for some comparisons may be lower. By using ANOVA, instead, you use a single common error variance for all comparisons. This means that all pairwise comparisons are conducted with the same error and same precise.
For choosing which kind of multiple comparison methods, it depends on your objective. If the aim is to just comparing to control then you can use a method for comaring samples to control (e.g for LSD, Dunnett's test). If all comparisons is required, then you have to use another method to reduce the inflation Type I error (e.g Duncan ).
Hope it is helpful.
You have to go for ANOVA test (data should be in interval or ratio scale). if its found to be significant go for post hoc analysis like bonferroni or tukeys or LSD etc
Given the assumptions are resonable, Dunnett's test is perfect. You don't need to perform ANOVA. Dunnett's test uses a pooled variance estimate and controls the family-wise error rate (FWER).
ANOVA (only) is plain wrong. It tests something entirely different.
ANOVA followed by Dunnett is nonsense, as ANOVA is already not the right tool, and performing Dunnett's test only after the ANOVA is considered significant will not control the FWER at the desired level.
You should think if controlling the FWER is desired at all, and if so, at what level (depending on the context, 0.05 might be a rather stupid choice; this is even mentioned shortly in the linked Wiki article, too).
https://en.wikipedia.org/wiki/Dunnett%27s_test
Jochen: thank you very much.
Thank you others also for your contribution.
As Jochen has pointed out, there is no need for ANOVA prior to using Dunnett's test*. However, depending on what software you use, Dunnett's test may only be available as an option for an ANOVA procedure that will display the ANOVA table whether you need it or not. If that is the case with your software, just ignore the ANOVA summary table and go straight to the Dunnett's test results.
* This is true of most of the conventional multiple comparison procedures, actually (including the very popular Tukey HSD). The only common MC procedure that comes to mind where it is not is Fisher's LSD. It involves a two-step procedure: 1) Look at the omnibus F-test, and if it is statistically significant, proceed to step 2; 2) carry out all pair-wise contrasts using a modified t-test with the a pooled error term.
HTH.
@Bruce,
LSD is AFAIK the only multiple-comparison procedure that requires a significant ANOVA, as you said. I want to add that this is only the case when the number of groups is 3. For more groups, the FWER is not controlled at the desired level. Further, this method only provides a "weak control": if the omnibus test hypothesis is rejected, the rate of false rejections of the pair-wise comparisons is definitively higher than the desired FWER.
Hi Jochen. Yes, I completely agree that Fisher's LSD ought to be used only with k = 3 groups. For others who need references to support this (for skeptical bosses or supervisors), see David Howell's book Statistical Methods for Psychology or Thom Baguley's book Serious Stats.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics