In text-book statistics, one can use the one-sample t-test for comparing a population mean to a specific value. What would be the correct way to conduct a Dunnett's procedure of comparing multiple means to a specific value?
Specifically, in normal Dunnett's procedure for testing sample mean µi against a control mean µc one can use
t = (µi - µc) / sqrt( MSE(1/ni + 1/nc) )
where MSE is the pooled variance of all groups including the control. What should the formular look like when comparing to a specific value, rather than a control mean, and how should the degrees of freedoms be modified when obtaining the critical Dunnett's t-statistic?
Use the pooled variance from all samples to the the standard errors of the mean estimate. When S² is the pooled variance, then SEi = sqrt(S²/ni) is the (pooled) standard error of the i-th group (with a sample size of ni).
Calculate the t-values as ti = (mi-µ0) / SEi where mi is the sample mean of the i-th group and µ0 is the value for the null hypothesis.
Get the p-values from the t-distribution with N - k degrees of freedom (N is the total number of values, k is the number of groups).
Use the pooled variance from all samples to the the standard errors of the mean estimate. When S² is the pooled variance, then SEi = sqrt(S²/ni) is the (pooled) standard error of the i-th group (with a sample size of ni).
Calculate the t-values as ti = (mi-µ0) / SEi where mi is the sample mean of the i-th group and µ0 is the value for the null hypothesis.
Get the p-values from the t-distribution with N - k degrees of freedom (N is the total number of values, k is the number of groups).
I like Jochen's suggestion. But note that unlike Dunnett's test, it is not maintaining any control over the family-wise alpha. How important that is depends to some degree on how many groups you have. If you have a lot of them, you might want to use a Bonferroni correction, or maybe the Benjamini-Hochberg FDR method.
HTH.
I like Jochen's and Bruce's suggestions. Just to add a useful test to Bruce's list, Holm's correction is often viewed as an alternative to Bonferroni's correction. It is the default in R's p.adjust function. It `dominates' Bonferroni's. Here they are compared with 5 p values.
> p.adjust(c(.1,.012,.009,.5,.6),method="bonf")
[1] 0.500 0.060 0.045 1.000 1.000
> p.adjust(c(.1,.012,.009,.5,.6))
[1] 0.300 0.048 0.045 1.000 1.000
>
Thank you both for your swift replies. Jochen's approach is elegant and intuitive, but using the quantile of the t-distribution with N-k degrees of freedoms as critical values will be insufficient to control the family-wise error rate, as far as I understand from Bruce's answer.
Would I be able to challenge the t-statistic calculated from Jochen's approach with Dunnett's critical statistic, for the N-k and k degree of freedoms, and thus adjust for multiple testing? I am able to calculate the quantiles for the noncentral Dunnett's test distribution for any degree of freedoms.
You are right, the p-values are not corrected to control FWER or FDR.
It is usually a very interesting question if one should control for multiple testing at all (on scientific grounds - not because the rewievers unreflectedly want it anyway...) and what actually should be controlled (no need to mention that it is a stupid argument to control the FDR because it's more "liberal").
Ok, given the FWER should be controlled, then I think it is possible to simply substitute the t-values from the t-distiribution with the corresponding quantiles for Dunnett's distribution.
If the set of data agree with assumptions of ANOVA , you can perform Duncan Multiple Ranges Test by use MSE to calculate LSR
I'd suggest Googling "Tukey's Honest Significant Difference" (see link, for example). I'd also suggest questioning the meaning of your test : do you really have a scientifically interesting hypothesis to test ?
I'm currently having a similar problem and found this might be a viable solution:
https://ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/Procedures/PASS/Multiple_One-Sample_or_Paired_T-Tests.pdf
In short, yes use as many one samples as you need then shove a restrictive correction technique such as Bonferonni on top.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Modeling
- Analyses of Variance