See these links:

Article Power Analysis and Sample Size Planning in ANCOVA Designs

https://www.sheffield.ac.uk/polopoly_fs/1.531229!/file/MASH_ANCOVA_SPSS.pdf

https://www.pepgra.com/blog/clinical-writing/estimation-of-sample-size-and-power-analysis-for-clinical-research-studies/

https://www.researchgate.net/post/ANCOVA_controlling_baseline_in_RCT-how_to_perform_power_analysis_and_sample_size_estimation

Hello Joost,

In addition to the useful links offered by Remal Al-Gounmeein, let me answer your question analytically.

To the extent that a covariate reduces the otherwise unexplained variance in a DV, unless the sample size is exceedingly small, the loss of one df for the covariate (in adjusting from an independent t/one-way anova to an ancova) is more than offset by the reduction in the error variance.

So, a power analysis based on unadjusted means is likely to yield a higher-than-needed N (to achieve a given power to detect a given ES) than would a power analysis based on means adjusted for a covariate. The resultant power would be higher for the ancova than that targeted for the unadjusted means comparison (using the same N in both approaches). The only time this wouldn't hold is when the correlation between covariate and DV was zero.

Why did the authors of the study you cite use one method for power analysis and another for data analysis? While one possibility is that they had no a priori sense of the strength of correlation of covariate with DV scores, I suppose you'd have to ask them!

Good luck with your work.

David Morse nicely laid it out all. Although, methodologically, it is unlikely for the test statistics to suffer from low power, such design is not economically efficient under some circumstances (referring to the design and analysis discrepancy in the article).

In addition to resources provided by Remal Al-Gounmeein you may want to check this tutorial I recently wrote, which I think is relevant to your case:

Article Sample Size Determination and Optimal Design of Randomized/N...

For your second question, it may be more complicated than you may think. The following articles may be helpful for the multiplicity problem:

Article Statistical Power in Evaluations That Investigate Effects on...

Article Methods to adjust for multiple comparisons in the analysis a...

Good luck with your work.

Thanks a lot Remal, David and Metin! @Metin, do you agree that the power analysis (based on a two-sample t-test) is theoretically not correct when you perform ANCOVA for the the analysis of the primary endpoint?

Joost Boeckmans, devising an experiment based on a two-sample t-test may not be economically efficient, if one collects pretest or other covariates at the baseline (due to resource limitations?). Assuming that pretest or covariates explain some of the variation in the outcome one would need a smaller sample size for the ANCOVA design. In other words, sample size required to conduct a t-test (unadjusted mean difference) is larger than the sample size required for the ANCOVA design (adjusted mean difference). Since in the article they determined the sample size based on the t-test but continued the analysis with the ANCOVA design (for which a smaller sample size would have been sufficient), I would not say it is wrong. The bigger the sample size the better anyway.

Small note: One-way ANOVA with 2 groups is the same result as an independent samples t-test because t^2 = F with 1 df in the numerator. An independent samples t-test will produce the same result as the t-test of a dichotomous predictor in a bivariate regression, as well. All are based on least squares normality principles.

in Stata:

. clear

. set obs 2

Number of observations (_N) was 0, now 2.

. gen t = _n-1

. drawnorm mu

. expand 100

(198 observations created)

. gen y = rnormal(mu)

. gen x = .5*y+rnormal()

. anova y t

Number of obs = 200 R-squared = 0.6719

Root MSE = .889144 Adj R-squared = 0.6702

Source | Partial SS df MS F Prob>F

-----------+----------------------------------------------------

Model | 320.49458 1 320.49458 405.39 0.0000

|

t | 320.49458 1 320.49458 405.39 0.0000

|

Residual | 156.53411 198 .79057629

-----------+----------------------------------------------------

Total | 477.02868 199 2.3971291

. ttest y, by(t)

Two-sample t test with equal variances

------------------------------------------------------------------------------

Group | Obs Mean Std. err. Std. dev. [95% conf. interval]

---------+--------------------------------------------------------------------

0 | 100 -1.491209 .0888933 .8889334 -1.667593 -1.314825

1 | 100 1.040568 .0889354 .8893537 .8641004 1.217035

---------+--------------------------------------------------------------------

Combined | 200 -.2253207 .109479 1.548266 -.4412084 -.0094329

---------+--------------------------------------------------------------------

diff | -2.531776 .1257439 -2.779746 -2.283807

------------------------------------------------------------------------------

diff = mean(0) - mean(1) t = -20.1344

H0: diff = 0 Degrees of freedom = 198

Ha: diff < 0 Ha: diff != 0 Ha: diff > 0

Pr(T < t) = 0.0000 Pr(|T| > |t|) = 0.0000 Pr(T > t) = 1.0000

. di r(t)^2

405.39361

. reg y i.t

Source | SS df MS Number of obs = 200

-------------+---------------------------------- F(1, 198) = 405.39

Model | 320.494576 1 320.494576 Prob > F = 0.0000

Residual | 156.534105 198 .790576289 R-squared = 0.6719

-------------+---------------------------------- Adj R-squared = 0.6702

Total | 477.028681 199 2.39712905 Root MSE = .88914

------------------------------------------------------------------------------

y | Coefficient Std. err. t P>|t| [95% conf. interval]

-------------+----------------------------------------------------------------

1.t | 2.531776 .1257439 20.13 0.000 2.283807 2.779746

_cons | -1.491209 .0889144 -16.77 0.000 -1.66655 -1.315868

------------------------------------------------------------------------------

. anova y t c.x

Number of obs = 200 R-squared = 0.7417

Root MSE = .790887 Adj R-squared = 0.7391

Source | Partial SS df MS F Prob>F

-----------+----------------------------------------------------

Model | 353.80481 2 176.90241 282.82 0.0000

|

t | 155.09786 1 155.09786 247.96 0.0000

x | 33.310239 1 33.310239 53.25 0.0000

|

Residual | 123.22387 197 .62550186

-----------+----------------------------------------------------

Total | 477.02868 199 2.3971291

. reg y t x

Source | SS df MS Number of obs = 200

-------------+---------------------------------- F(2, 197) = 282.82

Model | 353.804815 2 176.902407 Prob > F = 0.0000

Residual | 123.223866 197 .625501859 R-squared = 0.7417

-------------+---------------------------------- Adj R-squared = 0.7391

Total | 477.028681 199 2.39712905 Root MSE = .79089

------------------------------------------------------------------------------

y | Coefficient Std. err. t P>|t| [95% conf. interval]

-------------+----------------------------------------------------------------

t | 2.047695 .1300399 15.75 0.000 1.791246 2.304144

x | .3969896 .0544007 7.30 0.000 .2897071 .5042721

_cons | -1.233082 .0866383 -14.23 0.000 -1.403939 -1.062224

------------------------------------------------------------------------------

. lincom t

( 1) t = 0

------------------------------------------------------------------------------

y | Coefficient Std. err. t P>|t| [95% conf. interval]

-------------+----------------------------------------------------------------

(1) | 2.047695 .1300399 15.75 0.000 1.791246 2.304144

------------------------------------------------------------------------------

. di r(t)^2

247.95747

.