In response to --> so why adjusting?

In a true experiment with random allocation to groups (i.e., an RCT) that has both baseline and follow-up measures on the outcome variable, the principle reason for including the baseline measure as a covariate is to reduce the error term. Variability in the follow-up measure (i.e., the DV) that is accounted for by the linear relationship between baseline and follow-up scores is partialled out of the error term. The cost is 1 df. But that cost is usually more than made up for by the reduction in SSerror.

I am a little confused about the design. SInce you are doing an RCT, the expected values of the baseline are the same, so are you conditioning/matching to increase power? Since your outcome variable is a gain score (post - baseline), why are you conditioning on baseline? Maybe by baseline you mean several baseline variables not including the pre score.

Daniel Wright, Luke said he would use the follow-up score or the change score as the outcome. And the F-test for Group is the same in either case.

Luke Norman, you might find this article useful:

Borm GF, Fransen J, Lemmens WA. A simple sample size formula for analysis of covariance in randomized clinical trials. Journal of clinical epidemiology. 2007 Dec 1;60(12):1234-8.

http://www.math.chalmers.se/Stat/Grundutb/GU/MSA620/S18/Ancova.pdf

HTH.

Bruce Weaver , I was unclear what he meant by post minus treatment, but you're right, he probably meant follow-up.

I believe the symbol in post-treatment is a hyphen, not a minus sign. ;-)

I don't know the answer to the original question, but ANCOVA always makes me queasy. Here is a really seminal paper by Miller and Chapman that is the likely emetic. Article Miller GA, Chapman JP. Misunderstanding analysis of covarian...

Since you seem to be planning a trial, can you discuss what baseline differences are you expecting to have across the treatment arms and why might ANCOVA offer a remedy for such differences? Or can you use these anticipated differences in some manner of stratified randomization? I have never thought that ANCOVA served to "correct" baseline differences especially if they occur non-randomly and are confounded with the DV.

Adding to the previous comment, it is not only about ANCOVA but about the idea of adjusting for baseline values in an RCT in general which can be a major source of bias. Here is an important editorial that very briefly clarifies many common misunderstandings of randomization and adjusting.

https://jamanetwork.com/journals/jamapsychiatry/fullarticle/2423197

To the original question: there are tutorials for conducting power analysis using a mixed effects approach which is probably more recommended here.

Hi Sebastian Trautmann ! It's nice to bump into you. We should probably have a huddle with Doug to find out how things are going with our fledgling collaboration.

Luke Norman Are you still at UM? I am fairly new here. If you are around, I am happy to find a time to meet.

David M. Fresco & Sebastian Trautmann, I may be mistaken, but I think the articles you've pointed to are mainly concerned with use of ANCOVA when the groups are naturally occurring or created by some means other than random allocation. Is that right?

Luke Norman has said that he is conducting an RCT, with the R meaning that the groups were created via random allocation. He has also suggested (if I understood correctly) that the baseline score for the outcome variable will be used as the only covariate. This use of ANCOVA is pretty uncontroversial, I think. See this BMJ Stats note, for example.

Article Statistics Notes: Analysing controlled trials with baseline ...

Apologies if I've misunderstood your concerns.

Bruce Weaver I gathered that Luke Norman was conducting a RCT, but I fear that the concerns of ANCOVA remain. This approach cannot salvage baseline differences that emerge even with careful sampling. At least that is what I have come to understand about ANCOVA. The Miller paper is cautionary broadly construed about the misuses of ANCOVA. The Chmura-Kreamer paper is more specific to random assignment, but I believe they arrive at similar conclusions. ANCOVA is NOT the great equalizer, absolver, corrective action, when there are theoretically conflated baseline differences. Apologies to Luke Norman if we have steered the question away from his intended direction but the issue remains regarding ANCOVA and if a scholar is planning a RCT or writing a grant application, I suspect the analysis plan would receive some score driving scrutiny.

Thanks for your reply, David M. Fresco. Just to be clear, I believe you are saying that in a true experiment with baseline and follow-up scores for the outcome, and random allocation to groups, an observed difference on the baseline measure is problematic, even though the expected difference = 0. Is that right?

Bruce Weaver That is a good summary

Thank you, David M. Fresco, for confirming that I was understanding you correctly. That view is certainly at odds with the more conventional view that ANCOVA is fine (and even recommended) when there are no expected differences on the covariate (due to the random allocation).

I hope you'll bear with me as I probe a bit more. How concerned would you be about use of ANCOVA (with covariate = baseline measure, DV = follow-up measure) with non-randomized groups if there were negligible differences amongst the groups on the covariate? Putting it another way, do the expected differences play any role at all in your thinking about this? Thank you for clarifying.

Hi David M. Fresco, yes, let's get together.

Bruce Weaver The paper I recommended does specifically discuss RCTs and the pitfalls associated with adjusting for baseline values.

Hello Sebastian Trautmann. My (admittedly) quick scan of Helena Kraemer's editorial suggested to me that she was more concerned about situations where people include covariates in post hoc fashion (because the groups differed significantly on those variables in Table 1, for example). On p. 962, she says:

"The bottom line is that covariates proposed a priori should always have strong rationale and justification and should be as few in number and as noncorrelated as possible."

Surely the case of a true experiment with random allocation to groups, and with covariate = baseline measure of the outcome and DV = follow-up measure fits that description. (I understand "as noncorrelated as possible" to be talking about correlations among covariates, and if the baseline value is the only covariate, this is a non-issue.) YMMV! ;-)

Bruce Weaver This part is true. But its is also about (1) the goal of randomization being a comparison of two random samples from the same population and not of two matched samples, and (2) the fact that ANCOVA assumes that there is no interaction between covariates and treatment.

Since these things a so often overlooked, my point was just that it is important think these things through and not to adjust for baseline values as a reflex.

Bruce Weaver and Sebastian Trautmann / David M. Fresco , is it possible that you are talking about two different designs? As I understood it, the OP was asking for a 2 factorial design, with 2 treatment groups and 2 measurement points, which Buce seemed to have in mind, and Sebastian and David (and the cited article by Kraemer) talked about a 1 factorial design, i.e. only one measurement point of the treatment effect, but where this will be adjusted for other covariates. Therefore, the former design uses the first measurement point t1 as covariable (i.e. the same variable as t2) and asses the adjusted effect at t2, whereas the latter design adjusts for completely different covariables as the DV itself (in the cited article the DV was the treatment effect and the CV was age).

If my understanding is correct, I would say that you three are all correct, the former design is quite uncontroversial as ANCOVA (see Huitema, 2011 for example), whereas the latter has the known problems of possible bias and interpretation. Since the OP asked for ANCOVA as alternative to a rm split-plot design, this should work nicely with a RCT design.

Rainer Duesing What you say makes good sense and is the most likely meaning as I re-read the original post. ANCOVA is such a triggering word for me, I guess.

Rainer Duesing I think the Kraemer article is specific about baseline characteristics, so there are at least 2 measurement points. But we indeed moved away from the original question, so sorry for that. But the area of adjusting is so full of bad common practices, so I guess this was my trigger.

Sebastian Trautmann I really can't find anything in the Kraemer article which considers 2 measurement points. Could you please provide the passage, maybe I am missing something.

Luke Norman to answer your question: maybe the best way to do an a priori power analysis it to simulate your design. It seems quite straigth forward and you will be able to test violations of assumptions. Typical software like G*Power does not do this and specifically, in the ANCOVA dialogue your are not able to include the t1 to t2 correlation of your treatment effect, which is used and varied in the Walters et al. article.

Rainer Duesing You are right, there is no statement about the number of measurements, but I assumed that from the term baseline measures, suggesting a pre-treatment measurement point. But maybe my main point (which I cited below) was too far away from the original question.

"Since in a randomized controlled trial no baseline factor is correlated with treatment choice, no baseline factor (including baseline symptoms) can be a confounder. " (Kraemer, 2016, Am J Psychiatry) --> so why adjusting?