Hello Catherine.  I think you need to provide more information.  Because you are talking about F-values, I suppose you are describing an OLS linear regression model with a hierarchical approach (i.e., entering one or more variables on the first step, then entering one or more on the second step, etc.)  Which F-value are you referring to, the overall F-ratio for the model, or an F-test for a variable that was in the model on both steps?  Thank you for clarifying.

Bruce

Assuming that you are talking about ANCOVA here, the most likely possibility is that the covariate is a suppressor variable...in that case, adding it to the model actually increases the effect of your main predictor (condition, or whatever it may be; see MacKinnon, Krull, & Lockwood, 2000 here: http://download.springer.com/static/pdf/426/art%253A10.1023%252FA%253A1026595011371.pdf?auth66=1418746746_4b7f61476d64d20ea6a90471c4df373b&ext=.pdf). In short, it's inclusion into the model controls for some unrelated portion of variance in the main predictor, making the effect you care about even stronger than without the covariate.

Yes indeed I was talking about ANCOVA Bruce. 

Thank you Lucas for your clear answer. I'll read the paper but I think I have already a better idea of the phenomenon. Thank you again.

Salut Catherine! J'espère que tu vas bien. Une réponse en français pour changer. Je suis d'accord avec les réponses de Lucas. J'ai été confronté au même résultat (étrange au départ mais qui a du sens au final). Un effet de l'âge sur la mémoire qui augmente quand on contrôle le niveau de vocabulaire. En effet, la mémoire sémantique (le vocabulaire) permet de diminuer les effets d'âge et sert en quelques sortes de réserve cognitive aux personnes âgées. Mais si tu contrôles le score de vocabulaire alors tu augmentes l'effet d'âge donc le F-value.

The F ratio is the ratio of MS for an effect to the MS for the error term. The MS (mean square) is the average SS (sums of squares) per degree of freedom. In ANOVA and ANCOVA the F ratio for an effect can therefore increase because the MS for an effect has gone up (which can happen with suppression) or the MS error has gone down (because the new variable adds explanatory value - meaning that it removes error variance and hence is what might usually be termed a confounding variable), or both.

If it is the F ratio for the model it implies that MS error for the model has been reduced and MS model has increased.

@ Catherine & Badiâa (aux autres aussi, mais réponse en français pour une fois...) : cela suggère aussi sans doute que, sans la covariable ajoutée, les conditions d'application ne sont pas vérifiées : résidus mal centrés, ou pas gaussiens sur la droite de Henry, ou encore avec des variances dissemblables... Tous indices qui suggèrent qu'il manque quelque chose dans le modèle.

L'idée étant alors que ce qui manque est compté comme du bruit, et non du signal, et masque donc le signal cherché, d'où le F qui diminue sans cette variable (puisque le F n'est jamais qu'une mesure du rapport signal/bruit...)

Thank you everyone, it's a lot clearer to me and I'm feeling better with the rationale and meaning of my analyses

I have a similar problem in process mediation. I the variable I want to use as a control variable has a suppressor effect. My IV predicts the DV as long as I control the "suppressor." But actually, it is not unlogical according to theory my model based on. But I don't know what the correct way is statistically and how to interpret this. Is it wrong to include the suppressor as a control variable?

May someone help me with the nature of "suppressor variable" ?

Thanks in advance

Pınar Acet I am not an expert at all (just started using ANCOVA myself). However, I think it it a good idea to include the "suppressor variable" (I would just have called it a covariate in your model). For my data for example, I have two groups (following different interventions), and each individual has been measured at baseline and follow-up. Here I then use the baseline-values as covariate in my model, like this: Follow-up ~ Baseline + Group. When running this model, I will get an estimate for the "covariate" Baseline and for the factor Group. When I am reporting the result, I will state something like this (if the Group factor is significant): "There was a significant effect of Group at follow-up after controlling for the effect of baseline, F(df,df)=..., p=...". If you want more comments on your problem, maybe you could give some more detailed information about your design and what you want to get out of your analysis. If you are using R for your analysis, I recommend this ANCOVA guide as a "starting point", at least it helped me:

https://www.datanovia.com/en/lessons/ancova-in-r/

Thank you very much Ann Marie for your answer and the link will certainly be helpful.

Anne Mari Herfindal the experimental design suggests you may simply do a paired T-test of Follow-up - Baseline [or on log Follow-up/Baseline] to test for a Group effect on follow-up. This is a special case of ANCOVA assuming the slope for Baseline is 1, but is also much easier to interpret, check, present and so on. Because you just asks then "Are the difference Follow-up - baseline different between Groups", which is straightforward to understand.

Note however that if you need to have a slope different from 1, don't forget to check for linearity… and for an interaction between Baseline and Group.

Emmanuel Curis , thanks for the feedback! :) I'm quite new to "ANCOVA" (or similar types of regression models with both continuous and factor variables), just so that's clear...

You say "the experimental design suggests you may simply do a paired T-test of Follow-up - Baseline [or on log Follow-up/Baseline] to test for a Group effect on follow-up". I am not sure what you meant by paired T-test here. Could you try an explain again? Did you mean that I for all individuals calculate some sort of change value (follow-up minus baseline or some sort of ratio), and then just test if the change is different for the two groups by performing a independent t-test? If this is what you meant, I do not agree I think. At least from what I have read about the topic, people recommend adjusting for baseline in the model as a covariate, which is not the same as calculating the change score. To me that made sense at least. But maybe I misunderstood what you meant? I'm trying to get my head around "This is a special case of ANCOVA assuming the slope for Baseline is 1"... Please help!

I decided to use the model based on different sources, including (but not limited to) the papers and book I have linked below (the second paper as actually about cases where you have multiple repeated measurement, so they apply mixed model with individual as random, but the general idea when it comes to the "adjustment for baseline" (or what you want to call it) is transferable to this "simple ANCOVA" I'm modeling here).

Before I settle with the model Follow-up ~ Baseline + Group, I always have some inspection of the linearity (visual by plotting and by running lm(Follow-up~Baseline) for both groups) followed by fitting the model Follow-up ~ Baseline*Group to see if there is any interaction between baseline and group that I should pay attention to. I guess that is what you meant by " Note however that if you need to have a slope different from 1, don't forget to check for linearity… and for an interaction between Baseline and Group"?

Analysing controlled trials with baseline and follow up measurements: Article Statistics Notes: Analysing controlled trials with baseline ...

Different ways to estimate treatment effects in randomised controlled trials: Article Different ways to estimate treatment effects in randomised c...

Misunderstanding analysis of covariance: https://pubmed.ncbi.nlm.nih.gov/11261398/

Medical statistics in clinical and epidemiological research: https://www.gyldendal.no/faglitteratur/medisin/laereboeker/medical-statistics/p-93120-no/

Anne Mari Herfindal

Thanks for the references. I would say as a quick answer that the choice is delicate and depends strongly on what you expect to occur in your sample, with and without treatment, and the relative importance of within-patient and between patients variabilities...

Yes, indeed, when I spoke about paired T-test, I was indeed thinking about comparing the differences between the two groups; that was a confusing shortpath, sorry.

I'll try to post a more detailed answer later, when I'll have time, but meanwhile, you may find interesting to think about the (mathematical) model that gives the link between time (before[=baseline], after) and outcome, including patients effects, treatment effects and randomness. Not as a software command line, but really as the mathematical equations. You may arrive at some of the equations of the second paper you gave... or not!

Anne Mari Herfindal

Now that I have a little time, a few ideas on the underlying model and its variants, to discuss advantages and limitations of various methods depending on the situation.

The easy part is for the baseline measurement, which can be said to occur at time = 0. Let call Y[i,1] this measure for patient i. It can be described as the sum of

— a mean value (population value), µ[1],

— an eventual correction of this mean value in case of treatment, d[1]×1[T] where 1[T] is the indicator of being in group treated. Note that d[1] should be 0 for a correct randomization, but there's no need here for this hypothesis,

— a correction for the fact that it is this patient specifically, u[1,i], that is (the realization of) a random variable with mean 0 and variance s²[0] and is the « between patients » variability,

— and an error term (let's say, measurement error), e[1, i], that is (the realization of) a random variable with mean 0 and variance s²[1], and independent of u[1,i], and is the « within patient » variability,

so Y[i,1] = µ[1] + d[1]×1[T] + u[1,i] + e[1, i], E(Y[i,1]) = µ[1] + d[1]×1[T] and V(Y[i,1]) = s²[0] + s²[1].

This model is not identifiable, unless there are several repetitions of the measure for the same patient at that exact time, but that's not the point here.

Now, the second measurement, Y[i,2], at time t > 0 — to a unit change, we can assume t = 1.

A first idea is to use the same model, Y[i,2] = µ[2] + d[2]×1[T] + u[2,i] + e[2, i], but it makes the effect of change induced by the treatment difficult to see, hence to estimate and test. Beside, dependencies between the two measures is not obvious and will appear only through dependencies between u[2,i] and u[1,i]. This idea leads to "GEE" methods, but will not be developed further here.

A second idea is to introduce a change with time. Since here there are only two measures, a straight line is usable, hence Y[i,t] = a[i] + b[i]×t + epsilon[i,t]. Assuming that the treatment makes something means b[i] = b0[i] + d[b]×1[T], where b0[i] is the natural change without treatment. Assuming a different evolution between patients means b0[i] = b0 + u[b,i] where u[b,i] is a random variable of mean 0 and variance s²[b]. Last, when t = 0, Y[i,0] = a[0] + epsilon[i,0] = Y[i,1], which gives a[0]. That makes Y[i,2] = µ[1] + d[1]×1[T] + u[1,i] + (b0 + d[b]×1[T] + u[b,i])×t + e[2, i]

This model introduces a correlation between the two measures, that depends on the relative variances (and correlations between u[1,i] and u[b,i]). It naturally leads to study Y[i,2] - Y[i,1] = (b0 + d[b]×1[T] + u[b,i])×t + e[2, i] - e[1,i], with mean b0 + d[b]×1[T] — hence a T-Test between the two samples — and variance t² s²[b] + s²[2] + s²[1], which ma be quite high if s²[b] is great, that if if there is a strong variability of changes between patients (that's even more true if the treatment effect, d[b], is also assumed to be different between patients).

A third idea is to say that Y[i,2] = a + b * Y[i,1] + d[a]×1[T] + e*[i] — that is, the analysis of covariance model. By replacing Y[i,1] by its own model, one has Y[i,2] = a + b µ[ 1 ] + b d[1]×1[T] + b u[1, i] + b e[1,i] + d[a]×1[T] + e*[i]. Identification with the previous model gives b0 t = a + (b - 1)µ[ 1 ], d[b] t = d[a] + (b - 1) d[1], u[b,i] t = (b - 1) u[1,i] and e[2,i] = e*[i] - b e[1,i] — which, with t = 1, gives the final link between the models.

However, assumptions for the analysis are different for each model. The covariance model has assumptions on e*[i], that is on e[2, i] + b e[1,i], and imposes a correlation between the slope and the intercept, that is u[b, i] = (b - 1) u[1,i]. If b = 1, this means u[b,i]=0, hence all patients have the same "natural" evolution [which could be b0 = 0, no evolution], and this also a case where the T-test has not the t² s²[b] part of the variance, hence a reduced variance...

You can play with these models to see that 1) T-test on changes is indeed a special case of the ANCOVA when b = 1, 2) depending on the relative variances of the different effects, some methods could be better than some others etc etc.

You may even generalize the model to more complex cases ;)