It is valid to use inverse probability weighting (IPW) to “adjust” for bias that can arise from loss to follow-up, but it has limitations.

Inverse probability weighting was developed for surveys in the 1950s (Horvitz and Thompson, 1952). The use of these weights solves the problem caused by surveyees having unequal probabilities of being sampled. For a cohort study, the method can be used to yield inferences which apply to the population you sampled from initially, as opposed to inference which apply only to the population of those who do not drop-out.

The IPW method begins by generating a model for remaining in the study. The predicted values of this model are calculated, and the IPW is the reciprocal of this predicted probability of remaining in the study. For example, a subject who has remained in the study, but according to the model had only a 1% chance of remaining in follow-up, is given the weight of 100. The rationale is they represent not just themselves but 99 others “like” them who are not represented in the follow-up sample.

The model is built as a function of subject characteristic (e.g., age, gender). Anything that might be related to follow-up should be part of the model building process. This brings up the issue of characteristics predictive of follow-up that are not measured (e.g., collected as part of the study). If any such characteristics exists, which are in turn related to the endpoints you will ultimately be estimating with your weighted models, you will incur bias.

It is also useful to draw a distinction between characteristics that predict follow-up in the future and characteristics that are cross-sectionally related to loss to follow-up. For example, it could be that baseline characteristics have little ability to predict loss to follow-up but loss to follow-up is correlated with the dependent variable of interest., such as any study of a disease in which patients with the most severe disease do not return for follow-up. The use of IPW can do nothing to resolve this type of informative drop-out.

It is valid to use inverse probability weighting (IPW) to “adjust” for bias that can arise from loss to follow-up, but it has limitations.

Inverse probability weighting was developed for surveys in the 1950s (Horvitz and Thompson, 1952). The use of these weights solves the problem caused by surveyees having unequal probabilities of being sampled. For a cohort study, the method can be used to yield inferences which apply to the population you sampled from initially, as opposed to inference which apply only to the population of those who do not drop-out.

The IPW method begins by generating a model for remaining in the study. The predicted values of this model are calculated, and the IPW is the reciprocal of this predicted probability of remaining in the study. For example, a subject who has remained in the study, but according to the model had only a 1% chance of remaining in follow-up, is given the weight of 100. The rationale is they represent not just themselves but 99 others “like” them who are not represented in the follow-up sample.

The model is built as a function of subject characteristic (e.g., age, gender). Anything that might be related to follow-up should be part of the model building process. This brings up the issue of characteristics predictive of follow-up that are not measured (e.g., collected as part of the study). If any such characteristics exists, which are in turn related to the endpoints you will ultimately be estimating with your weighted models, you will incur bias.

It is also useful to draw a distinction between characteristics that predict follow-up in the future and characteristics that are cross-sectionally related to loss to follow-up. For example, it could be that baseline characteristics have little ability to predict loss to follow-up but loss to follow-up is correlated with the dependent variable of interest., such as any study of a disease in which patients with the most severe disease do not return for follow-up. The use of IPW can do nothing to resolve this type of informative drop-out.

IPW can only handle drop-outs that are predictable using observed information. If the drop-out depends on the future time of the outcome, which is obviously unobservable, then we cannot capture that dependency in a reliable way without making speculative assumptions.

When you use the technique of IPW, it is preferable to use stabilized IP weights. They are less variable, and hence yield a more efficient estimator. This is how stabilized IP weights are calculated.

Suppose that R_t is a binary variable denoting whether or not an individual was in the study, with R_t = 0 denoting dropout; and X is a vector of fixed baseline covariates, and H_t is the history up to time t (this includes all time-varying covariates). We can calculate Stabilized IPW as: Prob(R_t = r) / Prob(R_t = r | H_t, X), where r = 0, 1.

There is a nice paper in the J Statistical Software that demonstrates the IPW methodology with a nice R package.

van der Wal and Geskus (2011). ipw: An R Package for Inverse Probability Weighting, J Statistical Software, vol. 43.

In my experience, non-informative drop-out is fairly rare, so IPW probably would not be my first choice here. Having said that, the other methods that make the missing at random (MAR) assumption are all susceptible to the same kinds of bias.

If you have any way of measuring patient-level trends prior to dropout, those data have the potential to help you build a less biased model, but that has limitations as well. If you're only using baseline data to get at the selection bias, it's almost certainly not enough.

I disagree, however, that conclusions must be in doubt if you have more than a few percent who are lost to follow-up. We should all do everything possible to avoid loss to follow-up, but I think a few percent is an unreasonably high standard for many studies. It depends entirely on the size of the effect that you've observed with your available data and the plausible explanations for missing data and how those explanations are likely to be correlated to the outcome of interest.

I think what is most important, in using any of these methods, is to acknowledge as a limitation that they do not offer a complete adjustment for bias and that the adjustment may be pretty minimal if the true reasons for dropout would lead to very different expected outcomes for the patients who dropped out (e.g., if transplant is a reason for study dropout).

Bias is a measure of incomplete study design and thus cannot be manipulated statistically if it's there it is there just my two cents

I disagree, to some extent, with Jay because his statement is too strong. If we understand the nature of incompleteness, then we can still obtain useful results, under justifiable assumptions. For example, it is sometimes possible to obtain useful bounds on effects of treatment/exposures, even with a biased study design.

Depending on the proportion missing if it is lunch there's no way to control for bias bias is just biased

Depending on the proportion that all missing selection bias is selection bias there is no way to get rid of it

I think of IPW methods for assessing bias due to loss to follow up as a sensitivity assessment for the main analyses. I think you can often do a decent job of modeling predictors of loss to follow-up, but I doubt you can fully explain it with the data that is usually at hand. It is possible that unmeasured factors that predict loss to follow-up may cause bias even after IPW methods are applied. But I think these approaches provide a "sanity check" on the main analytical results and let you assess the general magnitude of bias that loss to follow-up may be causing.

I think you are talking about unmeasured confounders which can occur in any epidemiological study but it's not bias

bias has to do with faulty design

Thank you very much for all your valuable comments and suggestions, which help me a lot!

Okay if you want to reinvent the wheel that's fine with me and

All I have to say on the subject

My experience is that IPW usually comes from fix sampling fractures. It helps but induce large variance. That is, it is not optimal. I recently developed a new method that use a conditional prob of cases or controls given others as the corresponding weight. It will be public ally available soon.

I might be getting old but how do you make missing data

NonMissing

I think missing data can be independent problem.

Anything we do with missing data is just a statistical guess when something is missing it is missing

And please remember that statistics Is only eig

A tool improve science it is not in in in itself when I was young I made that mistake of thought

Let me try this again please remember that statistics are only A tool to aid in

scientific or health discovery it is not an end unto itself

when I was young I made that error of thought

Chuanhua Xing called attention to the potential large increase in variance when using IPW estimates. This increase in variance due to the variance in the weights for observations was noted in the older sampling literature on weighting for probability of response. Regression type estimators of the weights (e.g., logistic regression estimators of probability of response) are particularly prone to this problem since 1) potentially every subject can have their own weight; and, 2) extreme values of predictors can lead to extremely large or small weights.

I haven't looked into this area for a number of years. A brief survey of these issues is found in Kish, Weighting for unequal Pi, Journal of Official Statistics, 1992; 8:183-200, where it is noted that choice of weighting scheme involves a trade-off in the MSE of the resulting estimate between increased variance and bias reduction. Among solutions consider to reduce this trade off were grouping subjects into a small number of strata with similar probability of response, and using stratum average weights and shrinkage estimators for the weights. I would be interested in hearing how much this literature in the survey sampling area has influenced the practice of weighting in the IPW literature.

If data is missing completely at random then there is no expectation that the variance will change nor the mean scores,

In this case that data is not missing at random, there is a distinct bias, consequently the variance of the follow up set is artificialy low, that is, a significant source of variance has been removed

hence the increase in variance is returning the data to a closer estimate,

however taking the advice of Jay Fisher above missing data is exactly that, missing,

the use of multiple imputation and other regression based models aim to provide a best guess of the likely answer for missing data, and variance and means should alter when the data is meaningfully missing, in the best scientific tradition you could attempt to predict the direction and magnitude of these changes prior to running the analysis.

Agree

Since your issue is selection bias in your study itself, not use or non-use IPW, is there a way methodologically to correct for your bias? I.e. can you re-sample your population? In addition, if you have a reasonable estimate of how long the drop outs would survive you can include that parameter in your model, which could lead to using tools in addition or as a replacement for IPW.

I'm not a biostatistician so I'm not sure what all your options are, but I would suggest choosing an outcome measure that maximizes the data you obtain from the dropouts, and look to see what underlying factors the drop-outs share. This could lead to you better understanding what issues are relating to this population, which could have clinical or epidemiological significance.

Good luck!

How about a sensitivity analysis if you assume that the missing subjects inone group all did well then analyzes it that way if you find results your results will be conservative

Dear Jay Fleisher,

I am not a biostatistician by training. I couldn't understand your question on using sensitivity analysis. Could you explain a little more please.

It's not a statistical thing it's a logical thing perhaps I used the word sensitivity analysis in the wrong contexts.

Make a conservative assumption about the missing people in group one make the assumption that they all did better

then analyze your data if you get

results they will be conservative if you don't there's nothing else I can think of

Thanks. I think Robert Brooks reminded us of something important: to decide on what you want to do prior to running the analysis, even better to decide prior to collecting the data. That would reduce potential bias.

Yes that is the key to develop your design in mind with how you're going to analyze it that separates out the boys from the men/women

I think there is a problem in the study design itself. Dr Jason is searching for the risk factors of frail and saying "the subjects lost to follow up were more frail at baseline". In the cohort study the exposure should be followed i.e. risk factors and end up with the incidence of frailty.Therefore I think he should not find any case of frailty at baseline, thus the proper design is the case control one in this case.

I agree, if you are using a cohort design to investigate the risk factors for developing frailty, then it would be inconvenient to have frail subjects at baseline. If this is the case, and if you exclude from the analysis the "frail subjects at baseline" who were lost to follow-up, is this still even considered "selection bias"?

The key issue here is whether the follow-up duration is comparable between exposed and unexposed groups. If they are, it is not a big issue. If not, I would recommend to either use incident risk ratio or match on the duration of follow-up.