Dear all,
I am working with my resident survey data (n=8356) including 59 items, most of which are ordinal variables scored from 1 to 7, and others are continuous variables (e.g., age, residency length).
However, all items have some missing values, ranging from 40 to 200, which follow an arbitrarily missing pattern. This leaves me with 6001 complete observations but more than 2000 incomplete records. How can I impute the missing values in this situation?
I realize that I could use single or multiple imputation method, while it seems that both require variables without any missing value, which is not the case in my data.
Additionally, I want to generate some composite variables based on the individual items and then run my analysis model, so how to get the combined estimates if I use multiple imputation method such as chained equations?
Thanks for any help.
Hi,
Paul Louangrath with all the respect, you have no clue what you are talking about.
I always hesitate to be so harsh but associating modern missing data procedures with ethical violations is not only lacking any foundation but is also an inappropriate offense against Maoliang Ling
Now to the issue: Maoliang Ling,
multiple imputation or estimation methods (such as FIML) assume missing at random (MAR) that is that the missing-generating process (or its causes) is conditionally unrelated with your dependent variable Y. In plain words that means that variable(s) that cause missing data in Y may correlate with Y but this correlation does not exist anymore when you control or partial out one or more of the model variables. Here is a nice and easy graph theoretical introduction of the central concepts MCAR, MAR and NRM:
Thoemmes, F., & Mohan, K. (2015). Graphical representation of missing data problems. Structural Equation Modeling: A Multidisciplinary Journal, 22(4), 631-642.
In these cases imputation and FIML always work better than listwise deletion; and pairwise only works under MCAR (missing completely at random; that is the missing process is uncorrelated with the model variables--even Y). The common sense in the literature is NEVER to use listwise deletion because it combines bias with lack of efficiency & power.
In addition, the goal of imputation is not "inventing data that does not exist" but to recover higher order statistics (such as means an variances/covariances) that are biased by the missingness. In addition, you can include "auxiliary variables" which are correlated with Y and the missingness status to use this information in the imputation process (see the McKnight et al. book below to get some practical tips).
In their review of missing data methods, Abraham et al. report papers that even imputed large amounts of missing data (up to 62%). Enders and Bandalos (2001) show that listwise is always bad and pairwise is bad unless the missingness is MCAR. In MAR conditions, pairwise is bad.
Abraham, W. T., & Russell, D. W. (2004). Missing data: A review of current methods and applications in epidemiological research. Current Opinion in Psychiatry, 17, 315-321.
Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8(3), 430-457.
The essential point is: It is not the size, but the causal process leading to MI. This is very similar with other self-selection or attrition processes leading to persons move in or out of the study:
Elwert, F., & Winship, C. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40, 31-53. doi:10.1146/annurev-soc-071913-043455
Haneuse, S. (2016). Distinguishing selection bias and confounding bias in comparative effectiveness research. Medical Care, 54(4), e23.
That is: if you goal is to estimate a regression or causal model, and your dependent variable suffers from missingness, then traditional deletion will lead to bias of the effect but with imputation/FIML to unbiasedness and higher efficiency when the process is not MCAR (which is very likely).
Beyond the excellent recommendation by Rosario, here is a further list of references one should read in order to get an idea:
As the most practical introduction, I would recommend the book by McKnight et al., since it overs not only a good introduction into imputation and FIML but also practical tips how to analyse the amount, pattern and correlates of missing data.
The book by van Buuren is about his MICE package in R. Software packages such as lavaan, Stata for estimating SEM have FIML implemented.
HTH
Holger
- - - - - - - - - - - - - - - - - - - - - - -
Allison D, P. (1987). Estimation of linear models with incomplete data. Sociological Methodolgy, 17, 71-103.
Collins, L. M., Schafer, J. L., & Kam, C.-M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330-351.
McKnight, P. E., McKnight, K. M., Sidani, S., & Figueredo, A. J. (2007). Missing data - a gentle introduction. New York: Guildford press.
Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549-576.
Graham, J. W., Olchowski, A. E., & Gilreath D, T. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8(2), 206-213.
Newman, D. A. (2014). Missing data: Five practical guidelines. Organizational Research Methods, 17(4), 372-411. doi:10.1177/1094428114548590
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147-177.
Schlomer, G., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57(1), 1-10.
Van Buuren, S. (2012). Flexible imputation of missing data. Boca Raton: CRC Press Inc.
MISSING DATA means incomplete or missing data points. You have 2000 (two thousand) of such items and you want to "impute"---meaning put in data that does not exist? A common sense approach is to work with the "complete" data that you have, as for the incomplete 2000 items are treated as defect. To "impute" or make up 2000 pieces of data that do not exist raises the issue of data integrity and ethical lapses of the researcher. This crosses the line of methodological innovation to an issue of ethical violation.
An accessible and short introduction to missing data is Allison, P, Missing data, Sage series. There you can find a non technical explanation on the topic. In any case you have a 1/4 of your data missing and this raises substantial issues that no tecnique could solve.
Hi,
Paul Louangrath with all the respect, you have no clue what you are talking about.
I always hesitate to be so harsh but associating modern missing data procedures with ethical violations is not only lacking any foundation but is also an inappropriate offense against Maoliang Ling
Now to the issue: Maoliang Ling,
multiple imputation or estimation methods (such as FIML) assume missing at random (MAR) that is that the missing-generating process (or its causes) is conditionally unrelated with your dependent variable Y. In plain words that means that variable(s) that cause missing data in Y may correlate with Y but this correlation does not exist anymore when you control or partial out one or more of the model variables. Here is a nice and easy graph theoretical introduction of the central concepts MCAR, MAR and NRM:
Thoemmes, F., & Mohan, K. (2015). Graphical representation of missing data problems. Structural Equation Modeling: A Multidisciplinary Journal, 22(4), 631-642.
In these cases imputation and FIML always work better than listwise deletion; and pairwise only works under MCAR (missing completely at random; that is the missing process is uncorrelated with the model variables--even Y). The common sense in the literature is NEVER to use listwise deletion because it combines bias with lack of efficiency & power.
In addition, the goal of imputation is not "inventing data that does not exist" but to recover higher order statistics (such as means an variances/covariances) that are biased by the missingness. In addition, you can include "auxiliary variables" which are correlated with Y and the missingness status to use this information in the imputation process (see the McKnight et al. book below to get some practical tips).
In their review of missing data methods, Abraham et al. report papers that even imputed large amounts of missing data (up to 62%). Enders and Bandalos (2001) show that listwise is always bad and pairwise is bad unless the missingness is MCAR. In MAR conditions, pairwise is bad.
Abraham, W. T., & Russell, D. W. (2004). Missing data: A review of current methods and applications in epidemiological research. Current Opinion in Psychiatry, 17, 315-321.
Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8(3), 430-457.
The essential point is: It is not the size, but the causal process leading to MI. This is very similar with other self-selection or attrition processes leading to persons move in or out of the study:
Elwert, F., & Winship, C. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40, 31-53. doi:10.1146/annurev-soc-071913-043455
Haneuse, S. (2016). Distinguishing selection bias and confounding bias in comparative effectiveness research. Medical Care, 54(4), e23.
That is: if you goal is to estimate a regression or causal model, and your dependent variable suffers from missingness, then traditional deletion will lead to bias of the effect but with imputation/FIML to unbiasedness and higher efficiency when the process is not MCAR (which is very likely).
Beyond the excellent recommendation by Rosario, here is a further list of references one should read in order to get an idea:
As the most practical introduction, I would recommend the book by McKnight et al., since it overs not only a good introduction into imputation and FIML but also practical tips how to analyse the amount, pattern and correlates of missing data.
The book by van Buuren is about his MICE package in R. Software packages such as lavaan, Stata for estimating SEM have FIML implemented.
HTH
Holger
- - - - - - - - - - - - - - - - - - - - - - -
Allison D, P. (1987). Estimation of linear models with incomplete data. Sociological Methodolgy, 17, 71-103.
Collins, L. M., Schafer, J. L., & Kam, C.-M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330-351.
McKnight, P. E., McKnight, K. M., Sidani, S., & Figueredo, A. J. (2007). Missing data - a gentle introduction. New York: Guildford press.
Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549-576.
Graham, J. W., Olchowski, A. E., & Gilreath D, T. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8(2), 206-213.
Newman, D. A. (2014). Missing data: Five practical guidelines. Organizational Research Methods, 17(4), 372-411. doi:10.1177/1094428114548590
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147-177.
Schlomer, G., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57(1), 1-10.
Van Buuren, S. (2012). Flexible imputation of missing data. Boca Raton: CRC Press Inc.
Thank you very much for your helpful suggestions Holger Steinmetz. I will read these references you provide carefully. My critical issue is that all my survey variables have some (but only a few) missing values. It seems that many multiple or single imputation methods require variables with complete data to be the predictors for those containing missing values. I am not sure how to conduct a imputation in this case. Should I adopt some iterative methods that use the variables with least missingness to predict others and then repeat this process until the whole dataset is imputed completely?
Also thanks for the comments by Rosario Scandurra and Paul Louangrath , which offer me with the opportunity to further clarify my missing data pattern. Although about 2,000 of 8,000 residents missed some of 59 variables surveyed, the degree of missingness is very small: Most of them just missed one (1,000 respondents) or two variables (500), while the variables they missed are quite different for different respondents. So I think deleting these incomplete observations would not only endanger the statistical power and efficiency of my survey, but also be a huge waste of information.
Many thanks to other suggestions.
ML
Hello all: First, I'm relieved to find that you have such a small amount of missing data, Maoliang. From the way that you wrote your question, I think that both Paul and Rosario thought that you had a massive amount of missing data - in which case I think their advice was well worth heeding. That's why I recommended their answers - also because I thought you had a massive amount of missing data.
I also recommended Holger's answer because of its sophistication and comprehensiveness.
As things have turned out, I think you could either pursue Holger's advice or conduct pairwise missing, rather than listwise missing, analyses - but I'd be happy to hear Holger's view about that.
Hi all,
Robert Trevethan I tend to disagree.
1) Even her first description did not indicate a too large amount of missingness.
2) Further, she described a arbitrarily missing pattern which indicates MAR.
3) As far as I understand, missingness is only critical for dependent variables ( Maoliang Ling hows the amount in your DV(s)?)
4) As simulation studies show, pairwise is only recommended under MCAR (which is less likely than MAR).
Go for FIML or multiple imputation. Thinking abit and analysing the missingness (as presented in McKnight et al.) could improve the issue, though).
This is so standard, that I have to rub my eyes when reading this thread ;)
Best,
Holger
Dear all,
Sorry for the ambiguous description of my missing data issue in the original post.
Holger Steinmetz There are six variables measured for my DV, each of which contains a really small amount of missing values, ranging from 34 to 47.
For each variable with missing data in the survey, I'd like to use other variables to predict its missing values by MI approach. However, two questions bother me:
1) Since all variables have some missing data, honestly I have no idea about how to conduct a MI approach. MI seems to require some auxiliary variables with complete data as predictors for missng values of the target variable (e.g., in Stata, those with incomplete data are not allowed to be the auxiliary variables)
2) After imputing the missing data by MI (hence there are multiple imputed datasets), I need to generate some composite constructs based on the variables belonging to the same construct (e.g., take their means), and then run my analysis model (multilevel ologit model). Since this model is not supported by Stata's - mi - command at the moment, I am trying to figure out how single estimates from individual imputed dataset are combined and how final estimates are calculated.
Best,
ML
Dear Maoliang Ling
the amount of missingness is so ridicously low that it is a real waste of time to further discuss this, but here we go :)
I don't know where you got the idea that you need complete data in the predictor variables. I might err, but I think that this is not necessary. But I admit that my knowledge is limited.
To test this, i attached a simulation that you can easily re-run downloading R (even if you have never worked with it. This simulation also contains some simple SEM/lavaan code that you could use for your own work.
The simulation is drastic as I created a data set of N=8000 and deleted 60% of the cases. The underlying model was a simple mediation model with X-->M-->Y. Furthermore, I deleted 60% in all three variables (hence, even X has huge missings.
The results showed that the SEM models with listwise deletion downwardly biased the effects but the bias is (admitted) the bias was not life threatening. I would, however, think that things work differently in real data with systematic missingness in the DV. the FIML estimator implemented in lavaan, in contrast, EXACTLY recovered the parameters from the complete data set (N=8000).
With regard to the second question, in case of using multiple imputation, you combine the specific estimates by the Rubin-formular which should be implemented in Stata. But again, FIML is much easier. The advantage of MI however is that you can incorporate auxiliary variables that support the MI process. But given your case, don't waste that time.
Best,
Holger
Hello Maoliang,
First up, thank you for being so sweet as to acknowledge that your original post / question might have been, or was, ambiguous. No problem. You're forgiven!
Holger has kindly attempted to help you, but I wonder whether I might be able to offer a simpler solution - at least for some of your data. First, however, am I right in thinking that a lot of the data are items on scales and that those items are used to form a composite total, or mean, on those scales?
Also, could you tell me whether or not you're using, or have access to, SPSS please?
Maybe let us know what statistical analyses you anticipate using with your data, too, please, e.g., multiple regression.
Robt.
Dear Holger Steinmetz, thank you very much, especially for this simulation test.
Dear Robert Trevethan , thank you and, yes, all independent variables of interest are ordinal variables measured by a 7-point scale. They are the indicators for different psychological constructs and those measuing the same construct would be merged in the next statisitic analyses (e.g., by taking their mean value). Some personal demographic features would be controlled for in my analysis model, too.
My analysis model is multilevel model using ordered logit as the linking function since 1) the DV is a ordinal variable and 2) my samples (residents) are (geographically) nested within different communities and I need to handle this nested data structure. As for my statistical software, I usually use Stata or SPSS.
In this study, I orginally planned to use hot-deck method to impute the missing values but changed my mind later as I realized the bigger capability of MI and FIML.
I would appreciate for your further suggestion.
Best,
ML
Maoliang Ling. Good to hear back from you. I suspected that what you have written might be the case.
In SPSS (I'm not sure about other stats packages), it's possible to compute the mean of a set of items if some are missing by specifying how many items you would allow to be missing before you would feel comfortable with the presence of missing data.
Here's an example that could be used on a 7-item subscale from a burnout inventory if you were happy to calculate a mean as long as there were no more than 2 items with missing data:
Mean.5(Burnout1,Burnout2,Burnout3,Burnout4,Burnout5,Burnout6,Burnout7)
I hope that makes sense. It's really pretty simple and works beautifully, and it avoids the situation in which a mean might be calculated on the basis of only a very small proportion of valid responses on a scale.
Please feel free to get back if I might be able to help further.
With regard to missing demographic data, I'm afraid I can't make any suggestions about imputation.
Feel free to let me / us know if you have success!
Robt.
- Badges
- Science topic
- Similar topics
- Biological Science
- Kinesiology
- Running