I have a panel data set: 39 countries and 12 years (n = 468).
My key variable of interest (Variable A) has missing data, along with other variables, and 387 cases are available out of 468. I have conducted missing variable analysis by using SPSS. Separate variance t tests show that T tests are significant between some variables and Little’s MCAR test is significant at a 0.001 level which rejects missing data are NOT missing completely at random (MCAR).
Thus, I have used multiple imputation (MI) technique (five imputations) and replaced missing values of all variables. The key variable was not statistically significant when I analyzed with SPSS linear mixed level modeling technique that uses maximum likelihood (ML). However, the same variable became significant with the data set with the multiple imputations (MI), n = 468.
My question is what are the ways to convince myself if the mixed level modeling estimation with MI is better than the one with missing data? Thank you in advance.
Don
Interesting question. I would test the models with a validation dataset (if not available, split your data in a "training set" and a "test set"; see cross-validation) regarding accuracy and/or precision.
Hello Don,
Apart from multiple imputation, you can also use FIML (Full Information Maximum Likelihood). It doesn’t actually ‘replace’ the missing data, but rather is a direct estimation technique and operates by directly analysing the incomplete data set to yield unbiased parameter estimates and accurate SEs.. I believe SPSS uses FIML when you select Analyze>Mixed Models>Linear and select MIXED procedure. You can also use ‘auxilliary variables’ that help inform FIML (See Collins et al. ref below).
Many adhere to MI or FIML. One of the nice features about FIML is that the model fit indices are useable (e.g. CFI, TLI, RMSEA, SRMR to name a few for SEM). If you are going beyond regression results and using these types of fit indices, FIML is useful. Unfortunately, these fit indices for MI are not useful, as there is no literature on how to use the average of these fit indices when using MI as a means of replacing missing data.
Another comment regarding MI that comes to mind is the number of replications do matter – See Graham et al. (2007) below for some interesting reading.
A useful ref:
Newman, D. A. (2014). Missing Data. Organizational Research Methods, 17(4), 372-411. doi:10.1177/1094428114548590
Graham, J., Olchowski, A., & Gilreath, T. (2007). How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation Theory. Prevention Science, 8(3), 206-213. doi:10.1007/s11121-007-0070-9
Collins, L. M., Schafer, J. L., & Kam, C. M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330.
Wishing you well in your research, Don.
Kind regards,
Tim
Dear Jochen, Heiko, and Tim,
Thank you so much for your helpful comments and suggested readings.
Kind regards,
Don
- Badges
- Science topic