I am estimating a multiple regression model (with one 2-way interaction) using Maximum Likelihood estimation (MLE). Due to some substantial missingness on some important covariates (30-60% missing; n=19000), I estimated the multiple regression model using two missing data treatments (Listwise Deletion, Multiple Imputation). These methods, however, produced different results - example, interaction was significant when using multiple imputation, but not listwise deletion.
Is there a method/test to evaluate which approach (listwise deletion or multiple imputation) is more trustworthy? In papers I've read/reviewed, people often seem to find concordance between their model coefficients when using listwise deletion and multiple imputation.
Also, for those interested, these models were estimated in Mplus, and I implemented a multiple imputation based on bayesian analysis to generate imputed datasets followed by maximum likelihood estimation.
Thanks much,
Dan
Hello Dan,
Yes; different tactics for addressing missing data frequently yield inconsistent results.
Usually the first order of business in trying to select a suitable method is to determine whether the data appear to be missing completely at random (mcar), missing at random (mar), or to have systematic relationships with presence/absence of data points.
Having said that, I suspect the multiple imputation approach you used is likely to be more warmly received than a listwise deletion approach that costs you 60% or more of the data set.
Good luck with your work.
David Morse Thank you so much for your response. Is there a test or procedure for determining whether missing data is MCAR, MAR, or MNAR?
Also, to follow-up on your second point, I'd like to show results for both listwise deletion and MI (multiple imputation) for transparency sake. If data is MAR or MCAR, could I say that the results using MI is more valid than results produced by listwise deletion? Thank you, David!
There is no such thing as more valid. You choose the analysis as best you can based on the information at hand. Suppose I tell you that by british accounting your bank account has $10,000 and $15,000 by American accounting.. you however don't get to choose which model you like best. If your bank is in London you're out of luck. now if under relevant conditions the british model could be valid as statisticians define it(usually it means gives a good prediction with the predictors you used). same with the american model. Both models could be equally valid and produce different predictions. You have to answer Dr Morse' questions as best you can use the appropriate system and live with the results. For medical examples see : https://www.amazon.com/Clinical-Prediction-Models-Development-Validation/dp/3030163989/ref=sr_1_fkmr0_1?keywords=steyerberg+emount+clinical+prediction+models&qid=1565839826&s=books&sr=1-1-fkmr0
Here somebody can die based on your choices. All this really matters.
Dan Lee you can determine if your data are MCAR with the Little's MCAR test. In SPSS go to Analyze - Missing value analysis, check the box for EM and click ok and move the relevant variables to their respective window. In the output, under the EM Means table you will see a footnote for Little's MCAR test and if the value is significant then that means your data are not MCAR.
Another method is to go to analyze- multiple imputation - analyze patterns and after you move the relevant variables in the analyze across variables window, type 0.01 in the Minimum percentage missing for variable to be displayed box. This will produce a graph (missing value patterns) where you can visually determine if your data are MAR or not.
To my knowledge, these methods do not give you a definitive answer about the nature of the missing data so you may also want to use knowledge from your domain to determine the reason for the missingness.
About your second point, I believe it is good practice to compare results from different methods (e.g., listwise deletion and multiple imputation) and i think this is called sensitivity analysis. You should also mention the assumptions made for when you performed these methods, so if data are not MCAR then listwise deletion will probably produce biased results
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python