Hi Karolina,

as far as I know (please someone correct me) Little's test evaluates whether missings are completely random (MCAR). Its failure means that missing are either missing at random (MAR) or NRM. Given your low amount of missingness, I would not assume a bias even if the missings are NRM. It is important to note that missingness is a causal and thus hidden process. The main issue in NRM is that the outcome variable drives missingness. Hence, consider whether this is likely.

Best,

Holger

I agree with Holger. If missing is not MCAR, it can still be MAR. However, if it is concluded that data is NMAR, I would not do multiple imputation no matter how many or few missing you have. Missing at random is an central assumption, justifying the application of multiple imputation.

If you are using mixed random effects models they are based on the rather forgiving MAR assumption not the more demanding MCAR. This means in practice that you should not get biased results if you can condition on variables that affect the missingness - so if young men do not turn up for re-interviewing, include age and gender and possibly their interaction in the model.

NMAR is when the values of the response variable affect the missingness - for example the alcoholics do not turn up for clinic in a study of drinking behavior. If this is the case you must model the missingness explicitly, that is jointly modeling the response and missingness, which can be tricky.

See, for practical advice

See https://www.lshtm.ac.uk/research/centres-projects-groups/missing-data

One further thing,

if you have problems to understand the differences between MCAR, MAR and NRM, this paper is an enlightenment (if you prefer graphical models).

Thoemmes, F., & Mohan, K. (2015). Graphical representation of missing data problems. Structural Equation Modeling: A Multidisciplinary Journal, 22(4), 631-642.

You will find Kelvyn's explanation of NRM as the key of the problem. The point is: you can never know. As always: it is all about assumptions and their defense.

Best,

Holger

I agree with the answers so far, but would like to add some points:

-the difference between MAR and NMAR is continous rathern than categorical, you cant test it, but you have to make educated guesses what is more plausible. This can be done by thinking about the mechanism which led to missingness (e.g. technical problems, MCAR or MAR more plausible) or if specific questions are avoided (NMAR more plausibel). Addditionally, you could check if missingness (not the variable which contains missings!!) is correlated/associated with other variables (which do not have to be part of the model). If yes, MAR is more plausible (simply speaking).

-I am not as pessimistic as Jesper concerning multiple imputation (MI). Simulations have shown that MI does not produce worse estimates in case of NMAR as other procedures (Schafer, J. L., & Graham, J. W. (2002). Missing data: our view of the state of the art. Psychological Methods, 7(2), 147.), but are still biased. Therefore, first of all you have to make assumptions about your missingness (see previous paragraph).

-Your amount of missingness does not raise any large concerns, as the others already mentioned, but if you want to be sure, make some form of sensitivity analysis (what is somehow part of MI algorithms, which show you the variance of the target parameters for all imputations, the reason why I prefer MI over single imupation, EM or FIML methods). Here you could simply impute extreme values of the specific scales (in either direction and in the bounds of the original scale) for the missingness and calculate your model for different imputations. This can give you an impression by how much your model could change in case that the missingness would be extreme answers (this does not take into account possible correlations of missingness with model variables, but may give a "feeling" for the possible influence of the missings).

Good luck.

Dear Dr. Holger Steinmetz, Jesper Fabricius and Kelvyn Jones, first and foremost I would like to thank you all for taking the time to answer my question. Your input is most valuable.

If I may, I would like to follow-up on your answers to ensure that I am understanding correctly, and take the adequate next steps.

My Missing Values Analysis revealed a significant Little's MCAR test,. As mentioned by Dr. Holger Steinmetz this indicates that my data is either missing at random (MAR) or Not Missing At Random (NMAR). However, I am having trouble deciphering how to determine if my data is either MAR or NMAR? How should I proceed with diagnosing the correct missingness pattern?

From my readings, MAR occurs: " (...) when (1) not MCAR, indicated by Little’s MCAR test being significant; and (2) missingness may be predicted by other observed variables and does not depend on any unobserved variables.? (Galson et al., 2015). As such, is there a specific way to conduct such an analysis in SPSS?

Please note: I am running my preliminary analyses in SPSS, but will be running my primary analyses (i.e., Actor-Partner Interdependent Mediation Model) in MPLUS. I am aware that MPLUS is quite robust and capable of handing abnormal distributions and such, hence I am wondering if in my case, I can ignore the significant Little's MCAR test and use EM to impute my 46 missing cases, before my main analyses?

Again, I truly appreciate your help; I am fairly new at these kinds of analyses and self-teaching my way through statistics for my doctoral dissertation.

Kindest regards,

Karolina

Karolina Sztajerowski I am a bit jealous now, that you seem not to consider my answer... ;-)

" From my readings, MAR occurs: " (...) when (1) not MCAR, indicated by Little’s MCAR test being significant; and (2) missingness may be predicted by other observed variables and does not depend on any unobserved variables.? (Galson et al., 2015). As such, is there a specific way to conduct such an analysis in SPSS? "

In SPSS you can do this under Analyze --> Missing Value analysis, but this will give you only t-test for continous cariables or cross tables for categorical. You could also do a logistic regression, to take more variables into account simultaniously, but you have to do this by hand. Relations may give evidence that at least some part of the missingness is due to MAR, rather than NMAR.

Dear Dr. Rainer Duesing , I truly apologize; I only saw your answer once I refreshed my page. Please be assured that the only reason why I didn't consider your answer in my initial response was because I didn't see it.

You raise some an excellent first point; I am reflecting on educated guesses related to missingness of data. However, my best guess so far is that some of the participants accidentally skipped a question. I looked at variables with high percentages of missing data (e.g., highest percentage of missing data was 2.7%, followed by 1.6% and 1.1%), and these items would not be items that participants would be unwilling to answer (in my opinion).

I am very curious about your suggestion: "you could check if missingness (not the variable which contains missings!!) is correlated/associated with other variables (which do not have to be part of the model)". I have 4 variables in my model: attachment anxiety (as measured by a 18-item subscale), attachment avoidance (as measured by a 18-item subscale), common dyadic coping (as measured by a 5-item subscale), and relationship satisfaction (as measured by a 4-item scale). I have missing values scattered across these items (please see attached picture). If I am to conduct correlations without variables with missing variables, then I would not be able to correlate items from my model?

Thank you most kindly!!

Karolina

This note from The Analysis Factor website might be helpful. It makes many of the same comments and suggestions that Rainer Duesing made in his first post.

• https://www.theanalysisfactor.com/missing-data-mechanism/

My comment about being jealous was not meant seriously ;-) dont mind.

" If I am to conduct correlations without variables with missing variables, then I would not be able to correlate items from my model? "

I think you got me wrong. You have to differentiate between variables that contain missings and the missingness itself. You would for example create a new variable (for each variable containing missings), dummycoded with 0 if the person delievered a value or 1 if the person created a missing. You can now analyze/correlate this "missingness" variable with other variables (a t-test is just another representation of a point biserial correlation) to check if missingness is associated with other variables (which do not have to be part of the model). You do not have to do this by hand, but SPSS will do this for you in the above mentioned dialogue, but only as t-tests (for continous external variables) or cross tables (for categorical variables) and not more complex. i.e. logistic regression.

And I would not impute with EM algorithms, but MI, since EM gives you only one value (you cant check the variability of imputations) and underestimates the standard errors, therefore not suitable for inferential statistics. If you use SEM, use Full Information Maximum Likelihood (FIML) algorithms, which show similar/good unbiased results as MI in case of MCAR or MAR-

Hi

Also concerning inference after imputation

I agree with Rainer: always choose a method with multiple imputation rather than one single value

It is well known that the type I error increases using multiple imputation, i.e. you should use a test that was designed to be run with missing values

If you use imputation and then run a method as if there were no missing values your type I error will increase, take that into account, aka do not a use predetermined threshold like 0.05

I think the link given by @Bruce Weaver clearly explained the difference,(MAR , MCAR) and the options you can take. Please take a look at the explanations.

In addition, If neither MCAR nor MAR holds, then we speak of missing not at random, MNAR, i.e., on significant Little's MCAR test [ it's a null hypothesis test not diagnosis, given your attached graphic/map, of 8 females and 39 male missed data, perhabs ruling out sex as covariate, sincerely you can't ignore MNAR assumption [to assume MAR hence MI ] but should complement with diagnosis for the missing mechanism, "some unseen, underlying value." An Smuk, M., Carpenter, J.R. & Morris, T.P.(2017)], possibly a callback on the missing respondent to filling in the missing opinion scale choice, first obtaining reason for non response and reframing the questionnaire question to acceptable 'value inclusive for a response'. it's not just only 46 out of 187 and neither can you delete but as suggested by Prof @Kelvyn, a possibility is Modeling the missingness explicitly, and using dummy, [0 for missing, 1 for not missing, selection model for reduced missingness] , which SPSS can handle (or by 'pattern-mixture' model given the prior distribution can be expertly obtained.)

Also, you can't go on making imputation [MI or MI+] without considering the true most likely underlying mechanism for the missing data, except you are sure it is MAR, i.e., meeting it's assumption!

#As long as the distribution and percentage of missing data is not so great [but how much? definitely not 47 out of 187 missing] that it negatively affects power, listwise deletion can be a good choice for only MCAR missing data. So the important distinction here is still whether the data are MCAR as opposed to MAR and diagnosing [not testing] for randomness of the missing data is advised here, and you need to use your substantive scientific knowledge of the data and your field, or educated guess (statistical correlation isn't feasible here).

Again, a callback to those respondents may reveal if the missing values are subjective - to value variable, [ie , MAR ] and perhabs an update to substantive number of missing responses could be possible, (if really not due to MNAR, and since you said your best quess was that "some of the participants accidentally skipped a question"). Really, the more sensitive an issue is(and how it was asked), the less likely people of a particular sex or education or class are to tell you (just skip) being subjected to such response by sensitivity or value relative to the survey question or any other variable not included in the research. This is then obviously MAR (a systematic relationship is established for the missingness case) and not MCAR nor MNAR.

See https://www.google.com/url?sa=t&source=web&rct=j&url=https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-017-0301-0%23:~:text%3DThe%2520MNAR%2520assumption%2520states%2520that,on%2520some%2520unseen%252C%2520underlying%2520value.&ved=2ahUKEwizxv7s6rTtAhUCxhoKHQrQAZwQFjABegQIARAE&usg=AOvVaw0k-uV67ekYeUdlTB1-TgpB

Alt. ref: Smuk, M., Carpenter, J.R. & Morris, T.P. What impact do assumptions about missing data have on conclusions? A practical sensitivity analysis for a cancer survival registry. BMC Med Res Methodol 17, 21 (2017). https://doi.org/10.1186/s12874-017-0301-0