What about your independent variables? Maybe you could try some unsupervised statistical learning and see what clusters/structures you get and whether there is some association with your dependent variable (which could also be the frequency of NA) ...

Best Matthias

Hi Frank,

what do you mean with "remove them"?

Missing data especially in dependent variables are a critical issue and given that you have 93% missings, it is unlikely that they are missing at random which means that there will likely be a systematic variance restriction (or better a correlation between the dependent variable and the missingness process even after considering/conditioning on model variables (e.g., the independent variable).

I doubt that there is something what you can do. Matthias' suggestion is surely worth some try in order to learn about potential processes. You should definately create a missingness-variable which has a "0" when the case is observed and "1" when the case is missing. See the very nice book in which some missingness-analyses are reported.

McKnight, P. E., McKnight, K. M., Sidani, S., & Figueredo, A. J. (2007). Missing data - A gentle introduction. New York: Guildford press.

HTH

Holger

Thank You very much Holger. In my datasubset the wage variable is somehow correlated with 10 other variables so that when I try to remove all NAs from the data, it remains then only NAs values for those 10 variables.

I've also tried to replace the NAs with the mean of the dependent variables ( mean(wage, na.rm = TRUE)) but I am sure that my professor could not accept it, because the results will have no meaning...

Hi Frank

hence, you did some listwise deletion as well as mean imputation. Unfortunately (as you yourself notice) these are the worst methods :).

Better methods are multiple imputation (you impute varying guesses of wage based on the values of potential predictors and combine the estimates) or full information maximimum likelihood. Both approaches, however, rely on MAR (missing at random). And as I wrote in my first response, with 93% missingness, this is very unlikely. Furthermore, imputung 93% is a too strong stretch. There were simulations imputing around 60% which worked but in this case?

I hate to not be able to offer something. So please bear with me....

Best,

Holger

P.S. here are some prominent reviews about the different approaches

Abraham, W. T., & Russell, D. W. (2004). Missing data: a review of current methods and applications in epidemiological research. Current Opinion in Psychiatry, 17, 315-321.

Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549-576.

Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147-177.

Thank you very much Mr Holger Steinmetz. I would have another question. I ve chosen another wage variable and this is much more clean than the latter. Now I ve problem when performing the interval regression. It appears in the result an error : "NA in the initial value". What does that mean and How can I pass through this problem.

Thank

Hey Frank,

unfortunately, I have never seen this error and I can't imagine what "initial value" should mean. What is the software you are working with. I would suggest posting a question in a relevant forum (stackoverflow or R-nabble).

At any rate, see if your software offers some way of multiple imputation (e.g., the mice package in R).

Best,

Holger