There is no rule as to how many missing scores you are "allowed" to have. What is key is by which mechanism the responses are missing. If they are missing completely at random (MCAR) or missing at random (MAR), then those missing values can be properly addressed by multiple imputation (or potentially FIML), regardless of how many missings you have. (Of course, with more missings, you will have less statistical power.) If the scores are missing not at random (MCAR), then things are potentially a lot more difficult to address properly (i.e., without introducing bias into the statistical analysis).

I would recommend studying the mechanism by which data are missing in the first step. Try to find out why people did not respond to certain items. Are there any correlates of missing scores?

Also, it is worth reading some good applied resources on modern missing data handling such as

Enders, C. K. (2022). Applied missing data analysis (2nd ed.). Guilford Press.

Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. https://doi.org/10.1037/1082-989X.7.2.147

Thank you very much Christian Geiser, I will definitely check it out. By missing values I specifically mean "don't know" answers. Specifically, I have a scale that contains 15 items that creates three dimensions of five items each. However, one dimensions is "problematic" because the items are forward-looking and respondents often leaned towards the "don't know" option. At the same time, however, the responses to the items from the other two dimensions are relevant, so I just do not know how to handle this because I do not want to eliminate all cases of "listwise deletion".

I would say "don't know" answers aren't the same as truly missing scores. The "don't know" answers could be legitimate (if somebody feels they can't foresee the future etc.) and should probably be treated as such. You may have to treat people in the "don't know" category as a separate group in statistical analyses.

Depends how many DK responses you have, and how you wish to model them. Sometimes knowledge of an area could be associated with probability of DK responding. Some work exists on this (Article Locating "don't know", "no answer" and middle alternatives o...

Thank you very much Daniel Wright .

Christian Geiser Yes, I agree with you that this is a legitimate answer. However, if I just want to calculate an index of five items and three or four of them are "don't know", that is not possible.

Probably the most correct thing to do would be to discard all these cases (i.e. listwise deletion), but that would lose almost half the sample. So I am just wondering if a supermajority (i.e. 3) of the items will have valid values, then doimputing the remaining two.

The percentage of missing data has been controversial in the relevant literature. Some disregarded the issue of missing data as long as the total number of missing values did not exceed 5% of the overall observations. Others were of the opposite opinion, claiming that any amount of missing data can impair estimation quality. Accordingly, there has been no widely recognized rule that prescribes a ceiling for acceptable missingness—be it unintentional or systematic. However, researchers could use certain mechanisms (e.g., MCAR, MAR, MNAR), depending on the type of missing data, to minimize bias due to missingness. Here are some publications that could be of interest.

Hand, D. J. (2020). Dark data. Significance, 17(3), 42–44. https://doi.org/10.1111/1740-9713.01406

Kroonenberg, P. M. (2021). Multivariate humanities. Springer International Publishing. https://doi.org/10.1007/978-3-030-69150-9

Mirzaei, A., Carter, S. R., Patanwala, A. E., & Schneider, C. R. (2022). Missing data in surveys: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy, 18(2), 2308–2316. https://doi.org/10.1016/j.sapharm.2021.03.009

Good luck,

Mohialdeen Alotumi Thank you very much.