Hi Theresa,

I've probably missed this, but just for clarification, are all three groups responding to the same survey/questionnaire, which contain the same items/variables?  If this is the case how many items/variables does the survey/questionnaire contain?  I assume the NA is the middle (3) of your 5-point scale? Personally, I would include the NA.s if indeed they are a '3'. Otherwise you might have well have used a 4-point scale.

If it is the case that all three groups responded to the same survey, and you are not including NA's, why not combine the data from the groups.  You can then split the file into two equal parts (after counterbalancing for any potential bias in one half) and then do an exploratory and confirmatory factor analysis with each half.  That's assuming this is what you had planned to do.  You could also do this by leaving in the 'missing data' in the form of NA's.

However, by doing this you have to be mindful that you're not getting a true reflection of the resonses to your variables/items.  As such, you may end up with a model that doesn;t fit the 'real' data.  In addition, you need to bear in mind how many variables you have and the size of your sample.

I'm not sure if this has helped, but please feel free to message me if you would like further clarification,

David :-)

Hi David,

You are correct in that my question/explanation was not entirely clear. Yes, all 3 groups are answering the same exact 12 variables included in the scale. The original survey data was based upon a 6 pt Likert scale (6 being the NA) with a 'neutral' option in the center. So in deeming the NA's as 'missing' I'm left with a 5 pt Likert scale. Perhaps in keeping the NA's included, I should consider recoding the scale so that the  6 (NA) would become a 1 (currently 'strongly disagree') and the current 5 ('strongly agree) would become the 6. Does this information change the advice you would give?

I am not clear why NA is not a valid case. In general the question was responded to here.  It was responded to as directed as I understood the description.  So it is a valid case in terms of response rate and responding. 

If you want to calculate the score, or use only those without the NA response, you could simply use the stats software you have to recode the variables into a new set of variables with 6 as missing data.  Alternately use the software to develop a "include" variable. Here responses 1-5 = Include (1)  and response 6 is Exclude (0).

It seems each question has two resulting pieces of information. First, is it applicable and this can be coded 0 or 1, and then second a 5 point rating scale where some data are missing (and may not make sense).  There are statistical models for this depending on how the NAs arise, but it is essentially a missing value/censored data question unless I am missing something.

I agree with Adel that you could do some kind of "imputation" process to try and replace the Not Applicable responses, and you should probably do that with all the cases in the data, not just Group #1.

First, however, you should examine the Group #1 data in some detail. For example, are most of the respondents missing data on just 1 or 2 items? If so, then imputation will be more reasonable. If however the typical case with missing data in that group has multiple NA responses, that would make things more problematic. (And if you are lucky, you will find that one or two specific items account for nearly all of the NAs, in which case you might consider eliminating those items from the scale as a whole.)

But if you do consider imputation, note that your data do not meet the assumption of "missing at random," since there is a systematic difference in where they occur.

Finally, with regard to Daniel's point, you could assign each person a 1/0 dummy variable according to whether or not they have complete data. After you do that, my personal recommendation would be to run that variable in a t-Test against your dependent variables. That way, you will at lest know if the tendency to have missing data is also related to what you are trying to explain.

I would not think imputation would be a good model to apply as these are in fact responses (NA) so are not "missing" data at all.  These are purposefully responded to they are not random at all either.  As Daniel noted these data are typically considered censored type data.

I found a previous ResearchGate discussion on NA that includes a couple of useful links:

https://www.researchgate.net/post/Should_I_treat_the_option_Not_Applicable_N_A_as_a_missing_value

Hi Theresa,

I tend to agree with Daniel. The NA is a response as your participants deliberately chose this option rather than any of the 5 points available.  In my opinion these responses are just as valid as the others, especially as you are developing a survey and testing out the variables/items for this.

David

I'd suggest you use qualitative techniques to further explore the 'not applicable' segment of your data. This should help you to understand why they are not applicable, and to elicit the best way to have them measured. Sometimes, it could be that you have asked the wrong questions or the right questions the wrong way, not necessarily that the variables are 'not applicable'.

You have already received a number of good suggestions. I agree that those who deliberately responded "NA" should not be treated as "missing," but other than doing multiple imputation (which is cumbersome and might not give you what you need), there is another option you might consider: you could recode your "6s" into "3s"--the neutral mid-point of your 5-point Likert response scale. This would be better than recoding them as "1s" [strongly disagree]; because the item is, in fact, not applicable to the respondent, it is true that the respondent can't really agree or disagree with the content of the item.

Whatever you end up doing, you should also report the percentage of "NAs" for each of your 12 items, and you might also want to see (and report) if the percentage of NAs per item differ among your 3-groups (which I suspect will be the case). Any item that has a high percentage of NAs (let's say > 20%) for the whole sample probably should be eliminated from your scale in the future.

Hope this helps.  Ken

This is a difficult question for answer without knowing the content of your survey.  It is possible that the NA responder may constitute a different comparison group.  Do you see a pattern in those that chose the NA option?  Even though we start off with research questions and/or hypotheses, we find unexpected and unanticipated trends or phenomena.  We find answers to questions we did not ask!

I agree, too, that too many NAs may indicate an issue with the survey or your population.