You should consider what was the effect of treating your missing data as blank. It is possible that the total score was not computed if any of the data were blank. So, I would run descriptive statistics and determine the actual N that your program used to determine the mean score.

Also, given that your data was scored 0-1, you can interpret your average score as a probability, i.e., the probability that the respondents agreed with the underlying content of your scale.

I am not sure. I think what you used looks like Yes, No, Not sure or not applicable. The typical Likert scale as we know in education or psychology comprises 5 to 8 itemI'm afraid I have to disagree re the use of Likert for a true/yes type of questions. I do not have from what you described the objectives of the questionnaire and why did you decide to use this close-ended type of questions. Reviewers will ask you. Third, what is the basis for your suggested calculations, you need to refer to studies that have used such a calculation method. Again, reviewers will ask you. Fourth, what did you do for participants who chose not applicable (neutral) in 50 or 60% of their responses? This is a worry. Six, I see you are a PhD student, I do not think for a PhD such methodology and ten questions using Yes/No is appropriate. My students in second-year medicine are doing far complex methodology than what I described.

In addition to David L Morgan 's point about missing values imputation, it depends what the questions are what you can conclude. Why did you ask ten questions? Presumably it is because you construct is multi-faceted. Aren't you interested if the items from a uni-dimensional scale or if your construct is more complex? What was the variability in agree responses for the items? How were items associated (e.g., present a table of log-odds-ratios). Also, you need to report the uncertainty of your estimate. Saying the mean was X does not help with the uncertainty in trying to make a statement about the population. So like Samy A. Azer says, it is best to step back and consider the issues more fully, perhaps consult a methodologist at your university.

I agree that a Yes-No format should not be called a Likert-scored item, but it could be appropriate for many types of questions.

Can you say more about the topic for your questions?

Hello Erini,

As long as all 10 items/questions/stimuli relate to the target model under investigation, then your basic plan to use a summated score is fine. It won't matter whether you use sum (0 to 10), proportion (as you are using), or per cent (100*score/10), they will all behave the same way (simple linear transforms of one another).

The only potential issue is the handling of "Not applicable" as blank; that will effectively lump such cases in with the disagreement (e.g., zero score) responses. Doing so says that your coding is actually: 1 = Agree; 0 = Disagree OR Not applicable. If that makes sense, then you are fine.

Can you do more? Sure; you could try treating the 10 questions via item response theory models (most likely a one- or two-parameter model), as this might give you more insight into the individual elements and how they sort out to yield a potential scale. But, if your research goal is simply to ascertain agreement or endorsement, your approach (with the caveat noted above) will work.

Good luck with your work.

David- I actually conducted a study to determine 2 cohorts' preference for 3 health care models, and compare their preferences. However, for the sake of simplicity I provided one example in my question above. The 3 models, one consists of 10 questions, one of 28 questions, and one of only 4 questions. Participants were to choose from "agree", "disagree", or "not applicable". Some questions relate to interpreter use and language deficiency, which do not apply to the English-speaking cohort. Therefore, a "not applicable" was included as an option.

I am not an expert in quantitative. In saying what you are suggesting would you say that it is better to treat it as Agree=3, Disagree=2, and not applicable=1. I know that these are just labels. I don't have any missing data, I was just advised by someone that not applicable should be treated as a missing value, however, I am not sure that this is accurate.

Hello again, Erini,

Given that your inferences about cohort preference for models depends on different question sets, that limits your ability to compare models. Cohorts for model X, yes; Model X to Model Y, not so much. If the latter is of concern, one option would be to fall back to common question subsets (if these exist). If not (and especially for the 28-item set), it might be worthwhile to try factoring the 3 items sets separately, to see whether assuming a unidimensional structure is warranted. (When creating a total score for each person, a single factor is implied.)

Treating "not applicable" as a scored value below that of "disagree" will muddle the meaning of the overall scores. I would suggest that a better approach would be to drop "not applicable" responses from a participant's response set, and to compute your preference score from the items that they did answer (agree or disagree). You would likely need a rule for determining what was a sufficient number of (agree or disagree) responses was for an individual case's answers to be used in your final tally.

Good luck with your work.

Thank you all for the helpful suggestions. Kind regards

Hi

I suggest using

Agree=3

Disagree=2

Not applicable= 1

If yes=2 and no=1

How calculate score for (poor /good) for teacher's performance

You code sum the scores and assess their reliability with the Kuder-Richardson formula, which is the binary form of coefficient alpha. But first make sure all your items are coded in the say direction.