I have developed a survey with closed ended questions and I want to develop a scoring algorithm afterwards to assign individuals into categories based on their responses (to essentially validate the survey), would the best responses be in checklist format? a rating scale? or single closed ended choices?
I'm not sure one can separate participants into groups or categories to validate a new measure, unless participants are known to differ, a priori, on the variable of interest (i.e., known groups paradigm). For example, if I want to validate my new depression scale, I could recruit a sample of (1) individuals currently diagnosed with depression and (2) individuals who are not depressed. If my measure is valid, the depressed group should score significantly higher on the scale than the non-depressed group.
The problem you'll run into is circular reasoning:
A) I'm using my survey to separate participants into groups
B) I'm using the different groups to validate my survey
I suggest first validating the survey, and then using it to separate participants based on cutoff scores, quantiles, or whichever criteria makes sense given your variable.
Hello Jeanine,
I'm in agreement with Blaine Tomkins' remarks about grouping, unless there is at least one external variable that is salient to your research against which you can attempt to maximize correct classification with your measure.
Now, on to your question: What response format might work best for open-ended stimuli being converted to some sort of fixed response option/s? I think the choice depends on: (a) the nature of the stimuli as adapted; (b) the likelihood that a given format would be understood and responded to as you intended by the target population; and (c) the intended analytic method/s. Yes, that's vague, but without knowing more about what you're trying to elicit, that's about as specific as I can offer.
Good luck with your work.
Dear Jeanine Blumenau , the best type of response will partly depend upon the context to each question. However, in general terms, 7 point answer questions are good because respondents are able to distinguish between 7 responses so it provides the richest kind of reliable item data.
You then need to consider how to classify people into groups. You appear to be suggesting a dimension reduction approach then using the scores on a single dimension. I would not recommend this as you cannot assume all your items with contribute to the same single dimension. A better technique might be cluster analysis. This will take into account differences between the responses to all your questions.
- Badges
- Science topic
- Similar topics
- Statistics
- Survey