Because of a formatting error my scale with a 0-9 response is misinterpreted by some participants as having only a 1-8 scale, advice?
I effectively have 2 response sets:
1) participants who respond as the scale intends from 0-9
2) participants who respond incorrectly as if the scale is 1-8
Is there anyway I can keep the added information from the 1st response set without just collapsing theirs into the 2nd? Advice gratefully received :)
If the scale is approximately intervall, and if you know which participants responded incorrectly, the you might correct their responses by a simple rescaling / linear transformation:
corrected score = (wrong score - 1) * 9/7
This way a "wrong" score of 1 would give a corrected score of 0 and a "worng" score of 8 would give a corrected score of 9.
If you have doubts that the correct raters will more likely avoid extreme scores (0 and 9) like the wrong rates (the scores 1 and 8), you may flatten the transformation:
flatFactor = (10-8)/8/2 (or some other value)
corrected score = (wrong score - 1) * (9-2*flatFactor)/7 + flatFactor
As you can see is the "flatFactor" the distance of the most extreme ratings of the wrong wroater to the limits of the correct scale. With flatFactor = 1 the ratings of the wrong rates would not be changed,
A couple of hopefully quick clarifications
Is this is scale that has meaningful mid-point (like going dislikes X to likes X) or on without (like going from no opinion of X to likes X), and if it is the former is the midpoint in the middle of the extremes (and did the formatting error affect the mid-point interpretation?)?
And on the formatting error, did everybody get the same error or did some get it so you have some information about the shape of the distribution from those not exposed to the formatting error?
Thanks for a generously detailed answer.
What does the flatfactor do and does it matter which flatfactor I pick? I think if I go with it I'd just pick yours.
This assumes you can tell which read it incorrectly. If these have a meaningful midpoint your ability to tell seems unlikely to me. It seems more likely you can't tell.
If is a sampling issue, some place gave the incorrect instructions, and you can isolate them the easiest way would be to standardize the scores and go from there.
I would make a histogram for each of the two different versions. Does the form of the distribution look similar in both? Are the any reasons why it should NOT be similar? If not and they look similar, you can assume that they both refer to the same underlying distribution, and make different types of transformations (as described by previous answers) without invalidating your analysis.
Given that you have made the assumption of both referring to the same underlying distribution, you can just standardize both samples separately.
If you for some reason want to avoid standardization, and you want to confirm that you got the transformations right, you can check them by doing an t-test and F-test. The logic is a little different from the usual, as now you would be tweaking your transformations until you can get a confirmation of that these two samples are indeed from the same population. This is a dirty and quick solution, but close enough to increase your own trust in your choices. I would not use it as an argument in a paper.
I would again stress that any process assumes you are certain you can identify the two groups clearly. If you are not able to be certain of your identification then it would be a significant issue.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics
- Survey Design
- Questionnaire