I am designing a perceptual study, with repeated measures design. Participants will hear some audio stimuli and be asked to respond to a target stimulus with the question "was the stimulus early, on-time, or late". There will be several experimental conditions, and all participants will undergo all conditions. I want a dependent variable measuring how accurate the participants are on this task, with the ability to see differences between early, on-time or late stimuli.
One option is to calculate the percentage of correct responses in each category, but i believe there is a problem of response bias with this approach. E.g., if someone responds "on-time" to every stimulus it will look as if they were great at recognising the on-time stimuli, but actually they were just biased towards that option.
In the past, I have used signal detection theory on 2-choice categorical data (i.e. yes/no data), to produce measures of sensitivity (d prime) and bias. This accounts for response bias in the data.
My question is, is there a way to extend this yes/no signal detection analysis to data with three response options?
Or is there another way to account for response bias?
Thanks!
Are you familiar with California Verbal Learning Test? It uses d' for the 2-choice categorical data (true/false) and also for the number of correct responses (correct recall words). If you know which response is hit or miss, you can dummy code the three response options, manipulate it in a way that you have a sum of correct responses, then you can manually compute d'. There are also several R packages (https://cran.r-project.org/web/packages/psyphy/index.html). You can check their source code and adapt it for your needs.
Another option would be using item response theory (search for IRT adjustment for guessing), which can penalize partial guessing behavior.
- Badges
- Science topic