To be more specific, this question does not concern central tendency bias/error -- where respondents are inclined towards centralized responses -- but the concept that mean values expressed on Likert-type response data tend to be centralized due to the issue of using truncated variables (e.g. 5 or 7 points on a finite scale with no continuum).
For example, if you administer a 5-point scale to two respondents, the possible number of combinations for them arriving at a combined mean score of 1 or 5 is one. However, the possible number of combinations for them arriving at a combined mean score of 3 is five.
Obviously, if you increase the respondents to three, four, five, etc. the possible number of combinations to reach a combined mean score of 3 grows exponentially; a plethora of combinations is possible with even a few dozen respondents. Yet, the possible number of combinations for arriving at a mean score of 1 or 5 remains stagnant at one.
How do you approach this dilemma when analyzing data? How can you associate a degree of 2 or 4 with more "oneness" and "fiveness" respectively to account for the central tendency of respondents?
Forced distribution seems feasible, but the practice of imposing a hypothetical normal distribution curve on data seems to me a sub-optimal and outdated practice.
Keyword searches into this problem have brought up concepts like entropy, or ordinal regression, but I am not sure if they address the issue (or perhaps they do, but their application simply goes over my head).
Many thanks for reading. This question is attempting to 'fix' the dilemma of differentiating centralized mean values (e.g. 2.3/5 and 2.8/5) to account for the aforementioned issue of centralization when assessing their differences (e.g. 2.8 - 2.3 = 0.5) so that "lower" or "higher" values (e.g. 2.3) can be interpreted as "closer" to the end of the scale (e.g. 1) than towards the middle of the scale (e.g. 2 or 3).
Hello James,
I would like to ask a follow-up question about this. Are you scaling people or scaling stimuli? There is an approach I use for each, but I want to make sure that I give you the most relevant information.
BB
I would do myself a question at first: does it really make sense to look at a Likert score as it is quantitative? In most cases, I guess it doesn't... and if it doesn't, then why should you compute a mean? I'd rather work on medians instead.
Hello Marco,
LOVE IT! One should totally be handling Likert data as ordinal. There are several great approaches to do that, depending on whether one is most interested in scaling stimuli or scaling people.
BB
Dear Ben,
too kind ;-) btw it also depends on James' purpose. Maybe it doesn't even require a complicate method to deal with it
Dear Marco,
I should clarify, my scale is Likert-esque in that it measures ordinal points on a five-point scale. However, it is not traditionally a Likert scale because it doesn't deal with degrees of agreement or satisfaction; my scale concerns the allocation of 27 PPP project risks on a public-private spectrum where 1 = solely public, 2 = mostly public, 3 = equally shared, 4 = mostly private, and 5 = solely private.
While the data is still ordinal, literature on PPP risk allocation via Likert-esque methods treats it as interval through methods like half-adjusting principle and forced distribution.
Thank you for making me clarify potentially relevant information.
Dear Ben,
I am assessing the total mean scores of the 27 project risks themselves. In the name of sensitivity, I have run analyses measuring the degree to which my respondent pool (comprised equally of public and private practitioners) agrees on the allocation of risks both within and between the two respondents groups (i.e. public and private).
This includes nonparametric tests like: Kendall's W, chi-squared, and Mann-Whitney U. I also ran an independent two-sample t-test, despite the issue you and Marco both brought up already about the data not being interval.
Results show that my respondent pool's agreement levels within sectors are statistically significant. Results show that agreement levels between sectors are high for 21/27 risks, meaning I should be more skeptical of the combined mean value of the 6 risks facing sectorial contention.
You could run your data through a paired comparisons model with the public vs. private as a predictor variable (possibly even an interaction variable, though I have never really tried that). There are also Rasch rating scale models that have similar functionality.
- Badges
- Science topic