If the reliability of a scale is between 0.50 and 0.60, what justification it could be supported with? If there are any references, please share.
Hi Mazhar, I don't think there is any way to justify the use of a measure with low reliability. In a model with two variables low reliability will attenuate the observed association (the correlation you calculate will be lower than the true population value), but in a multivariate model with more than one predictor low reliability can result in estimates of association that are too high, too low, or just about right - the problem is that you won't know which.
So there is no way to justify, but there are ways to correct for the deleterious effects of measurement error. For example you can use a correction to disattenuate (if that is a word!) the correlation or use latent variable modelling that allows an explicit measurement model to be included in a statistical model.
Mark
How thoroughly did you check the variables that make up your scale? Two common problems are missing data that did not get deleted and reversed items that generate negative correlations. You can check the first of these by looking at the frequency distribution for each variable and the second by examining the correlation matrix among the variables.
Beyond that, you can look at "alpha if item deleted" to see whether one (or more) of your items is particularly problematic.
Another issue is that alpha does not assess whether a scale is "unidimensional" (measures only one thing). For that you need Exploratory Factor Analysis, for which I would recommend using Principal Factors as an extraction method with oblique rotation (correlated factors).
The low reliability may be because of different reasons, but most prominent one we found in our research is presence of some items which could have been deleted in the Factor analysis.
I recall encountering the paper of a widely used scale with alphas around .5 or lower and asserted that lower alphas may be due to the conceptual breadth of the subscales.
"The modest alpha coefficients likely reflect the small number of indicators per scale and the fact that items were chosen to represent the conceptual breadth within each construct (see rationale for item selection in Measures and Procedure section) rather than to maximize internal consistency."
The Paper: The Structure of Psychological Well-Being Revisited. Journal of Personality and Social Psychology
I wanted to share that, yet I am not sure whether this position has sufficient grounds.
Inline with Morgan, I will suggest to check validity of your scale, as valid scale is mostly reliable.
If all the variables are properly coded so that they produce positive correlations, then the only reason for a low alpha is that the set of variables are not strongly related to each other (i.e., they have a low average correlation).
One reason for this might be a lack of unidemsionality, so that the items tap into multiple factors, rather that a single dimension, which can only a low value of alpha if the two (or more) dimensions are weakly correlated with each other. It is just as likely that a moderately low but acceptable level of alpha (e.g., .7 or .8) can mask the presence of multiple dimensions that are relatively highly correlated with each other. Either way, an Exploratory factor analysis with oblique rotation (correlated factors) is the best way to investigate the possibility of multi-dimensionality.
- Badges
- Science topic
- Similar topics
- Psychology