Alpha does not indicate internal consistency, but internal consistency in relation to number of items per test scale. The average inter item correlation per scale is a direct measure of internal conistency (cohesion). A alpha of .59 over 5 items means a cohesion of .22; an alpha of .59 over 20 items means a cohesion of .07, which is almost negligible. 

Alpha is meant as a substitute reliability measure. An alpha of .80 is not too impressive. On a scale of 5 items it implies a cohesion of .44, which is quite high but, then, there are only 5 items. On a scale of 21 items it implies a cohesion of .16, which is rather low.

See Douglas G. Bonett, on Research gate, for confidence intervals around alpha.

Why not use item analysis on the basis of item-cluster correlations to see which items are doing bad in the clusters?

If you have a subset of variables that you are using in a regression, and are believe this subset was caused by the same latent variable (in the sense people talk about with latent variables), then you might look into SEM. If you are trying to see if the subset really are different, then again probably CFA would be used by more people, as opposed to worrying about specific alpha levels (which vary with so many things, like the number of items).

Cronbach's alpha assumes that all items have the same factor loadings. If this assumption is violated, alpha value will be reduced. I guess that the two items have different factor loading values.

Thank you, I was thinking of CFA myself. I am saving factor scores for each of the items, so I guess the loadings are different?

I agree that CFA is the way to go, since you are proposing that a 3-Factor model will fit the data better than a 2-Factor model. CFA will also take care of the Cronbach's Alpha assumption of equal loadings. Further, you can see if there "correlated errors" between the items assigned to different factors.

In terms of the way that the new factor is working in the regression, that would take you into Structural Equation Modeling, which would not be a big step after creating a measurement model through the CFA.

Note that the low Alpha basically translates into a relatively high rate of random error, which is not capable of correlating with anything. If your new scale has low reliability as a measure but still has significant effects in the regression, this could indicate that a better measure would have even stronger effects. In essence, the low reliability of the scale causes you to underestimate the effects of the concept you were trying to measure.

Hi

Yes, .59 is a low internal consistency. Try to find  if in your new factor there are items that are unrelated with the others at all. If you subtract them, your Alpha can increase. 

Low internal consistencies implicate that correlations between some items of your suggested factors are low and might therefore represent different factors or concepts. The best way to address this issue is to conduct multiple CFAs including theory-driven models. But consider, specifications for certain relations such as between error terms have to be made precise und reliable parameter estimations require a sufficiently large sample size in SEM. A good rule of thumb is to have N> 200 or to have about 10 times more sample cases than paramters to be estimated.

Franz:

I would argue that a low internal consistency doesn't suggest correlations between items of the scale in any way. While that is possible and multiple factors are there, I see no reason to believe that a low cronbach alpha alone is suggestive of multiple factors- especially since coefficient alpha is basically looking at the relation between items itself.  It could be equally possible that items are merely unrelated entirely. Such as would be the case with a scale made of 4 items like "I like shoes", "I enjoy spring more than winter" etc where there is no thematic relationship. You would expect both a low internal consistency and a low rate of correlation between items.

Besides that, while it is possible to start with several CFA models including theory driven ones- why not start with an EFA? A split sample EFA/CFA to confirm the factor relationship would be a final way, but again- you should probably have more than low internal consistency to start as a basis on such a fishing trip, and thus I think that is an aside since my first point still stands.

Thong:

if there is a theoretical rationale for your combining of models, why not perform a CFA with the two models (default and yours) and compare fit statistics. If you demonstrate significantly improved fit then you have some argument. Otherwise you have a bad coefficient alpha and a worse CFA and I would say your hypothesis is having a harder time with those items.

Try removing the items with very high correlation or very low correlation with other variables and try again. Alpha measure is a superior tool for measuring internal consistency of items.

Short of doing CFA, the simplest tool for examining a hypothesized factor structure is to reorder your correlation matrix to correspond the to the factors. That is, start with items that go with factor one, then the items that go with factor two, and finally the items for factor three.

If you are right about the factor structure, you will see three sets of high correlations along the diagonal of the matrix (i.e., the within factor correlations), with low correlations everywhere else (i.e., the between factor correlations).

This basic visualization will also show whether: 1) there are some items in a proposed factor that have lower correlations than the other items in that set; 2) whether there are some items that are highly correlated with things you predicted to belong to other factors.

In general, Cronbach's alpha is a very large blunt instrument that will not tell you much about the measurement properties of your scales.  If you want to know more about the psychometrics of your items, take a hard look at how you are analyzing it (i.e., would IRT or an SEM variant at the measurement level tell you more than factor analysis?).  But if you want to know about how the scales work in a predictive model, the fact that you have results in that analysis is important information.

The folks recommending CFA are probably giving you the best advice. An alternative would be simply to run a standard exploratory factor analysis (principal components analysis) and see how many factors emerge. If you are correct, there will be three with eigenvalues above one. Of course, for any factor analysis you ought to use a large sample - at least 10 participants per item.

An alpha of .59 is relatively low. Even with somewhat higher values, you can have more than one underlying factor. The big problem is that the two original scales had higher levels of internal consistency. That suggests that they may be "better" scales in the sense of being more cohesive. Why is your scale "working" in regression, then? Maybe because it has properties of both "real" scales? Not sure.

Alpha does not indicate internal consistency, but internal consistency in relation to number of items per test scale. The average inter item correlation per scale is a direct measure of internal conistency (cohesion). A alpha of .59 over 5 items means a cohesion of .22; an alpha of .59 over 20 items means a cohesion of .07, which is almost negligible. 

Alpha is meant as a substitute reliability measure. An alpha of .80 is not too impressive. On a scale of 5 items it implies a cohesion of .44, which is quite high but, then, there are only 5 items. On a scale of 21 items it implies a cohesion of .16, which is rather low.

See Douglas G. Bonett, on Research gate, for confidence intervals around alpha.

Why not use item analysis on the basis of item-cluster correlations to see which items are doing bad in the clusters?

Hi everyone, thank you. A PCA still shows two factors  based on a scree plot and eigen values above 1, so it looks like the original factor structure is being upheld. Taking the two sub scales separately though (5 items each) gives an alpha of 0.72 for one and 0.65 for the other, so not much of an improvement. Deleting items does not increase alpha for the 10 item composite scale.

Hi there, you might want to check the literature on formative vs. reflective measurements as well. Reflective measurements should have a high internal consistency while formative do not. I don't know you scale, but if it can be argued that the concept you are investigating is the sum of the items (or subscales) rather than should be reflected equally across the items (or subscales), then you have a good argument for a formative measure and hence less need for a high Cronbach's alpha.

As Peter noted, the number of items in a dimension have an effect in the reliability. For two variables, it's somewhat demanding to get high reliability if the variables are not highly correlating with each other. For two variables, .59 is actually quite nice value. Not high enough to make strict inferences, though.

Most probably CFA (SEM or ESEM) would give you support to split the dimension into two. Worth of checking. 

Hi Professor,

Your tip on formative vs reflective measurements is really helpful. I think my two sub scales reflect different facets of the one latent factor (in a sense, a CFA / SEM), so the factor is not equally represented across all items. It feels almost like the people who would score highly on this factor would be multivariate outliers in that they have an unusual combination of scores on the original scales. Having said that I am starting to change the items in my scale because I think the way I was doing it initially was conceptually inaccurate. 

Remember that PCA assumes the factors are uncorrelated, whereas running two separate Alphas makes no assumption about whether the scales are related. In particular, I suspect that if you just add up the variables in each scale (which is the assumption with Alpha), that you will find the two resulting variables to be correlated.

This is not necessarily a bad thing. Consider the names that the previous authors assigned to the two scales -- are these theoretical/conceptual "variables" things that you would reasonably expect to be uncorrelated in the real world? Or are the things that scales represent likely to be related to each other?

Again, Confirmatory Factor Analysis can handle the possibility of correlated scales directly. Since CFA seems to be the most comprehensive way to address the various issues that you want to consider, I think you should take the trouble to learn CFA rather than trying more and more versions of EFA. 

Exploratory structural equation modeling combines the good things of CFA and EFA. Have a look at that:

Asparouhov, T., & Muthèn, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438. (Proposal and explanation of the method)

For  examples

: Guay, F., Morin, A.J.S., Litalien, D., Valois, P., & Vallerand, R.J. (2014). Application of exploratory structural equation modeling to evaluate the Academic Motivation Scale. The Journal of experimental education, 0(0), 1-32. DOI: 10.1080/00220973.2013.876231.

Marsh, H., Ludtke, O., Muthèn, B., Asparouhov, T., Morin, A.J.S., Trautwein, U., & Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22, 3, 471-491.

You might check out

Lucke, J. F. (2005),  'The alpha and the omega of congeneric test theory: An extension of reliability and internal consistency to heterogeneous tests.',  Applied Psychological Measurement, 29(1), 65--81.

You might want to do different questions... for example: how did you decide that some of the items of each scale can compose a different score?  Did you run an EFA with all of the items to check how the correlate with each other? That might give you some precious cues on the construct you "feel" that is there. An alpha such as .59 is not heaven, but it is not the true depths of hell either. Still, you might be pulling items that content-wise might be similar according to your impressions, but this might not be true according to the empirical data you acquired. That type of thing happens and that's why we need to investigate structure.

Don't be seduced by a significant regression coefficients as well. That alone means nothing. Having a significant predictor doesn't mean that you have a reliable measure. Results can be inflated due to the violation of assumptions if you think about that. 

I attached some papers on McDonald's Omega as it provides a internal consistency measure that handles better heterogeneity when compared to Alpha, but I don't think that alone can do the magic. Take a look at how your data is going to behave.

http://personality-project.org/revelle/publications/revelle.zinbarg.08.pdf

http://www.personality-project.org/r/psych/help/omega.html

Thank you, everyone's articles have been very useful. I think I am dealing with a heterogenous scale, and also I can't really change the items that go in because to some extent, my study is testing the structure of the published scale and its sub scales. So I'm trying to see whether my latent construct is formed from two different aspects.

Dear: Tong,

Maybe you interest with this paper (attached)

*Hengky