Hello Floren,

Have these measures been used before, and, if so, is there any technical information to support the purported dimensions? If not, I'd suggest you try exploratory factor analysis to determine what, for your samples, might be a plausible structure. Then look at estimates of score reliability.

While Cronbach's alpha has been around for over 85 years, it is not necessarily the best indicator of scale quality. If you are stuck with using it, however, and are thoroughly convinced of the dimensionality of the measures used, then, your options are:

1. Drop scales with very low score reliability (however, that leaves you perhaps without the ability to address research questions of interest);

2. Modify the scales with very low score reliability, and try them again;

3. Replace the scales with very low score reliability, choosing better measures.

4. Use the scales, but acknowledge the low reliability as a limitation. If you have large sample sizes, and are only making group comparisons, the statistical power from the large N may be sufficient to overcome the noise that high amounts of measurement error introduce into the observed scores.

5. Use SEM, invoking a measurement model for each scale; the resultant factor scores would at least then be freed of measurement error.

Good luck with your work.

Hello Floren,

Have these measures been used before, and, if so, is there any technical information to support the purported dimensions? If not, I'd suggest you try exploratory factor analysis to determine what, for your samples, might be a plausible structure. Then look at estimates of score reliability.

While Cronbach's alpha has been around for over 85 years, it is not necessarily the best indicator of scale quality. If you are stuck with using it, however, and are thoroughly convinced of the dimensionality of the measures used, then, your options are:

1. Drop scales with very low score reliability (however, that leaves you perhaps without the ability to address research questions of interest);

2. Modify the scales with very low score reliability, and try them again;

3. Replace the scales with very low score reliability, choosing better measures.

4. Use the scales, but acknowledge the low reliability as a limitation. If you have large sample sizes, and are only making group comparisons, the statistical power from the large N may be sufficient to overcome the noise that high amounts of measurement error introduce into the observed scores.

5. Use SEM, invoking a measurement model for each scale; the resultant factor scores would at least then be freed of measurement error.

Good luck with your work.

Hi Floren

It's not clear how big the sample was for the pilot test. This is important as in some simulations we found that estimates of alpha were more variable with small sample sizes ( Article Coefficient alpha: A useful indicator of reliability?

In short I would not advise doing anything too radical, such as dropping scales, until you are sure that your estimates of reliability are adequately powered.

Thanks

Mark

In addition to the above answers I would like to point out that Cronbach's alpha relies mainly on the inter-item correlations and that in some scales this is not a requirement to form a valid scale. For example if we want to measure social support and we have three items about the levels of social support they get from 1) Family 2) Friends 3) Colleagues, these items do not (necessarily) correlate, yet, content-wise it is valid to say that people with higher scores, have more social support (and thus it forms a scale). You can always still use the data as long as you have a good theoretic background as to why these items should form a single (sub)scale.

If you want statistical proof that the model holds for your data, you could also test a confirmatory factor analysis (or perform an EFA like David suggested) for the originally proposed multidimensional model from the developers of the instrument.

Best,

Michiel

Thank you for answering this question. I really appreciate it. The research instruments were already used before and they showed good reliability. However, since I am conducting a study here in the Philippines, I was advised to do pilot testing first before using the measures in my actual data gathering. I wasn't able to mention how many respondents I used in my pilot testing. There were only 10 teachers whom I administered the measures. It was too small and I planned to repeat the whole process again and gather as many respondents I can. Nevertheless, your answers and suggestions were such a good help. Thank you. :)

Your translated and adapted version of instruments should exhibit similar reliability with the original version. Low alpha values could indicate that the items or questions of instruments may not be properly translated or not relevant in the target language, culture and setting. You should consider to repeat translation of problematic items to decide if to remove or to replace with items relevant in your context.

Thanks Mr. Wong, I did review the translations and made necessary changes to those items which are confusing.

Hello

After you defined the alfa you can analyze it.

Look my doctoral thesis

Ingrid