Dear All,
I am using Mini IPIP questionnaire by Donnellan, et al. (2006) for measuring Big Five personality traits.
Although the questionnaire was reported to have acceptable reliability by the author and some other researchers, the reliability in my pilot study in Malaysia with 70 respondents is very low (Cronbach Alpha is 0.387). Factor loading with the use of SmartPLS are also lower than the threshold 0.7.
1-Do you have any idea why it is very low in my study?
2-Should I replace this questionnaire with another questionnaire?
I would first check that you have estimated the internal consistency of each of the 5 subscales (not a total). In some published sources these range from .65 (imagination) to .77 (Extraversion). If the .387 you report is for the highest of the subscales then the likely reasons are a) cultural differences; b) understanding (if used in English by a sample for whom English is not a first language); c) meaning differences if the items have been translated and this not checked; d) the constructs differ across settings. You could run a confirmatory factor analysis (your sample is just large enough using the 3 observations per item guide) to test the factor loadings / scale structure. The psychometric properties of tools are themselves important findings and so is likely to be worthy of reporting.
If the .387 is the best of the subscales then you will want to try to replace the questionnaire if you need a measure of personality. I would suggest using a measure that has been used with similar populations or one that has been used with various global samples and appears relatively culture free. You may also want to consider the language issue. It may be that further test development is necessary to establish a robust measure.
Hi Maryam,
1. The Cronbach Alpha is very sensible to number of the observad variables and number of the respondents.
HAIR, J.; HULT, G. T. M.; RINGLE, C.;SARSTEDT, M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM),Los Angeles: SAGE Publications, 2014. in p. 101 wrote:
"Internal Consistency Reliability
The first criterion to be evaluated is typically internal consistency
reliability. The traditional criterion for internal consistency is Cronbach's
alpha, which provides an estimate of the reliability based on the intercorrelations
of the observed indicator variables. Cronbach's alpha
assumes that all indicators are equally reliable (i.e., all the indicators
have equal outer loadings on the construct). But PLS-SEM prioritizes
the indicators according to their individual reliability. Moreover,
Cronbach's alpha is sensitive to the number of items in the scale and
generally tends to underestimate the internal consistency reliability. As
such, it may be used as a conservative measure of internal consistency
reliability. Due to Cronbach alpha's limitations in the population, it is
more appropriate to apply a different measure of internal consistency
reliability, which is referred to as composite reliability..."
2. correlations or loads values below.70 it is not a problem, because we need observad if the AVE is bigger ou igual to .50. If AVE > .50 the model has convergent vality. Loads values bigger than .70 are ideal conditions.
So, I suggest you keep on the study and continue using the scale. Of course, get more repondents.
Best regards
Dirceu
Agree with both Comments. I would like to suggest either to get more responses which lead better CB. I have worked on the same traits and faced the same problem even with 140 respondents. But I increased sample size and my variables were measurement with enough items so I dropped the problematic items.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics
- Survey Design
- Questionnaire