Hi all,
I'm using one scale with only three questions. It's creative uses questionnaire so the more adequate uses one count for each question the higher the score is for this question.
The answers are then ranked for their originality (suitable answers that were given by less than 2% of the participants) fluency (how many solutions were given for each question) and flexibility (how many categories were created from the answers that were given by the participant for each questions).
I run reliability test for this three scores (originality, fluency and flexibility) between the three question items. Fluency and flexibility had good Cronbach's alpha ~0.8. Originality was pretty low (~0.5). I then used factor analysis that extracted only one component of the three. The loading of the first two questions were high (~0.85) but the third (Uniqthver as seen in the attached component matrix) didn't.
Basically the originality score was the lowest (natural if you consider the strict criteria of defining answer as original) so to better understand I checked for frequencies and found out that while total of 30 original answers were given for the third question, the two first only have ~20, plus they were more equally distributed among subjects (well, just from looking on the data).
So how should I continue now with the third question? I don't think that I can throw it (or the two first) but apparently they don't measure in the same way originality (or they do and only because of one extreme observation at the third question it seems like they don't, should I throw that observation then? Doesn't sound like good science).
This creativity test is one of my IVs so the question is important.
The changes in the Chronbah's alpha in different studies could be attributed to reasons such as the new setting, different participants, the translation of the instrument, its validity in the new culture, the desinge used..etc.
The researcher has to report this, possibly as a limitation and
it will be useful to run EFA to examine the structure of the instrument in the new setting
Thx Muayyad,
Just another question. I used two newly translated tests in my study (I translated them with the help of two specialists, using proper translation and back translation process) the Cronbach's alphas are good for both tests aside for the sub score that I mentioned (originality) from the creativity test. My goal is not validating these two tests but to use them as part of the study. But as originality reliability is low, should I report on exploratory FA for both tests? Only one test? perhaps only the problematic sub test of originality? Naturally, as I want to keep it an article of ~10 pages with main focus given to the study and less for the validation of the tools, can I report on validation process that included EFA only for the originality factor?
Dear Gilad,
It seems that originality is quite different from the others two. I ask if you really need to test reliability - reliability means "repeatability" or "consistency". A measure is reliable if it give us the same result over and over again – do you need to test that? (the proportion of intercorrelation between the 3 variables?). If yes, try to perform alternative measures, like split-half or test-retest (or even parallel forms). Alternatively, inform in your research that alpha coefficient depends on the number of items in the scale.
Good luck with you research!
Gilad,
similar to Lisete's post, I am wondering if a factor model (and, hence, internal consistency via alpha) is really appropriate here. While I am no expert on creativity, it seems to me that those three indicators are not necessarily reflective of the same latent construct.
In other words (as Lisete pointed out), can we expect to find the same scores on all three indicators from individuals with a certain level of underlying creativity?
Aside from the factor structure, I am wondering if the scores on the indicators satisfy the assumption of normality. If not, this may be an issue in factor analysis. See Chapter 9 in Brown's excellent book: Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford Publications.
Oh no, they are not the same at all. This are three different constructs. Three factors actually (fluency of ideas, flexibility of ideas and originality of ideas) each construct is made of three items. Reliability was calculated for each construct alone (meaning, between three items). For originality the Cronbach's alpha was low (0.6) which is reasonable given that many times when one have original answer for one of the three questions he don't have for the rest two.
EFA can show that at least the three items have high loading, so they are valid. What do you think?
Thanks Lisete,
Just thinking now on reporting alpha of 0.6. It's really not a good one but I can explain the reason behind it easily, it's not a product of bad measure but probably the nature of it (don't have much previous studies with it, it's a new one).
ok, go ahead
Your reliability is low because the items don´t measure the same kind of creativity (explain this in your research)
Hi some cues:
- number of items (generally a low number of items in a subscale could decrease Alpha in that subscale);
- construct issues (the argument of Lisete)
- number of respondents
- direction of hierarchy scale
- pleas pay attention to assymetry and kurtosis of those items
All best
Paulo
Gilad,
sorry about the confusion.
Still, I think the fact that people may have high originality on one item but not on the others is a problem. Given that all three items should be affected by the same underlying factor, they should not be relatively independent like that.
Also, I am doubtful EFA is appropriate here. While this may be a new sample, you do have a "theory" about the factor structure.
- Badges
- Science topic