I am using an instrument that has been found to be reliable and valid twice in published literature. However, the focused population is different (i.e. different countries, similar careers).
The original instrument EFA determined six subscales in both studies, however had a larger number of participant entries. However, I have a smaller sample size (n = 81). I ran a factor analysis which found 9 factors meeting the eigenvalue of 1 or greater. However, the factor matrix won't load.
I believe I should just run the entire instrument as a whole; the permission I received by the published indicated I needed to use the instrument exactly as is. Therefore, I wouldn't be able to exclude questions.
When I have received permission to use an instrument "exactly as is," I personally have interpreted that as that I must ask all the questions with the same wording as in the original. But that is not the same as using the results in the same way, so if your analysis does not support the conclusions from the original studies, that could well be due to differences between your population versus the ones where these items were used previously.
Note, however, that eigenvalues greater than one is a relatively weak way to determine the number of factors. At a minimum, I would use a "scree test" to examine whether there is an "elbow" or break point in the number of factors. Even better would be to conduct a Confirmatory Factor Analysis to compare the 6- and 9-factor solutions.
Finally, you need to think about your small sample size. The usual recommendation is that you have at least 5 and preferably 10 observations for every parameter (loading) that you estimate. It appears that there may be a large number of items in the data set you are examining, so the small N could lead to relatively unstable solutions.
Dimensionality and factor analysis are different statistical techniques. So researcher must conduct dimensionality analysis first (like scree plot or DETECT etc. analysis) then conduct factor analysis. In this situation confirmatory factor analysis is better fit with research questions.
Sample size is more important issue here. N:81 is a really slow sample size. Most publications offer at least 300 sample size or the number of observed variables multiplier by 10.
When people use a scale in a new country, they might do three things.
1. Confirm that the internal consistencies (coefficient alpha) for the subscales are similar to prior uses of the scale.
2. Conduct a confirmatory factor analysis to see how well the original factor structure fits in this new sample. You can run it, but your sample size is really too small to give stable results.
3. Conduct an analysis of measurement equivalence/invariance. There are structural equation modeling and IRT approaches. You would need a sample from the source language/country you wish to compare too. Again, your sample is really small to be doing that. We discuss this approach here:
Article Methodological and Substantive Issues in Conducting Multinat...
- Badges
- Science topic
- Similar topics
- Instrumentation Engineering
- Instruments