Guys! I'm writing in English but you can answer in Portuguese. Currently I'm conducting a research using validated innovation capabilities scale of 4 factors developed for manufacturing firms (validation with 1300 companies). After obtaining 50 questions as pilot test (which was supposed to be a pure formality), I conducted the factor analysis which showed completely different results from the factor in original scale, without any logic of allocation of variables into factors, and the number of factors in much higher, and cronbach alpha for many factors are below 0.6.
Some details: The application process have been going really well. The sample I'm using is composed by manufacturing firms located in the same country of the article's survey which validated the construct. What I noticed is that in my sample vast majority of firms are either high-capabilities or medium-capabilities, while in the survey used by the article that validated the scale the majority are low-capabilities. Therefore, in my sample the standard deviations for each factor is around 0,4, while mean is about 3.9-4.4 in scale of 5.
Andrei, there are 2 issues you should look into. First, as you mentioned, you may have a biased sample that affects the generation of consistent factors.
Second, I assume here you are using the original Likert-scale variables to build the factors. Is that correct? In this case, what method are you using? If you rely on PCA, you are violating the normal distribution assumption. This can be less problematic if your non-parametric, Likert results resemble a normal distribution, but, following your question, I understand you have a highly asymmetric distribution. The problem here is not the standard dev, but rather the concentration of responses in higher values. You could/should try extracting the factors from the polychoric correlation matriz (instead of the Pearson correlation matrix). This can be easily done in Stata. In SPSS you have the option of using Categorical Component Analysis, under the Optimal Scaling menu (dimension reduction option). Not sure how this is going to play out because of the first issue (sample bias).
Check this out: https://stats.idre.ucla.edu/stata/faq/how-can-i-perform-a-factor-analysis-with-categorical-or-categorical-and-continuous-variables/
Hope it helps
It seems that the problem is with the sample as you mentioned. You conducted a pilot test on 50 respondents. This could make a bias sample and the results of the factor analysis can be misleading.
Hello Andrei,
A sample of 50 is too small for exploratory factoring for a measure with four or more factors. In your case, if any analysis were to make sense, I'd say it would have been a confirmatory analysis, checking whether the previously validated factor structure works adequately to explain the observed relationships among variables in your sample.
Why can factor solutions vary from what has been previously published?
1. Sample variation. The smaller the N, the greater the risk that sampling variation alone is sufficient to yield "disparate" factor structures for the same measure, even when the samples are drawn from the same population.
2. Cultural/language differences. Perhaps, in your case, the culture of the companies you selected is sufficiently different from that of the companies used by the developers of the measure that respondents are not interpreting the items, questions, or stimuli in the same manner.
3. Different extraction / rotation / salience decisions applied to the factoring process.
4. Changes in perception due to time or some event (e.g., the market crash in 2008).
5. And so on...
The fact that you're observing much smaller variation in factor scores could suggest: (a) an overfitted factor model (too many factors); (b) a sample that, compared to that used by the measure developers, was far less heterogeneous; or (c) some combination.
In any event, I'd hold off on disseminating these results as being "gospel" until I had more data in hand.
Good luck with your work.
Bruno Fischer , first of all, I thank you a lot for your suggestion. I'll try the SPSS option and I'll learn about Stata software. What happens in my case is that the manufacturing sample is supposed to be composed of firms with a quite high capabilities. That's because this is one of the characteristics of the Brazilian firms that interact with scientific research group in comparison with general sample of manufacturing companies. So, probably, any scale I use will have more or less the skewness problem. Another point is that I chosed the current scale because it was development for measuring particularly the capabilities of manufacturing firms and previously it have been widely applied in many Brazilian surveys. The data collection is for my Phd thesis so I'm a bit lost. But thank you so much!
Dear @Andrei,
Please refer to my study guides on EFA and scale reliability with small samples. It is not impossible to undertake a reliability analysis with a sample size of 50 according to simulation research. However, if your instrument is already validated you should be using CFA not EFA.
Olá, A amostra que usou é muito pequena, o que compromete a confiabilidade. Ademais, dado que as características da amostra pesquisada (não sei se é só em relação aos respondentes ou na pesquisa geral), diferem da pesquisa original, terá um viés nos teus resultados.
Dear Peter Samuels , where can I find the referred guide? Thank you in advance!
- Badges
- Science topic
- Similar topics
- Statistics
- Survey