Dear All,
First of all I hope everybody is doing good in these weird times.
Second, I also have a question regarding a non converging factormodel. I translated a questionnaire from English to Dutch, and tried to validate it using an EFA, CFA and even parallel analysis up to 11 factors, but still no logical structure. Because we are also interested to see whether the psychometric properties of the questionnaire correlate with other self-report questionnaires that we conducted, we wanted to use te total score of the questionnaire to correlate with the other self-report.
Opinions are a bit divided within our team about the theoretical correctness of using a total score when the factor structure is not converging.
I was wondering what you as other researchers think about this and if there is a maybe another way of interpreting the psychometric properties of the questionnaire with the other self-report when the factor structure is not as planned?
Thank you so much in advance!
Dear Carmen,
I am not sure what you mean by a logical structure. Certainly 11 factors is way too much. Generally a factor needs at least three variables to be a meaningful and interpretable quantity. I would be surprised if you had three questions for each of the eleven factors.
Why not do a Principal Component Analysis on your questionnaire? It will tell you what the structure of the questionnaire is and you might gleen what the problem is. I assume you have already checked the raw data and their distributions to see whether there are any rogue observations. I would not advise the use a total score until you know what it signifies.
PMK
Pieter Marinus Kroonenberg Thank you!
I will answer you with more information in the e-mail.
Hello Carmen,
if your CFA does not converge, there is a reason and hence, the goal is not to avoid the model approach at all but to find out what the problem is.
First of all, it is important to understand what the factor model (which you test in a CFA) really means and how it differs from simple aggregates or composites. I don't want to repeat that as I wrote a detailled presentation here:
https://www.researchgate.net/post/Dear_scholars_how_to_design_a_Structural_Equation_Modeling_surveys
and here
https://www.researchgate.net/post/Validity_and_Factor_Analysis
Deviations from the factor model often occur when researchers generate items on a very specific level while their "construct" is rather an abstraction (i.e. aggregation) of these specifics. It may well be that the specific items follow a factor structure but most often this is not the case. Hence, correlations will not follow the implications of the factor model and the factor model, consequently will not fit or worse, not be identified (i.e., converge).
Hence, a bit more background information on the constructs and question wordings would help to judge that. If you theory is however clear, but is empirically invalid, the software may provide some information--for instance about Heywood cases (i.e., negative variances). Further, a breakdown of the overall model in pieces would be a manageable way to the goal (with some specific implications about the communication of the results and possible replication or enlargement issues).
I agree partly with Pieter, that 3 items would be nice but only if the three items follow the factor model. Three items, however, are not necessary (here I disagree with Pieter, respectfully). Two clear, valid, and strong items may be a realistic goal with no downsides or their usefulness (e.g., prediction), see this excellent example:
Antonakis, J., & House, R. J. (2014). Instrumental leadership: Measurement and extension of transformational–transactional leadership theory. The Leadership Quarterly, 25(4), 746-771. doi:https://doi.org/10.1016/j.leaqua.2014.04.005
If a 2-indicator model is uncorrelated with all other variables in the model, you may run (again) into identification issues but you would be an unexpectable uninformed researcher if you incorporate constructs in a network that have no correlation. If this is the case, why keep them there? Besides that, identification issues can be treated by incorporating constraints (e.g., equality constraints). Even models with one crystal-clear indicators are possible and surely a better choice than having 3 misspecified indicators.
One example: with some colleagues, I submitted a paper in which we used single-indicator measures of a database on corona-related worries, acceptance of social distancing measures etc. The reviewer (as always) criticized the single-indicator measures. So we did a further study (N=612) in which we generated 2 further indicators for the 5 factors each (to have 3 per factor). The CFA model failed by the chisquare test (not by the fit indices, just noted) and after inspecting the diagnostics of the software, some indicators had to be eliminated as it was clear that they did not measure the factors. We ended with a model, in which two factors had three indicators, two factors had two indicators, and the last one had only 1 indicator (we had generated two further items that completely failed to match with the key indicator from the data base). The model had a non-significant chisquare, loading above .90 and reliabilities ranging from .89 to .93. In a second step, we enlarged the model with 6 specific predictors which not only provided evidence on their construct validity but also served as a stress test for the fitted factor structure.
HTH
Holger
Without going too deep into the issue I would think that:
* a one-variable factor is the variable itself, imperfectly measured.
* a two-variable factor models the correlation (covariance) between the two variables. The factor may a valid one underlying, or causing the correlation (covariance). But would it not be a bit uncertain what this latent variable (factor) measures just based on the two variables, especially if the correlation (covariance) is not well fitted ? Of course, it might very obvious, but it will probably not always be so.
- Badges
- Science topic
- Similar topics
- Psychology
- Psychometrics