Dear Sara,

not only RMSEA but also CFI and TLI should be close to .95 or greater. Additionally, your chi-square is stat. significant, but keep in mind that Chi-square is very sensitive to the sample size. Thus, if you have a large sample, this is ok.

I would however check for areas of localized strain in the solution by inspecting residuals, modification indices, or wald test. Those can tell you in which part of the model the potential problems are.

If you have a strong theory behind your model CFA is completely ok.

Hello all,

if you allow, I would present a different perspective. The chisquare is highly significant, meaning that the model fails in predicting the data beyond the amount expected by chance. This may be trivial or substantial, you don't know it. High sample size does not cause misfit, it increases the statistical power (which it should as it is a statistical test). The cut-off values of fit indices are complete bogus as they cannot be generalized beyond the scenarios tested in the Monte-Carlo simulations which underled the values. These were artificial and minor and do not express those fundamental misspecifications a CFA model often implies.

Hence you have to diagnose the problems and come up with a proposal.

Here's a thread in which a paper is cited that shows the process how to do it

https://www.researchgate.net/post/Model_fit_measures_in_CB_SEM

Best,

Holger

Hello Sara,

There are not exact cutoffs for fit indices (unless you insist that they all be perfect, of course, which never happens in an over-specified model except in contrived data sets). There are only suggested guidelines, which may or may not be useful (facetiously: they're useful if your results meet them; they're not otherwise!).

It's all judgment, based on your research aims (what you're trying to accomplish by evaluating the model in the first place), your results, and whether benign model modifications exist that would not cause spurious overfitting or threaten the integrity of whatever theory/rationale was undergirding the model. Failing that, then the challenge is to suggest plausible reasons to rethink the model in a more profound way.

Good luck with your work.

Ответ экспертов@@

I see. Thank you so much Jelena Veletic

Hello Holger Steinmetz

Thank you so much for your insightful comment. I followed the thread, it was quite helpful.

Bests,

Sara

Thank you so much David Morse

You are so right. I just had the idea that my work has no value unless the model fits perfectly! But your insights made me see the issue more realistic. I will delve into the results and see why and what may cause this model fit and how I can possibly explain that.

Bests,

Sara

Sara Hoseingholizade You are welcome. Of course, the cut-offs are just recommendations, and should not be followed blindly but they give us insights into our models and help to decide about further steps.

Good luck with your work.

There is no rule of thumb for a universal cut-off points for fit indices. The debate is still ongoing as to what exactly constitutes a perfect model fit. But, the literature gives values from .80 up to 1.0. any values in this range are considered appropriate. I also suggest you perform an EFA before CFA especially as the instrument has been translated. Who knows, you might have to delete some of the items before performing CFA based on the EFA results.

I see. you are right, I should run an EFA to see what's causing problem. Thanks, Sebastian Gyamfi

Hi again, a EFA will show the problems if you have a whole set of items which the CFA restricts on to few factors (e.g., 5 items being forced to load on 1 factor where in reality 3 items reflect one factor and the 2 remaining reflect another). Often the problems may be more complex or / and you have single items. In these cases, the EFA may not only be of little help but dangerous (as it will suggest low loadings of very important items). Using an EFA for selecting indicators is mindless.

Avoid the narrow factor perspective. Learn path tracing and d-separation and how models imply data structures. Then you can reverse engineer possible alternative structures that may have caused the data (and the trouble).

And again one comment with regard to the fit indices. If you allow me this nitpicking, but values can only a informative when they carry some information. Any value of a fit index may or may not be informative. In face of existing problems, they will NOT inform you whether the problems are trivial or substantial.

HTH

--Holger

If you have used existing valid scales without any modifications, then there is no need to run an EFA. The problem is with the responses received. Either some sort of biasness or no data cleaning is making results a bit deviated. Removing outliers or data cleaning may help in improving RMSEA and pclose.

I agree with Mr. S M Fatah Uddin. You need not run EFA on a well-established scale used in prior studies. There might be univariate or multivariate outliers in your dataset which are causing model-fit concerns. I suggest you check Mahalanobis distance and drop responses with p

Thank you very much Holger Steinmetz . That was pretty helpful.

You were totally right; the EFA results for my single-factor questionnaire were not promising (it suggested 3 for eigenvalue and even when I forced it to 1, the eigenvalue was 2). Since so many editors ask for EFA, are there any references you recommend to support that EFA is a risky option for single-factor questionnaires?

For the d-separation and path tracing you mentioned, I searched a bit through it. Any specific recommendation and sources for learning it?

Bests,

Sara

Thank you so much S M Fatah Uddin and Priyanka Aggarwal

Well, you are right. I shouldn't rely on EFA results.

My data is pretty clean since I used a platform. But for the outliers, am I allowed to eliminate the outliers to gain a better result?

For another study I had, as I recall, researchers didn't recommend eliminating the outliers.

Bests,

Sara

Hi Sara,

path tracing goes back to Sewall Wright, the inventor of path analysis and, thus, SEM. It is quite easy to understand and simply a matter of exercise, repetition, and application. It consists of three simple rules which describe how a specified causal system implies data (correlations among model variables).

The wikipedia page about path analysis contains a description:

https://en.wikipedia.org/wiki/Path_analysis_(statistics)

There is nothing more to it. As I said: Apply it

For instance, a CFA/common factor model would imply that the correlation between two indicators follows from the product of both loadings. Nothing more. Hence, if the observed correlation between both exceeds the model-implied correlation, there must be something additional going on. This can be one indicator affecting the other, or an omitted further latent variable.

D-seperation is very similar to path tracing (if you want the other side of the same medal): While path tracing concerns the model-implied correlation, d-separation says that the correlation between two variable becomes zero ONCE you control or adjust for (some) other variables lying on the path. Regarding the example above this means that the correlation between these two indicators becomes zero if you control for the common cause / factor. This again is an implication of the model. And more important: It is a testable implication of the model. That is: once you specify a model structure you get a set of d-separation claims that reflect the overall test of the model. The chi-square test is a summation of all these tests.

Modells differ with regard to the number of d-sep claims: A fully saturated model has no d-sep implications and, thus, is not testable. Take for instance the fundamental difference between a full and a partial mediation model: For the full mediation model linking X and Y via M (X--a-->M--b-->Y), the path tracing rules would claim that the correlation between X and Y follows from a*b. Further the model has one d-sep claim and thus this is the only testable implication, namely that X _|_ Y | M (read: X and Y are conditionally independent given M or in other words: they don't correlate when you control M (controlling M "blocks" the path).

In contrast, a full mediation has the path tracing implication that the correlation between X and Y follows from a*b+c (c is the direct effect) but the model has no testable implication.

By using these things, you gain an substantial knowledge and understanding of the relationship between models and data and have some basis for respecifying a misspecifying model.

Here is a bit to read

Hayduk, L. A., Cummings, G. G., Stratkotter, R. F., Nimmo, M., Grygoryev, K., Dosman, D., . . . Boadu, K. (2003). Pearl's d-separation: One more step into causal thinking. Structural Equation Modeling, 10(2), 289-311.

Chapter 8 in

Kline, R. B. (2016). Principles and practice of structural equation modeling (Fourth ed. Vol. 156). New York, London: The Guilford Press.

HTH

Holger

@Holger Thank you so much for all the clarification and sources you mentioned. That is indeed pretty helpful.

By the way, are there any references you recommend to support that EFA is a risky option for single-factor questionnaires? I need some technical support to convince the editors why I shouldn't go with EFA.

Sara,

I would rather argue that once you have a theory/expectation the explicit TEST of this theory is warranted as the EFA exists to EXPLORE the data. An existing theory implies more constraints and by these reduces the space of all possible results.

As a reference, I would simply mention Brown:

Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford.

You may perhaps find some arguments in this thread where I argued for confirmatory approaches:

https://www.researchgate.net/post/EFA_exploratory_factor_analysis_CFA_confirmatory_factor_analysis_or_both#view=5c9937470f95f1a905604215

HTH

Holger

Thank you so much Holger Steinmetz