I'm not sure I fully understand what you are asking so forgive me if my answer is off topic. I'm assuming you are asking whether your exogenous factors on the left hand side of your path diagram should be allowed to correlate, that is, whether their covariances should be estimated as free parameters. The answer to this is that your model is probably misspecified if you assume no covariances / implicitly constrain/fix all of those covariances to zero because at least some of the covariances may be significantly different from zero. As a result, your structural regression coefficients may be biased towards significance. I would check to see whether the model without covariances fits significantly worse than the model that does allow the covariances. If the no-covariance model fits worse, then the model with freely estimated covariances should be preferred.

The covariances are important because they account for the partial redundancy of the exogenous factors in predicting your dependent factor. If you (incorrectly) fix them to zero, the results of the structural regression may be incorrect also.

Dear Christian Geiser

Thank you very much for your question. I would like to clarify my request.

In fact, the model without covariance fits better, as it shows more significant p-values. However as you pointed out, I am running into issues because of the disregard for partial redundancies, correct?

The covariance model causes issues.

Based on similar research, only 1 significant path, as in the covariance model in the screenshot, is very unlikely. I must have done something wrong or have overlooked something. To be 100% sure that we are talking about the same covariances I added a screenshot.

• (A) I want to report the no-covariance model because it has more significant p-values. Can I go ahead and do that even if AMOS tells me that exogenous factors need to be covariated?
• (B) If not, is there any way to influence the covariance model more towards significant p-values? As you can see from my prior screenshot, all items load pretty well. I have seen far worse item loadings with better results eventually.
• (C) How would you interpret/report my results?

I hope my request was clearer this time.

Best Regards,

Sven Kovacic

Sven Kovacic What are the chi-square values, their dfs, and p values for the two models? That's what you should look at first in terms of model fit/to test the models/find out whether they may be misspecified. The p-values for the structural paths (regressions) may be significant even for a misspecified / non-fitting / incorrect model. In fact, as I was trying to point out in my previous response, it could even be the case that the paths are biased towards significance in the model without the covariances. The reason is that the estimation of the rest of the model is done in such as way as to "compensate" for the covariance misspecification. This can lead to "significant" estimates that would not be significant had the model been correctly specified. The question of fit needs to be addressed based on global fit statistics such as the chi-square first, not based on individual parameter estimates.

Another way to look at this is to examine the covariances in the model version in which they are estimated. If one or more of the covariances are statistically significant, then the model that omits them is most likely misspecified and will potentially lead to biased estimates and p values.

Christian Geiser My high regression weights or high Chi-square/DF might be an indicator that something in my model is misspecified, still I cannot see what it is. I have redrawn the measurement and structural model to eliminate any small error. Please find on the link bellow a summary of all parameters you asked for. What suggestions do you have based on this?

https://drive.google.com/file/d/1j_8V0slWWM-tOpJkgrc69xfiAcB0K3a7/view?usp=sharing

Thank you very much for guiding me through. I hope that others will also get some benefit from it.

The two models presented in the Google drive file are, for all intents and purposes, identical except for the fact that the structural model introduces assumptions of causality. I echo the concerns of Dr. Geiser; eliminating all of the covariances from the model would in all likelihood be seriously flawed and not properly specified. Also, I believe that what Dr. Geiser was requesting was the chi square, fit indices, and measurement/structural parameters for the model where you go ahead and constrain the covariances to equal 0. In essence he (and I) would like to see how much the model deteriorates IF you constrain the covariances -- we know that it will deteriorate, but by how much.

Lastly, I must encourage you to be very cautious of p-hacking. Some of what I am reading here gives me reason to be concerned about this as a contributing factor.

Dear David Mangen & Christian Geiser , I usually take the results as they come regardless how disappointing they are. It is rather the highly surprising result, completely different from related studies, that makes me suspicious.

I can improve the p-values, but as you have suspected, the model fit deteriorates dramatically. I might be able to improve all model parameters further on, but at the cost of deleting more items. This iterative adaptation process seems normal, but in my particular case, I worry that I'm diluting the model too much while the complexity is increasing. What is your opinion on that?

Please find the requested figures on the link:

https://drive.google.com/file/d/1l0v8fV25tNNWgzRwRSVvvrdN9imprydQ/view?usp=sharing

Since my focus is on proving causality between the latent variables and behavioral intention rather than evaluating model fit, it is probably more reasonable to choose a different SEM approach that is not based on covariance.

In the meantime, I tested my model using PLS-SEM without encountering any particular anomalies. I think that the change towards partial least squares is also in the spirit of Venkatesh et al., founding fathers of UTAUT2 and many other researchers who also used the method in recent studies. (Lee et al. (2019), Cabrera-Sánchez et al. (2021), Rahi et al. (2019), etc)

- I have neither reliability nor validity problems

- I do not have to delete any item

- The explained variance is comparable to other studies, similar to the model fit figures in covariance-based SEM.

I was also able to find some direct recommendation on PLS-SEM (Joe F. Hair Jr. , 2017),

The reason I initially decided against it was that I thought PLS was basically only used in combination with EFA. In the meantime, however, I see more and more PLS-SEM explanations and studies that apply the logic of CFA/measurement and structural models. (Samar Rahi, 2017)

What do you think, would you advise me to switch to PLS-SEM?

Best,

Sven Kovacic

It is clear that the model should include most if not all of the covariances. I might consider adding all of them back, followed by eliminating the statistically non-significant covariates. Alternatively you could simply add covariances as necessary to approximate the saturated model that you do not like while leaving out the few that are non-significant.

Note that proving causality is going to depend more on your research design and data collection methods than it will on the exact form of the SEM. Too often people believe -- erroneously -- that simply using some variety of the SEM model gives them direct insight into causality. It does not; it helps clarify the knowledge regarding causality if we either assume causal direction or have collected the data in a manner that permits assumptions of causality, e.g., temporal ordering..

Why do you dismiss the effect that Lerw has on VA? With an effect coefficient greater than 3 times its standard error it too is statistically significant. You have another effect that borders on statistical significance with p = 0.053.

In my opinion switching to PLS-SEM solely for these reasons would basically be searching for alternative ways of finding statistical significance, i.e., p-hacking. If you had some other better, more substantively valid reason for doing so I could see a rationale. Rarely, however, does the choice of a method dramatically alter the results. The data are still the data, and you are trying to explain what is going on in these data.