Use of BIC requires you to predict the same DV in both models. Best wishes David Booth

Hello Sara,

you could run a real SEM (not PLS model which is no SEM but a regression model) and test both models with the chi square test. The main difference between both models in terms of testable implications is that in model 2, y2 and y4 should become independent when controlling for y1 and y3 whereas as in model 1, y2 becomes independent from y4 when you control for y3. Hence, both models are non-equivalent and can be differentiated.

If model 2 fits, however, you still don't have any evidence with regard to the effect direction of y1 which can take any direction on any of the DVs and you still get an equivalent model wrt. model 2. You would have to employ instruments for y1 to differentiate these.

All the best

Holger

@Holger I can't agree. How can you compare models that model different things, I mean the DV.? The DV has to be the same. I realize that SEM is magic but even that is not enough to compare apples and oranges is it? David PS all of this is regression see S. Wright 's papers on path analysis. Virtually all of statistics is regression when you look at it clearly and closely Best wishes David Booth

Dear David,

I never heard the claim that you can only compare models with the same DV.

Let me try this one: The whole model structure is defined by the proposed effects and non-effects (constraints or restrictions). They define the characteristics and whether two models are equivalent or distinguishable. Actually, I explained in detail the testable implications of both of Sara's models and how they differ but let's take another (more simple) example especially focusing on different DVs:

Model 1 (M1) is a full mediaction structure: X --> Z --> Y

Model 2 (M2) is a regression-type structure: X, Y --> Z (assuming independence of X and Y)

Both models have different structures and different (final) DVs and most importantly, have different testable implications (which as aforementioned define the notion of equivalence).

In this case, both model claim different things

1) "X is related to Y" (M1) vs. "not related to y" (M2)

2) X is NOT related to Y once Z is controlled (M1) vs. "X IS only then related to Y once Z is controlled (M2). This of course, only holds, wenn there is no link between X and Y in M2

However, both also share identical implications:

1) X is related to Z

2) Z is related to Y

A nice tool to inspect the testable implications is dagitty.net. Actually, my first hunch about Sara's models was that they are both saturated and thus, not testable and not distinguishabel but dagitty showed me that this is wrong.

In addition, I used one of her models to simulate some data and tested the other model with the data (which did not fit the data).

Beware, however (as I stressed in my response to Sara) that a model in total can fit very well but a particulare target effect has no support. The reason is that partitions of a model can be "closed" with no testable implications while the larger context provides some implications. I discuss a nice example in a new editorial in Management (Box 2 on p. 7):

Steinmetz, H., & Block, J. (2022). Meta-analytic structural equation modeling (MASEM): new tricks of the trade. Management Review Quarterly. https://doi.org/10.1007/s11301-022-00293-6

A simplified version of the structure can be seen here:

http://dagitty.net/mw4Q4tc

(a nice feature of dagitty that you can create links via "publish").

The model has two testable implications that involve the relationship between a,b,c and d but a, b and c themselves are closed. Hence, if the implications hold in the face of the data (sieh the frame in the upper right), the model does not support the effectg directions connecting a, b and c but only the effect of c on d (since any other model with other effect directions whould represent a different model)

The relevant paper (which I never read, I must admit :) is

Verma, T., & Pearl, J. (1991). Equivalence and synthesis of causal models. UCLA, Computer Science Department.

http://fmdb.cs.ucla.edu/Treports/910020.pdf

All the best,

Holger

Sorry for the delay but I just managed to get back to this research and I have a lot to think about thanks to your feedback. Thank you so much Holger Steinmetz for your precious time and help, and also David Eugene Booth for the inputs. I really appreciate it.

Best,

Sara