Hi Samantha,

first, latent variables are always present since all observed measures contain some amount of error. A path model has the problem that it makes exactly the critical assumption -- namely that the IVs are error-free. If this assumption is wrong, you create endogeneity which biases the structural effects. This will of course depend on the amount of error. Hence, incorporating some reflection of the error would be fortunate. That -- of course -- does not mean that you have to force any set of indicators into a common factor framework. This is the most prevalant error in setting up measurement models. The alternative could be to specify single-indicator-latent variable models (although in practice reviewers will never get you away with that *lol*). That is have a latent variable with one indicator and specify the loading to 1 and the error to some amount of assumed portion of errors. As mentioned, this will always lead to reviewers going mad and shout "you don't know that amount of error". The point is that every model is always a set of assumption and all inference is conditional on the correctness of the assumptions. You cannot escape this basic truth.

With regard to you section question, I can only guess. I would think that adjustment is always warranted when you have more than one "chances" to test on and the same hypothesis and you will regard a sucessfull test /corroboration when at least one test is positive. This clearly needs adjustment. Another alternative could be to clearly consider the number of chance and number of confirmed tests. This is clearly not very statistically founded, hence, adjustment may be more formal....

Just my 2c

Best,

Holger

Hi Holger,

Thank you for your response.

Yes - exactly - how does one determine that amount of error? Dodgy business.

And I agree, a latent variable with one indicator seems meaningless. Although I understand and agree with your philosophical argument that latent variables are always present.

Yes, all models are based on a set of assumptions - my question remains though -

Why choose ANOVA over Path Analysis (PA)?

One could presume that this is due to the restrictive assumptions compared to ANOVA? (uncorrelated residuals in PA vs normally distributed residuals in ANVOA). PA assumes IVs are error free - I guess ANOVA inadvertently assumes this as well. Whereas a SEM (latent model) would then take account for this error.

Hmmm.

Hi Samantha,

when you write "I agree, a latent variable with one indicator seems meaningless" do agree that it SEEMS meaningless (to others) or do you yourself share this impression? :)

Of course, having several indicators of a latent variable is always nice but there are some complexities:

a) Often we are stuck with just one indicator. Hence, you have to make a choice. Fixing the error var to 0 IS a choice (that's how it is usually done) but a very unplausible one. Hence, fixing the variance to any other non-zero value would be an improvement. A model is always a set of YOUR assumptions, errors are no difference. Beyond the issue of "knowing the true error", a different perspective is to REGULATE the conceptual distance between you indicator and the latent variable (how close should the distance be in order to "get" the latent you want? Thats is, you regulate the distance between the indicator and the latent by choosing a smaller (closer distance) vs. larger amount of error (larger distance).

Below I posted a few paper that illustrates this.

b) Incorporating / measuring several indicators that really conform to the common factor model is often tricky. Increasing the semantic heterogeneity will often lead to measuring different latents (which is indicated by the misfit of the model). However, incorporating to very similar (functionally equivalent) items will either 1) annoy the respondents, 2) lead to copy-and-paste behavior or serial effects of one item causing the other) or 3) discriminating processes. That is, the respondents will wonder why such similar questions are asked and will respond to the tiny difference in the subsequent questions. This again violates the assumption of the common factor model that all items are caused by an underlying factor and the indicators are locally independent.

All of that does not mean that one should go for single indicators but all possibilities have problems.

With regard to you question about ANOVA: I have no idea. I can only guess that the reason is simply institutional - that is ANOVA has ever beein applied by experimental psychologists because you have categorical variables. Simple culture. There is no such thing in econometrics, epidemiology or whatever. Further, ANOVA and regression/SEM are equivalent - that you can represent a set of categorial variables with a set of dummy variables in a regression.

The particular assumption of error-free-IVs in PA vs. ANOVA is likely be the result of the different domains (experimental laboratory studies vs. surveys) which allow a different amount of control (experiments have no error as long as the assignment process works vs. IVs in surveys are often random and endogenous).

I have the feeling that this will not really satisfy you :)

Best,

Holger

Hayduk, L. A., & Littvay, L. (2012). Should researchers use single indicators, best indicators, or multiple indicators. BMC Medical Research Methodology, 12(159), 1-17. doi:10.1186/1471-2288-12-159

Duncan, O. D., Haller, A. O., & Portes, A. P. (1968). Peer influences on aspirations: A reinterpretation. American Journal of Sociology, 74(2), 119-137. doi:10.1086/224615

Oberski, D. L., & Satorra, A. (2013). Measurement error models with uncertainty about the error variance. Structural Equation Modeling: A Multidisciplinary Journal, 20(3), 409-428.

Onyskiw, J. E., & Hayduk, L. a. (2001). Processes underlying children's adjustment in families characterized by physical aggression. Family Relations, 50(4), 376-385. doi:10.1111/j.1741-3729.2001.00376.x

Anova is used to evaluate quantitative data in function of categorical data... Path analysis relate cuantitative variables (path analysis for manifest variables, SEM for latent variables).

Thank you so much for your input Edgar.

Yes, indeed, you are correct.

I just wonder - as both continuous and categorical data can be included in a path analysis - why would someone run multiple ANOVAs on their data - then run a Bonferroni correction ... why not simply put all the variables into one model in terms of a Path Analysis?

Is there an advantage of running multiple ANOVAs over Path Analysis?

Path analysis require multivariate normal distribution, categorical data violate this assumption. I understood that you mention bonferroni correction for a post hoc means comparison.

Hi Edgar, are you arguing that we cannot include Gender (as an example, if we operationalise this as 2 categories) in a Path Analysis then?

Article A path analysis of gender, race, and job complexity as deter...

If you violate this assumption, you must use another types of correlation matrix (tetrachoric or biserial depending the level of measurement of the items), and review your estimation method (maximum likelihood, must be strongly avoided). Another option will be use an evaluation of invariance for those categorical variables...

Hi Edgar, Hmmm - I see what you are saying - but I don't think this is usual practice in the Social Sciences(?) I've come across numerous studies that include categorical variables in Path Analysis, without special consideration.

Also, MLE is the default estimation (in R anyway) - and most might not change this, unless assumptions are seriously violated - if they are, then the entire model might be re-evaluated.

Might this be a mismatch between the Natural and Social Sciences?

In your field would you ever choose to run numerous ANOVAs on your data? or do you mostly rely on Regression analyses? (or possibly another technique that we might not use in the Social Sciences?)

I recognise that statistical analyses should be consistent across disciplines, but in my experience, this is not always the case. Have you had experience of this - different interpretations of the underlying mathematics?

So do not waste time asking technical arguments when from the beginning you think you can justify a methodology on a bad research practice. And for your information, at this moment I am working in research in social sciences.

Folks, there no problem in using categorical variables in SEM/PA: By using the "fixed.x"-approach, values of the exogeneous variables are treated as fixed (not random) and thus, assumptions of multinormal distributions only concern the downstream variables in the model. I don't know how AMOS handles this but lavaan and Mplus applies this approach as a routine.

If you don't trust me, here's a post by Yves Rosseel, the inventor of lavaan:

https://stat.ethz.ch/pipermail/r-help/2011-March/272896.html

For categorical / binary variables, both programs provide options to model the latent response variable underlying the categorical response and use estimators that can handle that (e.g., DLWS)

Best,

Holger

Hi again,

because I was curious, I conducted a small simulation to show the equivalence of ANOVA, regression and SEM:

# The data are simple: A is a binary variable, affecting Y with an effect of .5

set.seed(911)

A = round(runif(100, 0,1),0)

Y = .5*A + rnorm(100)

data=as.data.frame(cbind(A,Y))

data$A = as.factor(data$A)

###########ANOVA model##########

anova.model The mean difference is .48, and its p-value is .01

############SEM / path model#########

library(lavaan)

pa.model |z|)

# Y ~

# A 0.484 0.189 2.557 0.011

# --> Both effect as well as p-value are the same

#############Regression##############

summary(lm(Y~A, data=data))

# Estimate Std. Error t value Pr(>|t|)

#(Intercept) -0.08951 0.14321 -0.625 0.533

#A1 0.48433 0.19137 2.531 0.013 *

# And again, everything is the same.

So the "bad practice" is actually legitimate practice, and Samanthas question was likewise legitimate.

With best and respectful regards

Holger

I saw the argument in the blog that you mention. But there are a problem, in regression, assumptions are made for residuals in the model, in SEM the assumptios are related to multivariate normality, quite different... And making one model in path analysis reduce error type I, compared to multiple regression models, quite different too... With best and respectful regards :)

This is very interesting indeed Holger Steinmetz. I assume the same would apply to more complicated models. I think the best would be to do a similar simulation on my data to confirm. Thanks again everyone for your contributions. I've thoroughly enjoyed this debate and discussion.

What? One example and you make a generalization? This is worst science possible...

Edgar,

What about a second, more complex, and more realistic example?

Here, we have tqo binary variables (D1 and D2) which haven an effect on M, which mediates their effect on Y (hence, a full mediation model). In addition, both D1 and D2 interact to make the example more similar to typical examples in ANOVA. In the simulation, the effects are: D1 (.20), D2 (.20), their product term D1D2 (.80) and M (.50).

I simulated a sample size of N = 100,000 to show that the estimator is consistent (by the way the crucial difference between SEM and regression, in which the normality of the residual has a relevance only for the estimation of the standard errors and not of the coefficients themselves. In SEM, the question is whether you violate the assumption of the ML fitting function which is relevant also for the parameter coefficients).

The simulation again shows that the parameters are consistent and don't suffer.

If you are aware of any context variables that restrict the generalizations, I would really like to know them as this would have indeed an impact whether to apply SEM to categorical data. I can only tell from the literature that I read and from overall recommendations by the true experts. If any of my examples have some flaws, again, tell me. In the end, I hope, we all want to know what the best procedures are, don't we?

Best,

Holger

P.S. the attached results also show that the model fits despite this large N which suggests that

#Data

set.seed(911)

D1 = round(runif(100000, 0,1),0)

D2 = round(runif(100000, 0,1),0)

M = .2*D1 + .2*D2 + .8*D1*D2 + rnorm(100000)

Y = .5*M + rnorm(100000)

data=as.data.frame(cbind(D1,D2,M,Y))

data$D1D2 = data$D1*data$D2

data$D1 = as.factor(data$D1)

data$D2 = as.factor(data$D2)

data$D1D2 = as.factor(data$D1D2)

#SEM

library(lavaan)

mod

Sorry, but your exercise still being inductive. Please review the theory about SEM and regression, there are a lot written and evaluated... In this way it is just pseudoscience...