Can anyone help provide statistical advice/insights for an error we are receiving within r when conducting higher-order factor analyses for the four primary order factors in a scale development/validation study?

Error message: CFA in R in determining a higher-order factor returns “lavaan WARNING: some estimated lv variances are negative”

Study 1

In the MTurk sample of working professionals (N = 373), the Study 1 EFA using principal axis factoring and oblique rotation indicated 4 factors.

Study 2

In Study 2 using another MTurk sample (N = 435), CFA with robust ML estimation confirmed the four factor solution from Study 1, and the model demonstrated satisfactory fit indices.

Higher-order factor analyses for the four primary order factors showed good model fit indices but reported “lavaan WARNING: some estimated lv variances are negative.” One factor had negative variance.

In Study 1, this factor (that appeared to have a negative variance in Study 2’s analysis for a higher-order factor) was highly correlated with another factor. These two factors do show overlapping constructs.

Possible reasons for error:

• MTurk samples display higher correlations than normal populations.
• The two correlated factors are one factor.

We are considering running a CFA in our Study 3 using an organization, not an MTurk sample, to see if error continues.

Extra Testing Conducted for Reference

Higher-order analysis using four factor solution in Study 1

We conducted a CFA with the Study 1 sample, using the four factor structure. Model fit indices were satisfactory. Higher-order analysis reported the same error of negative variance.

Specifying a three factor solution

We tested whether those 2 factors were actually just one factor, by conducting EFAs and CFAs for three factors.

Study 1: The two highly correlated factors did become one factor in an EFA using the Study 1 sample, when specifying a three factor solution. However, communalities in that factor were relatively low. With the Study 1 sample, higher-order analysis reported the same negative variance error.

Study 2: Parallel analysis indicated a three factor solution with the Study 2 sample. Model fit indices were satisfactory in the CFA. Model comparisons did demonstrate that a three factor structure and a four factor structure are significantly different. Higher order analysis for this three factor solution did not report negative variance.