Dear David

I am myself interrested in poorly recognized features of depression since years ("La Dépression et son traitement": aspects méconnus" L'Harmattan Paris 2011). 

Depression is poorly defined, mainly on psychological grounds and without any study of sensitivity of these definitions.

I would not be surprised if this component would be depression according to my concept.

I attach the english abstract of my book.

bmaroy.free.fr

INTERACTION EFFECT MEASUREMENT: If you input several factors into the program and the result came out only one factor is significant, it might be that the software is matching a linear relationship between Y and each X, several Xs that fails to show significant reading are dropped. This is a loss of information. If you determine interaction effect among factors, you might get more information. If you were expected to see, Y = B0 + B1X1 + B2X2 +...+  BiXi + error, but instead the result shows only Y = B0 + B1X1, you can go back to the multi-factors model and try to calculate the interaction among, say, X1 and X2.

Tx1x2 = (B1 - B2) / C

C = sqrt((D + E) / F)

D = (n1SE12)

E = n2SE22)

F = n1 + n2 - 2

If the T-test above is significant then the original equation: Y = B0 + B1X1 + B2X2 + error may be modified to include the interaction between X1 and X2 as: Y = B0 + B1X1 + B2X2 + B3X1X2 + error. If the T-test shows no sigficance, drop the interaction term B3X1X2. Repeat the process for all the sub-factors that you suspect may have had interaction to contribute to a significant

Hi David,

as you have so clear assumptions, the way to go is to do a confirmatory factor analysis that allows you to both specific a sub factor structure as well as a higher order structure. It may be that the primary factors (e.g, depression) are caused by a higher order vulnerability factor or that you have a bi-factor structure characterized by both primary factors and a general factor. 

Some caveats though:

a) think of such factor analyses not in ways of "reducing" data but in order of truely existing entities that cause other entities (i.e., the higher order entity affects true depression, and depression causes response behavior).

b) Be aware that factor analyses (esp. higher-order factors) can be complete nonsense, see Lee, N., & Cadogan, J. W. (2013). Problems with formative and higher-order reflective variables. Journal of Business Research, 66(2), 242–247. doi:10.1016/j.jbusres.2012.08.004

c) When you only have 3 primary factors, modeling a second -order factor provides no strong test of the second -order structure, as the latent structure is saturated (no df...no test even if the total df is >0).

d) there is some interesting work by Denny Borsboom and colleages that formulted alternatives especially for pathological constructs, e.g., 

Borsboom, D., & Cramer, A. O. J. (2013). Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9(1), 91–121. doi:10.1146/annurev-clinpsy-050212-185608

HTH

Holger

One option is to try different rotations in factor analysis.

Using oblique rotation may allow you to find factors that correlate. In a large sample, I would expect "psychological problems" to correlate significantly. So using an oblique rotation with a Principle Axes extraction (probably the least conservative combination)  will give you a good starting point for EFA. But I would follow this with CFA via SEM for instance.

Also, I would check the distributions for each measure, to make sure that you don't have a small number of outliers that skew the scores across measures.

(here is a short and a reasonable summary on rotations: http://jalt.org/test/PDF/Brown31.pdf)

happy to help David but might be best done back channel.  The place is to start is when you say you just get one big factor,what do you base that on?

The usual approach these days to the "number of factors" problem is to do a parallel analysis and start with the number of factors it indicates that you should retain: http://www.ats.ucla.edu/stat/stata/faq/parallel.htm. Are you basing the conclusion that there is only one factor on the fact that only one factor has an eigenvalue > 1 or on a scree plot or perhaps you have already done a parallel analysis? 

It can also happen, as you no doubt know, that  if you used a lot of affect words in your items that everything will tend to clump together on a general negative affectivity factor. In that case you can try to tease out variance not accounted for by the large underlying first factor by using a bifactor analysis: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2981404/.

David, As Holger remarked: you seem to have a priori ideas about which factors (symptom clusters) to expect, so it is a good idea to start from them, just by predicting them and have a test whether they are valid. Holger suggested CFA but when the ideas about the clustering of your items are still not well-circumscribed then approaches like the ones I suggested in a recent article may be preferable. Have a look at in on https://www.researchgate.net/publication/303029708

Good luck

Article Testing Predicted Clusters: A New Approach to This Powerful ...

Thanks to all who responded!  To clarify a couple of items:

Although I am pretty sure that there are factors in there, I'm not entirely sure what they are.  The initial idea was to ask about a very wide range (253 items, N = 2067 undergrads) of emotional, cognitive, and behavioral symptoms, and then see what the underlying structure is.  So it seems that an exploratory, rather than confirmatory, approach is most appropriate at this time.  As an example, I'm pretty sure we would get a factor that looks something like depression, something that looks like fear, something that looks like substance use, etc.-- but there are a lot of other items (e.g., questions about more nebulous things like rumination or impulsivity) that could end up being part of one factor, part of another, their own factors, etc.

When I do a PCA with no rotation (as one example) I get KMO = .988 and Bartlett p < .001.  There are 36 eigenvalues > 1.0 but the scree plot looks consistent with one huge component eigenvalue = 77.97, with the next factor's eigenvalue = 8.60, a big drop).

Now, intuitively it seems that the number of syndromes of mental disorder is probably greater than 1, and it's probably less than 36.  So we go to a parallel analysis, we get 4 random data eigenvalues that are larger than the obtained eigenvalues.  4 seems low, but let's go with that for now.

So we re-run the PCA specifying 4 components.  All of the items load onto component 1 (.72-.28). So it's pretty clear that this component just means "unhealthy in some way."  Not very meaningful, so let's ignore that for the time being.

Component 2 has 27 items that load > .30.  It's a mix of all kinds of sleep, anxiety, mood, and cognitive symptoms.  Hard to see much of an underlying theme here other than possibly "neurosis" or "negative affect."

Component 3 has 15 items that load > .30.   It is all of our aggression and psychotic symptoms-- which doesn't seem to make a whole lot of sense (aggression and psychosis tend not to go hand in hand).  My guess is that these reflect more severe or "rarely endorsed" items in the undergrad sample.

Component 4 has 11 items that load > .30.  No apparent rhyme or reason to them; an odd mix of neurotic, somatic, or bizarre symptoms that don't make any conceptual sense.

And 201 (79%) of our items don't load on to any of these factors.

Related to Helen's point, it doesn't seem to make any difference whether we go unrotated, varimax, or oblimin.

When we ignore the parallel analysis and look at different numbers of components (e.g., 10), the results don't improve.

First step I would do in that scenario is start with only the unrotated first factor. Any item that doesn't load on that unrotated first factor under the scenario of such a central first factor is a loser item and should be eliminated. If you want to be conservative, remove them one by one, rerunning the analysis each time.

Then, I think your ticket is using exploratory bifactor analysis in MPLUS. Initially, I would retain one or two factors more than what the parallel analysis says and see whether these factors are meaningful or noise. If they are noise, you know you are in the right ballpark in terms of number of factors and need to reduce how many you retain until you are left only with factors that make sense by and large. Then, one by one, eliminate any items that load on more than one specific factor (in bifactor models you have a single general factor and multiple specific factors) and this should clarify your factor structure.  You'll end up with items that load only on the general factor, and items that load both on the general factor and on only one specific factor--this is simple structure under a bifactor scenario.

Thank you!

Here's an example of syntax for bifactor EFA in MPLUS:

https://www.statmodel.com/usersguide/chap4/ex4.7.html

Further information in this pdf of a Powerpoint:  https://www.statmodel.com/download/handouts/MuthenV7Part1.pdf

David, Perhaps the problem is that you used students as subjects. A great majority of them will probably score rather low on a psychological complaints questionnaire, and in a non-typical way. The minority that has real compalints, representative for clinical conditions, will be too small to yield a sound correlation matrix. That might explain why you could find only one general factor. I hope you are in a position to have clinical groups fill in the quationnaire. Then, you may find clear-cut and interpretable factors by means of an obliquely rotated EFA.

After some further reflection I would like to add a suggestion to my last comment:

Since your sample is huge (over 2000 subjects), the portion that has more serious and typical complaints will be about 200 subjects. So, if you eliminate all subjects scoring low on almost all items, then a clinical sample will result that is large enough for a preliminary investigation. It will yield a much clearer factor structure, suited for improving the questionnaire before submitting it to clinical samples, and sufficiently encouraging for you to undertake this enterprise. Good luck.

Many thanks, Gary and Peter!

One disadvantage of PCA compared to "true" factor analysis is that the first component will account for the largest amount of variance.  If you are seeing one factor accounting for most of the variance, well, that's by design in PCA.

Thank you