Dear Phei-Chin Lim!

I would like you to go through this page:

http://www.hawaii.edu/powerkills/UFA.HTM for this description:

......"Scaling. A scientist often wishes to develop a scale on which individuals, groups, or nations can be rated and compared. The scale may refer to such phenomena as political participation, voting behavior, or conflict. A problem in developing a scale is to weight the characteristics being combined. Factor analysis offers a solution by dividing the characteristics into independent sources of variation (factors). Each factor then represents a scale based on the empirical relationships among the characteristics. As additional findings, the factor analysis will give the weights to employ for each characteristic when combining them into the scales. The factor score results are actually such scales, developed by summing characteristics times these weights. "

Hope this helps...

In Exploratory Factor Analysis, you'd first decide the number of factors to be extracted - how many factors does it take to explain most of the variation:

http://en.wikipedia.org/wiki/Factor_analysis#Criteria_for_determining_the_number_of_factors

Then chose the factor loading cut-off level. What you can then do (to help visually), is to copy your factor scores over to excel and paste the scores for each factor in a distinct column(if you chose 3 factors, 3 columns). The ones that you consider are significant scores above the cut-off level. So negative scores are out. With this information, you consider your research question, and see what makes the most sense. Now consider the remaining scores on each factor. The closer to 1 they are, the more important they are in explaining the variation in that factor. There may potentially be relationships among those elements in each factor, which the research question, or background information may help you interpret. Do the same for interpreting the other factors.

"More than one interpretation can be made of the same data factored the same way, and factor analysis cannot identify causality."

Since each factor is a combination of all the variables you are working with (eigen vectors are the weighed for each variable), each factor can be positive or negative. It is positive when the variables with greater weight take high values and it is negative when they take low values. Please remember that the zero value is the mean of all variables.

Factors are standardized to a mean of 0 and variance of 1, as they are latent variables composed of pieces of varying scales - think of it like the only way to fit everything together is within a standardized entity.

Assuming that you used a principal components extraction, and incorporated all variables in each retained factor, then each factor contains a portion of the total variance that is reproduceable by a linear combination of the items factored.

You MAY be able to identify (stick a name on) each of the factors you retain by looking at the sign of the weights that make up the loadings; doesn't always work out, though.

Using a rotation may help clarify what each factor represents...

Are you trying to interpret the factor loadings (the weight of each factor on the observed variables) or the factor scores (the weights of each observed variable in producing a score representing the factor)? These are two different things. The first are used to interpret what the factor is by judging the relative sizes of the loadings: high loadings suggest stronger factor contributions to those variables. Loadings of both positive and negative signs are possible, but having both positive and negative loadings that are large is rare and suggests problematic factors.

Factor scores on the other hand are composites of the variables that are used to make the latent factor into an observed variable; to give it a scale. You would use factor scores, for example, when the factor was of interest to use as a predictor or outcome in a regression analysis and you were not planning to use latent variable modeling methods.

Many different factor score computation methods result in factor scores with mean = 0 and SD = 1. Various statistical packages have an assortment of these methods for computing factor scores, but they all produce very highly correlated alternatives. Even a unit weighted factor score method (for all variables with loadings > .3 weight = 1, else weight = 0) produces factor scores very highly correlated with other methods. There's a good paper by Grice on factor scores (see his helpful website, pdf of the paper is there too: http://psychology.okstate.edu/faculty/jgrice/factorscores/).

You must start from Factor Loadings: they can be interpreted as the correlation coefficients between the original variables and components, given you know the meaning of the variables you have used , you will look at the variables highly loaded (as absolute value) on the components (in this case the loading can only vary betweem -1 and 1 so the interpretation is easy) and give a name to the components depending on which variables are more loaded on the components, like here:

http://pubs.acs.org/doi/abs/10.1021/ci2005127

Clearly you MUST KNOW THE SENSE OF WHAT YOU HAVE MEASURED, BUT I GIVE FOR GRANTED A SCIENTIST KNOWS WHAT HE/SHE IS DOING AND THIS IS AN ESEGESIS WORK CRUCIAL FOR YOUR WORK.

Having given a name to your components you turn to the scores and, depending on the system at hand you will see that component1 is significantly related to an external variable Y (let's say a drug treatment, or a continuous variable as age, or location or any other thing you are interested into..by the way if you performed a study you wanted to prove some hypothesis on the world outside). Component scores are like any other variable and you ca correlate them with any feature of interest of your statistical units.

In some other cases it is more convenient to start from factor scored, like in microarray analysis, where the genes are statistical units, so for example you find that loadings are able to separate cancer from healthy subject (interperetation y loadings), for example the loadings on factor2, well, now you turn to the scores and order them for their absolute value, and look at which genes have higher scores on the discriminating component, well you are a biologist, or you work with a biologist, he will look at these 'extreme relevant genes' and will say something about biology, like here:

http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0013668

http://www.febsletters.org/article/S0014-5793(01)02973-8/abstract

or even here:

http://www.biomedcentral.com/1471-2105/7/194/abstract/

Each item that belongs to a factor, must have a factor score greater than .400 in factor in where they belong. The remaining factor scores should not be significant (p

Thank you Mohammad Tahir, Janavi Kumar and Alessandro Giuliani for the link sent. As I am do not have statistic background, it is quite hard for me to understand the information in 1st link while 2nd link consists of steps in conducting factor analysis.

Thank you Bernardo Chaves, Michael Weaver, Scot McNary and Reynaldo Rocha-Chávez for your suggestion.

Checking my understanding on factor analysis, there are two type of outputs - factor loadings and factor scores. Since we are approaching our research question through an exploratory perspective or trying to build a model, different combinations and methods with different number of factor cut-off level are used before settle on a 'suitable' configuration. Then, rotated factor loadings are used to interpret the factors obtained before giving it a name/label. As read somewhere, factor scores can be treated as any other variable for further investigation. Hence, we are using factor scores to identify scales for the named/labeled factor (e.g. to obtain something like the scale use in questionnaire; strongly agree, agree, neutral, disagree, strongly disagree). Am I on the right track?

Dear Phei-Chin I suggest you not to rotate factor loadings, by the way is perfectly natural that the same variable is 'loaded' on different components, this simply means that the particular measure (variable) is influenced by more than one 'hidden variable'.

On the contrary if you rotate you completely loose the physical meaning of components aand this is very detrimental to your analysis.

Dear Phei-Chin. On the contrary, as I understand, the rotated solution is the only one that can be interpreted in terms of the conceptual content of the measure. My comment above refers to the rotated solution. As part of the outputs of an exploratory factor analysis, can obtain Z scores of each factor in the rotated solution. Those scores are measured on the corresponding factor, responsive to the conceptual content determined by the set of associated items factor. But, I do not think you can say that obtain something like the scale use in questionnaire; strongly agree, agree, neutral, disagree, strongly disagree.

Dear Reynaldo,

this is a very interesting point, those who work in psychology or sociology desire factor analysis confirm their previous statements (this is the reason for rotation) 'conceptual content' is given , statistics is useful to obtain scales usable reflecting conceptual content.

Who works in natural science (physics, chemistry) does exactly the opposite, the components (by the way this name come from analytical chemistry indicating the components of a mixture) give rise to the correlation structure of the variables (e.g. peaks of a spectrum) thus the same varible can be influenced by different components like in the composition of forces in physics, or different molecules sharing the same peak of adsorbance (among many other they do not share).

I think this should be the right attitude in psychology too (see tha attached paper) because if we are not sure of the stability across different data sets of our constructs in physics...can you imagine in psychology ????

But again this is a complex and intermingled question....

The scores in EFA are Pearson coefficient correlations between observed variables and "invisible" variables named "factors". For example "Intelligence" is measured by many tests. One factor can be "memory", second "calculation", third "association", fourth "3D imagination" and others. After VARIMAX rotation we obtain the simple stucture: high score values with one observed (input) variable and week with all another factors.

Weak

Dear Alessandro Giuliani:

I think that this difference between social sciences and natural sciences comes of the difference between of type of objects of study. In analytics chemistry there is a implicit supposed, the nature of study's objects is not dependent of the phisycal place where ones are. But, in social science, the stability of the one factorial structure depents of a one specific culture. So, the technique in social sciences is to repeat one study until to discover of the cultural limits of those factorial structure.

For other hand, there is an clarifier example. Take in eche row diferents mesures of one specific cylinder. Each row, maybe can be selectec by random proces. In the columns one can put: height; area of the base, lateral area, base radius, and other. The solución without rotation, could not be interpreted for other hand, the rotate solution could be easy interpreted in terms of the diferences of the mesuares' nature: length, area; mesuares of the base, height of the cylinder.

If you have many variables that are being used in the exploratory factor analysis, I would agree that the rotated solution is the one you should be looking at because they are no longer correlated. The convention is that you are looking for scores (positive or negative) >=0.7. However, at first in your 'exploration' you might find some variables that have a low score in all your rotated dimensions. My advice then would be to iterate the factor analysis leaving out these variables (as they very little) until each of your remaining variables has a high score in only one dimesion. This then becomes much easier to interpret what the dimensions mean.

Dear Allan Brimicombe,

I agree with your view point.

Thank you Reynaldo.

I always understood that the positive and negative signs in the rotated solution reflected the positive and negative correlations in the correlation matrix.....I've tried to find a reference to this but couldn't run one to ground....so I could be wrong.

Dear Reynaldo,

you say 'For other hand, there is an clarifier example. Take in eche row diferents mesures of one specific cylinder. Each row, maybe can be selectec by random proces. In the columns one can put: height; area of the base, lateral area, base radius, and other. The solución without rotation, could not be interpreted for other hand, the rotate solution could be easy interpreted in terms of the diferences of the mesuares' nature: length, area; mesuares of the base, height of the cylinder.'

The point is exactly what you are saying, in natural sciences we are interested in the cylinder orientation and specific shape , given the nature of the measures is alredy known and thus not interesting at all, in other words, the measures are simply probes for the reality and not the other way around, as a matter of fact any scientific paper starts with the description of the measures (already known) and ends up into the specific structure of correlation they aquire when interact wit a 'piece of matter' (whatever this piece of matter is, a protein, a set of rats, a population..) given the aim is to increase knowledge about the piece of matter (i.e. the real thing), like here, where a set of well known and clearly defined measure are made to interact with a set of proteins so to make their correlation structure to highlight the specific structural principles of the proteins (not the nature of the measures we know already), in this case is mandatory to use a physically motivate solution like unrotated PCA.

Thank you for this exchange of ideas.

Alessandro

This is the publication i referred to

Factor scores are useful to rank and prioritise the factors. the, you can suggest the factors which are most important and need more focus. Recommendation should also be based on these rankings.

Just on a practical basis I have found suppressing variable loadings on each factor

You really want to be working with rotated factor scores to interpret them. Varimax rotation is most commonly used. This ensures that the correlation between factors (also referred to as components or dimensions) is minimised. {Don't forget, use the scree plot to limit the number of dimensions you are working with to what is absolutely necessary.} Then a rule of thumb is that any variable scoring 0.7 or more in a dimension is a key one for interpreting what that dimension means. In fact in exploratory factor analysis you would progressively weed out variables which do not have a rotated score of 0.7 or more in any dimension. The end result is often that each variable only contributes strongly (0.7 or more) to one dimension and the interpretation of each dimension then becomes much easier.

Once you get to know that some components describe most of the factor in which they are, what are we supposed to do next? Do we remove the describing components and run the analysis again? And then, when we see that sufficient factor analysis is being carried out, what is the next step?

In response to Ashwini's question of what next...

If as I have suggested, for an exploratory analysis you have weeded out all variables that do not contribute strongly (>=0.7 rorated score) in any dimension, you are usually left (in my experience) with each variable contributing strongly to only one dimension and therefore you are better able to unambiguousky describe what each dimension is. The purpose of the factor analysis is usally to follow through to multiple linear regression (and therefore you shouldn't include the dependent variable in the factor analysis). You can do one or more of three things:

However, remember, just because you find n dimensions from your collection of variables, it doesn't mean they will all significantly contribute in a multiple linear regression model. You may still need to weed out some of the dimensions in building a good regression model.

Negative factor loadings can be caused by negatively worded items. Recode them in reverse order.  If this is not the case, i.e. no negatively worded item, then discard the items with a negative factor loadings

Where negative factor loadings occur, it indicates that the variable has a negative correlation with the dimension, similar to a negative coefficient in a regression model. Therefore I wouldn't suggest discarding all variables with negative factor loadings without first studying what part they may play in any model that is being built.

Yes, Allan. Discarding depends, on the relationships among the factors. I have done EFA of job attitudes including job satisfaction, organisational commitment and turnover intentions. And consistent with theory, whereas factor loadings on commitment and job satisfaction are positive, those on turnover intentions are negative. The negativity is explained by theory, so I can't discard items measuring TOI.

Dear all, can I ask a question inside a question? 

 the results of factor analysis, the number that I've as result, has a units? That is, if I want to investigate the effects of the,perturbed and volume on pressure the results has the units of pressure? Thanks a lot and sorry if this Is a silly question, I'm not an expert on statistics,but I'm courious.

Dear Mariano it depends on the kind of algorithm you use if you

Extract the eigenvectorsfrom the correlation matrix that is the

Obliged choice when dealingvwith variables with different mrasurement

Units factor scores are adimensional z scores but if youvhave variables

With the same mrasurement units you can extract eigenvectors from covariance

Masix so keeping the same mrasurement unit.

Dear Alessandro Giuliani,

thanks a lot.

Grazie tante e complimenti per la enorme competenza!

mariano.

Troppo gentile

Ciao

Alessandro

Dear Augusto,

probably the attached file could be of interest to you

Hello everybody

I have conducted a factor analysis to extract latent factors leading to gender inequality at workplace. I extracted 5 factors from 20 variables. Now i want to rank all the variables showing their contribution in gender inequality. I have conducted a factor score ranking of variables, is this a correct method to fulfill my objective of ranking?

Please suggest me ways to do the ranking.

hi i got three items which are loading negatively. I have done all reverse coding. Please tell me the solution.

Dear ladies and gentlemen

My problem with my work is how to interpret factors extracted through an exploratory factor analysis from physics tests items. I am unable to find what is the idea behind the set of items in each factor. Is there any way to do this? Or will have to be even through sensitivity?

Do any of you know any test or any tips that might be useful for me?

Augusto

Dear Hassan Bhatti, you can find information on the Internet on the importance/interpretation of negative factor loadings. Do bear in mind that an item with factor loading of -.8 is as important as an item with loading of +.8. THe negative sign indicates high score on factor with low score on the item and vice versa.

The most popular answer above, and the question itself, confuse two terms: factor loadings and factor scores. Factor loadings are the correlation between each item and each factor. Factor scores are a variable calculated for each factor as weighted sum of each item. You interpret factor loadings just as you would interpret (Pearson) correlations.

Dear Dr. Lim,

Thank you very much for your question!

Best wishes,

Jafra Thomas

Ph.D. Student, Kinesiology

Dear Dr. Navarro,

Thank you very much for your succinct and clear explanations. I was not sure if a negative coefficient represented a methodological error. Your overview will be helpful in my colleagues' and my interpretation of our study results.

Best wishes,

Jafra Thomas

Ph.D. Student, Kinesiology

As I understood from the above criticism that the highest factor score value should be selected as the one that belong to link as shown in the attached image. I am trying to figure the trend of my data as per the factor analysis.

How can I know if a each one factor scores is excellent, moderate, or low?. For example, I can said that 8.068 is an excellent score, and -0.7929 is a low score...but i need a rule for said that. Do you know any rule or intervals for this?