Dear Ahmand,

check this paper, it describes the conceptual differences between pca and factor analysis nicely. Basicly it is based on the made assumption. While factor analysis is based in classical test theory it tries to differentiate true score variance and variance due to measurement error, no differentiation between this both aspects of a measured score is made in pca. In pca you implicit assume perfect reliable indicators. 

Great paper:

http://ww.w.statpower.net/Content/312/Handout/Fabrigar1999.pdf

Best

Hi Ahmad,

In addition to the above LDA is a supervised method. Which means that - in contrast to the PCA for example - you predefine groups of objects. In LDA we search for latent variables that describe the difference between the samples so, that the variance within the group is minimal while the variance between the groups is maximal. 

In unsupervised methods you can reveal patterns that are "naturally" hidden in your dataset without prejudication.

Br,

János

Hi Ahmad,

The three techniques are very similar in that they fall within the so-called generalized linear model. I suggest you consult the book that deals with this subject.

http://www.francoangeli.it/Ricerca/Scheda_libro.aspx?ID=21599&Tipo=Libro&titolo=Factor+analysis+and+principal+component+analysis++

Hi !

Although these three methods have in common that they can reduce dimensionality in your data, they are used in very different cases.

LDA =>  This method identifies components (i.e., linear combination of the observed variables) that maximize class separation (i.e. between-class variance) when such prior information is available (i.e., supervised). E.g., you have a training set containing a variable specifying the class of each observation.

PCA => Aims to find components that account for maximum variance in the data (including error and within-variable variance). Unlike LDA, it does not  take into account class membership (i.e., unsupervised), and is used when such information is not available. Importantly, both LDA and PCA do not require any prior notion of how the variables are related among themselves, and the resulting components can not be interpreted in terms of an underlying construct.

FA => Tries to uncover latent factors that account for the variance shared between the observed variables (thus excluding error and within-variable variance). Ideally, the resulting latent factors represent interpretable underlying constructs. FA should be used when you assume that an underlying causal model induces covariance between several observed variables. Consequently, unlike PCA and LDA, the observed variables are linear combinations of the estimated latent factors.

Cheers,

Kim

How LDA work in R with visualizing?

Read these tutorial papers, so helpful

Article Linear discriminant analysis: A detailed tutorial

Article Principal component analysis - a tutorial