Could anyone please tell me about the main disadvantages of linear discriminant analysis (LDA)?
In other words, why we find other methods, such as PDA, used while the LDA gives better results?
Thank you in advance.
Dear Amer,
Advantages and disadvantages of linear discriminant analysis are dependent on your research question:
Article The Comparison of Five Discriminant Methods
Dear Amer,
Advantages and disadvantages of linear discriminant analysis are dependent on your research question:
Article The Comparison of Five Discriminant Methods
What do you mean by PDA? (I see a couple different possibilities when I Google that term.)
One disadvantage of discriminant function analysis compared to logistic regression is that the former can generate predicted probabilities outside the range 0-1.
HTH.
Section 4.5 of http://www-bcf.usc.edu/~gareth/ISL/ (free to download) compares LDA with some competitors (QDA, logistics regression, K nearest neighbors). Later in the book support vector machines are discussed, which are also used as an alternative in some circumstances. The ESL book covers these (and more) in more detail at http://statweb.stanford.edu/~tibs/ElemStatLearn/.
LDA (Linear Discriminant Analysis) and QDA (Quadratic Discriminant Analysis) are expected to work well if the class conditional densities of clusters are approximately normal.
For situations where we have small samples and many variables, LDA is largely preferred.
Conversely, because of the greater number of parameters to be estimated, classification rules based on QDA require generally larger samples than those based on LDA.
Hence, QDA seems to be appropriate only when the ratio between the sample size (N) and the variable dimension (p) is large, However, in such situations, nonparametric classification techniques are generally more appropriate according (Lachenbruch, 1975) and (Breiman et al., 1983).
According (Friedman, 1989), the regularized discriminant analysis (RDA) increases the power of discriminant analysis for ill-posed problems (i.e. when the class sizes are lesser than the dimension of variables (Nk
There are many problems of the discriminant analysis. I solved these problems by new methods and many examinations. Download my papers.
Dear Amer
The discriminant analysis is the big theme for you because there are many problems.
Good luck!
Dear Amer,
although Richard in his paper [R.G. Brereton / Chemometrics and Intelligent Laboratory Systems 149 (2015) 90–96] wrote that "there is no real statistical rationale as to why LDA should be favoured over for example QDA" I think there are some.
The initial assumptions (Gaussian distribution for each class, linear boundary) are the typical situations. Many new methods are tested on constructed examples: ‘class in calls’ situations where only circular boundaries are acceptable, though the most practical situations are far from being so.
If we do not know for sure the distribution of future samples we are better offusing simpler methods as LDA; i.e. even if QDA or (preferable RDA) IS BETTER FOR THE TRAINING SET, THERE IS NO GUARANTEE THAT IT WILL WORK BETTER FOR THE TEST SET.
The limitation that all classes should have the same variance can be by-passed easily, if defining different confidence intervals for various classes.
Independently form the above arguments the most frequently used supervised pattern recognition method is the linear discriminant analysis (canonical correlation or canonical variate analysis). In fact, SIMCA is also frequently used, but increasing evidences suggest its inferiority.
However LDA has serious disadvantages:
i) LDA does not work well if the design is not balanced (i.e. the number of objects in various classes are (highly) different).
ii) The LDA is sensitive to overfit and validation of LDA models is at least problematic. (However other methods as RDA, ANN, SVM etc. are even worse)
iii) LDA is not applicable (inferior) for non-linear problems (separation of orange- banana shape point clouds, class in class situations)
Prof. Shinmura thinks he solved the problems of discriminant analysis. It may well be true on existing data, on the training set, but not for future samples and by no means for generally, or for each individual situations.
It can easily be shown which technique is the best one using the fair method comparison technique termed as sum of ranking differences (SRD), as described in ref. [K. Heberger, Sum of ranking differences compares methods or models fairly TRAC - Trends in Analytical Chemistry, 29 (2010) 101-109], links to a downloadable program can be found in ref. [K. Kollar-Hunek and K. Heberger, Method and Model Comparison by Sum of Ranking differences in Cases of Repeated Observations (Ties), Chemometrics and Intelligent Laboratory Systems, 127 (2013) 139-146.].
Dear Karoly
I agree your claim in many points.
I developed the "k-fold cross validation for small sample" method.
I can validate eight LDFs by the mean of error rates in the validation samples.
Why do you ignore my recent papers about the 95% CI of error rate and discriminant coefficient.
Dear Shuichi,
Many thanks for your answer. I have not read your papers. Please send me copies!
kindest regards, Karoly
Dear Karoly
After I knew RG about two years ago, I decided not to wite papers in Japanese and upload English papers about the discriminant analysis. So, you can download all recent full papers from RG. Because I published 14 papers about the microarray data, you fourcu on the 14th paper that is summary and conclusion. You download another recent papers within 3 years.
Two main problems: (1) when the discriminative information are not in the means of classes and (2) small sample size problem.
Here are the problems of LDA., and some suggested solutions for it.
Article Linear discriminant analysis: A detailed tutorial
- Badges
- Science topic