It is obvious that discrimant analysis is a discrimination analysis. However, factor analysis also makes a sort of separation process by gathering expressions under groups. In this aspect, what distinguishes discrimant analysis from factor analysis.
https://www.statisticssolutions.com/discriminant-analysis/
https://en.wikipedia.org/wiki/Linear_discriminant_analysis
https://www.sciencedirect.com/topics/medicine-and-dentistry/discriminant-analysis
Hi Dear Nasser Said Gomaa Abdelrasheed
Your interest in my research and questions made me very happy. So thank you very much. Also, the resources you recommend are very useful. However, factor analysis and discriminate analysis do a grouping operation in the dataset. So the question I was wondering is, can a researcher doing factor analysis perform discrimanat analysis? or "what does he intend to do?"
Best Regards..
Discriminant analysis is a versatile statistical method often used by market researchers to classify observations into two or more groups or categories. In other words, discriminant analysis is used to assign objects to one group among a number of known groups.
Discriminatory analysis is a branch of computational mathematics that represents a set of statistical analysis methods for solving pattern recognition problems, which is used to decide which variables separate (ie, “discriminate”) emerging data sets (so-called “groups”). In contrast to cluster analysis, the groups in discriminant analysis are known a priori.
Discriminant analysis is used to decide which variables distinguish (discriminate) two or more emerging populations (groups). For example, an educational researcher might want to investigate which variables place a high school graduate in one of three categories: (1) going to college, (2) going to vocational school, or (3) dropping out of further education or training. For this purpose, the researcher can collect data on various variables associated with the students in the school. After graduation, most students should naturally fall into one of these categories. Discriminant Analysis can then be used to determine which variables give the best prediction of students' future paths.
The clinician may record various variables related to the patient's condition to find out which variables better predict that the patient is likely to have recovered completely (group 1), partially (group 2), or not at all (group 3). A biologist can record different characteristics of similar types (groups) of colors in order to then analyze the discriminant function that best separates the types or groups.
Computational approach
Computationally, discriminant analysis is very similar to analysis of variance (see ANOVA section). Consider the following simple example. Suppose you are measuring height on a random sample of 50 men and 50 women. Women are on average not as tall as men, and this difference should be reflected for each group of averages (for the variable Height). Therefore, the variable Height allows you to discriminate between men and women better than, for example, the probability expressed by the following words: "If the person is large, then it is most likely a man, and if small, then it is probably a woman."
You can generalize all of this to less "trivial" groups and variables. For example, suppose you have two populations of high school graduates — those who chose to go to college and those who are not. You can collect data on student intent to go to college a year before graduation. If the means for the two populations (those currently going to continue their education and those who refuse) are different, then you can say that the intent to go to college, as established in the year before graduation, allows you to divide students into those who is going to and who is not going to go to college (and this information can be used by school board members to appropriately guide the respective students).
In conclusion, note that the main idea of discriminant analysis is to determine whether populations differ in the mean of any variable (or a linear combination of variables), and then use this variable to predict for new members their belonging to one group or another. ...
Analysis of variance. The discriminant function problem posed in this way can be rephrased as a one-way analysis of variance (ANOVA) problem. One might ask, in particular, whether two or more populations are significantly different from one another in the mean of any particular variable. You can read the ANOVA section to explore how you can test the statistical significance of differences in mean between populations. However, it should be clear that if the mean of a particular variable is significantly different for the two populations, then you can say that the variable separates the populations.
In the case of one variable, the final test for the significance of whether the variable separates two populations or not is the F-test. As described in the Basic Concepts of Statistics and ANOVA sections, the F statistic is essentially computed as the ratio of the intergroup variance to the combined intragroup variance. If the between-group variance is significantly greater, then this should mean a difference between the means.
Multidimensional variables. When applying discriminant analysis, there are usually several variables, and the challenge is to establish which variables contribute to discrimination between populations. In this case, you have a matrix of common variances and covariance, as well as matrices of in-group variances and covariance. You can compare the two matrices using a multivariate F test to determine if there are significant differences between groups (in terms of all variables). This procedure is identical to the Multivariate Analysis of Variance (MANOVA) procedure. As in MANOVA, you can first perform a multivariate test, and then, if statistically significant, see which of the variables have significantly different means for each of the populations. Therefore, although the calculations for several variables are more complex, the basic rule applies that if you discriminate between populations, there should be a noticeable difference between the means.
Stepwise discriminant analysis
Probably the most common application of discriminant analysis is to include multiple variables in a study to determine those that best separate populations among themselves. For example, an educational researcher interested in predicting the choices that high school graduates will make regarding their further education, in order to obtain the most accurate predictions, register as many student parameters as possible, for example, motivation, academic performance, etc.
Model. In other words, you want to build a “model” that best predicts which population a particular sample will belong to. In the following discussion, the term “in the model” will be used to refer to variables used in predicting population membership; about variables unused for this we will say that they are "outside the model".
Step-by-step analysis with inclusion. In the step-by-step analysis of discriminant functions, the discrimination model is built in steps. More precisely, at each step, all the variables are examined and the one that makes the greatest contribution to the difference between the populations is found. This variable must be included in the model at this step, and the transition to the next step takes place.
Step-by-step analysis with exclusion. You can also move in the opposite direction, in which case all the variables will be first included in the model, and then at each step the variables that make a small contribution to the predictions will be eliminated. Then, as a result of a successful analysis, it is possible to save only "important" variables in the model, that is, those variables whose contribution to discrimination is greater than others.
F to include, F to exclude. This step-by-step procedure is "guided" by an appropriate F value for inclusion and a corresponding F value for exclusion. The F value of a statistic for a variable indicates its statistical significance when discriminating between populations, that is, it is a measure of the variable's contribution to predicting population membership. If you are familiar with stepwise multiple regression, then you can interpret the F value for inclusion / exclusion in the same sense as for stepwise regression.
Calculation on a case. The stepwise discriminant analysis is based on the use of a statistical significance level. Therefore, by their very nature, step-by-step procedures count on case, as they “carefully iterate over” the variables that must be included in the model to obtain maximum discrimination. When using the stepwise method, the researcher should be aware that the significance level used does not reflect the true value of alpha, that is, the probability of erroneous rejection of the hypothesis H0 (the null hypothesis that there is no difference between populations).
Interpreting the discrimination function for two groups
For two groups, discriminant analysis can also be viewed as a multiple regression procedure (and similar to it) - (see section Multiple regression; discriminant analysis for two groups is also called Fisher's Linear Discriminant Analysis after Fisher (1936). (From a computational point of view) all of these approaches are similar.) If you code the two groups as 1 and 2, and then use these variables as dependent variables in multiple regression, you will get results similar to what you would get with Discriminant Analysis. of aggregates, you fit a linear equation of the following type:
Group = a + b1 * x1 + b2 * x2 + ... + bm * xm
where a is constant and b1 ... bm are the regression coefficients. Interpretation of the results of a problem with two sets closely follows the logic of applying multiple regression: the variables with the largest regression coefficients contribute the most to discrimination.
Discriminant functions for multiple groups
If there are more than two groups, then more than one discriminant function can be evaluated in the same way as previously done. For example, when there are three populations, you can evaluate: (1) a function to discriminate between populations 1 and populations 2 and 3 taken together, and (2) another function to discriminate between populations 2 and populations 3. For example, you can have one function that discriminates between those high school graduates who go to college versus those who do not (but want to get a job or go to college), and a second function to discriminate between those graduates who want to get a job against those who who wants to go to school. The coefficients b in these discriminating functions can be interpreted in the same way as before.
Canonical analysis. When doing a discriminant analysis of several groups, you do not have to specify how the groups should be combined to form different discriminatory functions. Instead, you can automatically determine some optimal combinations of variables so that the first function will discriminate best among all groups, the second function will be the second best, and so on. Moreover, the functions will be independent or orthogonal, that is, their contributions to the separation of populations will not overlap. From a computational point of view, the system you are doing canonical correlations analysis (see also the Canonical Correlation section) that will define consistent canonical roots and functions. The maximum number of functions will be equal to the number of populations minus one or the number of variables in the analysis, whichever is less.
Interpretation of discriminant functions. As stated earlier, you will get the b coefficients (and standardized beta coefficients) for each variable and for each discriminant (now also called canonical) function. They can also be interpreted in the usual way: the larger the standardized coefficient, the greater the contribution of the corresponding variable to population discrimination. (Note also that you can also interpret structural coefficients; see below.) However, these coefficients do not provide information about which populations the corresponding functions discriminate between. You can determine the nature of discrimination for each discriminant (canonical) function by looking at the means of the functions for all populations. You can also see how the two functions discriminate between groups by plotting the values that both discriminant functions take
In this example, Root1 seems to mainly discriminate between the Setosa group and the Virginic and Versicol grouping. Along the vertical axis (Root2), a slight displacement of the points of the Versicol group downward relative to the central line (0) is noticeable.
Factor structure matrix. Another way of determining which variables "mark" or define a particular discriminant function is to use a factor structure. Factor structure coefficients are correlations between variables in the model and the discriminating function. If you are familiar with factor analysis, you can think of these correlations as factor loadings of variables on each discriminant function.
Some authors agree that structure coefficients can be used to interpret the real "meaning" of the discriminating function. The explanations given by these authors are that: (1) - the structure of the coefficients is probably more stable and (2) - they allow to interpret the factors (discriminating functions) in the same way as in factor analysis. However, subsequent studies using the Monte Carlo method (Barsikowski and Stevens (1975); Huberty (1975)) have shown that the coefficients of the discriminant functions and the structure coefficients are almost equally unstable until the sample size is large enough ( for example, if the number of observations is 20 times the number of variables). It is important to remember that the coefficients of the discriminant function reflect the unique (partial) contribution of each variable to the individual discriminant function, while the structural coefficients reflect the simple correlation between variables and functions. If the discriminating function is to be given separate "meaningful" meanings (related to the interpretation of factors in factor analysis), then structural coefficients should be used (interpreted). If one wants to determine the contribution that each variable makes to the discriminant function, then the coefficients (weights) of the discriminant function are used.
The significance of the discriminant function. You can check the number of roots that add significantly to discrimination between populations. Only those that are considered statistically significant can be used for interpretation. The rest of the functions (roots) should be ignored.
The bottom line. So, when interpreting the discriminant function for several populations and several variables, they first want to check the significance of various functions and then use only significant functions. Then, for each significant function, you must consider the standardized beta coefficients for each variable. The larger the standardized beta coefficient, the larger is the relative intrinsic contribution of the variable to the discrimination performed by the corresponding discriminant function. In order to obtain individual "meaningful" values of the discriminating functions, one can also investigate the factor structure matrix with correlations between the variables and the discriminating function. Finally, you must look at the means for the significant discriminating functions in order to determine which functions discriminate between which populations.
In general, Discriminant Analysis is a very useful tool (1) - for finding variables that allow the observed objects to be assigned to one or more actually observed groups, (2) - for classifying observations into different groups.
Regards, Pushkin Sergey Viktorovich
Здравствуйте Сергей Викторович! Мы заочно с Вами знакомы и я буду рада, если сложится сотрудничество в плане обмена - Вы нам жужелиц вашего региона, а от нас обработка и совместные публикации.
Насчет дискриминантного. Очень хороший анализ, когда надо оценить различия между двумя выборками по комплексу признаков. Мы используем его уже лет 20.
Hello dear @Puşkin Sergey VİKTOROVİCH
I read your answer with great interest and care. Thank you so much for your time and responsiveness. However, as you mentioned, the discriminatory analysis is similar to the ANOVA and regression analysis. My question begins at this point. What different? In other words, how is the hypothesis sentence of discriminatory analysis?
Best wishes...
Discriminant analysis is a statistical technique used to classify observations into non-overlapping groups, based on scores on one or more quantitative predictor variables. For example, a doctor could perform a discriminant analysis to identify patients at high or low risk for stroke.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS