If values of a specific feature for all samples in class A are statistically significant than values of the same feature of all samples in class B, would that give me any information or intuition regarding the outcome of classification? For example if you measure the blood glucose of 100 obese mice and in 100 normal mice and do ANOVA and find that the glucose level of obese mice is statistically significant than normal mice, would that give you any info about the output of classification?
Zeynab Mousavikhamene finding a statistically significant difference in means between groups may or may not help with a classification problem. It likely depends more on the distributional separation.
Take for example two data sets of 100 mice.
In the first data set there are two groups of mice: group "a" with a mean weight of 20 (g) and std. dev. of 2.5 (g), and group "b" with a mean weight of 30 (g) and again a std. dev. of 2.5 (g). See histogram attached.
In the second data set there are also two groups of mice: group "aa" with mean weight of 20 (g) and std. dev. of 5 (g), and group "bb" with a mean weight of 30 (g) and again a std. dev. of 5 (g). See histogram attached.
The t-test comparing the two groups within each data set (i.e., "a" vs. "b" and "aa" vs. "bb") shows a statistically significant difference. But when using weight to distinguish between groups, say with logistic regression, the separation in distributions in the first data set allows us more accuracy.
To demonstrate, I simulated these two hypothetical data sets and ran the tests. Below are the results.
First data set
t-test for a and b:
t = -22.452, df = 97.951, p-value < 2.2e-16
Logistic regression:
98% correctly classified
Second data set
data: aa and bb
t = -11.532, df = 97.951, p-value < 2.2e-16
Logistic regression:
87% correctly classified
*** R-code to reproduce these results ***
### Compare group means
set.seed(123)
a
Caleb A. Aldridge Awesome explanation! Thanks a lot.
Is there any other point that if I consider it along with statistical test can help me to have intuition if they will be separable with high accuracy or not? For example the second data set has higher std which showed lower accuracy.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA