Ok. I know how to the extent of statistical inference, when null hypothesis is discarded (data supports discarding, under certain risk...) , some event not random. But my question relates more to the extension of some conclusion to the mayor population.
Lets say we want to infer some conclusion on a very unbalanced population. Lets say a study on Parkinson's D. (PD is estimated to affect 100 -180 people per 100,000 of the population). In order to perform some test, we usually balance the sample.
Isn't this a good example of Dewey Defeats Truman case? There will be A LOT of subjects selectively discarded because of being from the control group.
However if the real group proportions are kept in the sample, even a random predictor would yield very high negative predictive value (yes) and probably truly good estimators for real Specificity and Sensitivity.
Perhaps someone could point me out a good work on this issue.
Hi Juan,
Maybe you misunderstand the meaning of "identically distributed" here. It means that all elements of the sample are drawn from the same distribution, but it does not mean that this distribution is identical to some other distribution. So, in this case, the balanced distribution is the same for all sample elements, but it does not have to be the same as the unbalanced distribution (and in this case is indeed not).
If you then want to talk about testing, you would need to be more precise and describe in detail what test is performed and how, and what the conclusions are.
I see...
Test? This case Mann-Whitney.
I traced back original H.B.Mann-D.R.Whitney's work. It says how originally Willcoxon only treated the case of N1=N2. How the generalization goes for different sample sizes. 1947... great to read.
I was just thinking of possible selection bias occurring when there is significant uncertainty within group-labels. (my case: mild-cognitive-impairment).
When matching most common group's size, there might be some change in the sample distribution. If this was to be the control group, chances of discarding a true negatives are not the same as discarding false negatives.
Anyway thank you very much. I see now.
Do you know Propensity Score approach? You can assess unbalance and correct it using estratification, matching or weighting. See package twan in R and its documentation for more information.
The key point is that -apart from either having PD or not- the patients groups and the control group should have similar properties (age, sex, life-style, education, other diseases, ...). When we now compare two such samples, we will see as a result a combination of effects of all these factors on our response measure. However, since only PD is systematically different in these groups, the influences of the other factors will average out (at least stochastically).
Statistically speaking, both samples were taken from an imaginary population that resembles all the features except having PD. Thus, under the null hypothesis that PD does not effect the response, we have all the same expectations about the population distribution of values from both samples.
Keeping real proportions in the sample might be a bad advice, especially if it is known or suspected that some factors may be accosiated ("confounded"!) with the predictor or the response (in large studies such confounders can be considered in appropriate models). For instance, it is obviousely stupid to take a groups of female and male persons (1:1) to study the influence of smoking on getting cervix carcinoma. Even if the "treatment" group is constituted of (smoking) women, it is reasonable to use only (non-smoking) women in the control group (men will never get a cervix carcinoma anyway). Instead of comparing two samples from the population of humans (man+women) we compare two samples from a population of women. Often, the connections are not that obvious, and the correlation of factors with the response are the only hints we have. So we construct the most likely population from the factor combinations we obsever in the treatment group. And for this population we can assume the same distributional properties in both samples (given the experimental factor has no effect).
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis