I have to compare the mean levels of a continuos variable y (ranging 1-20) subdividing my sample in two groups according to a dichotomous variables (i.e gender). Sample sizes of the two groups are unequal (m=20/f=80).
It happened that in the male (smaller) group, all subjects have y=1, while in the female group scores ranges along all possible scores, although the distribution is not normal and highly skewed towards the lower scores. As of these, the male subgroup has no variance in y while the female group shows it.
Considering these issues, I thought to use bootstrap on t-test to make a more reliable mean comparison between the two groups:
I first applied bootstrap, stratified for gender, with (welch) t-test.
I then tried a wild bootstrap approach with the same t-test. As a matter of fact, I can consider this also a special regression with a single dichotomous predictor, with an extreme hetehroscedasticy and non-normality of residuals and in regression with heteroscedasticy the wild bootstrap approach is usually recommended.
What relevantly differs between the two bootstrap strategies is that with a wild approach I’m bootstrapping residuals from one group to subjects of another group. Results are also very different: wild bootstrap provides much much smaller p/CI than the stratified bootstrap with welch correction.
My questions are:
1 is a boostrapped t-test valid in this situation? And if so,
2 what of the two bootstrap approach is the most correct in this situation?
3 in case I would add some covariates (i.e. ANCOVA with one dichotomous predictor and two continuos predictors), what is again the most correct bootstrap strategy?
Many thanks for your help!
Others who have answered your question are exactly correct! It reads like you are hunting for some statistical test to determine whether your groups are different. They are obviously different, and you do not apparently need to rely on any statistical inference to make that determination. It is a fact.
The question is to figure out what to do next. I think that simple description and comparison may be the bet way to lead you to understand what the data is telling you. Don't get bogged down in inferential statistics, stay close to the data.
You can describe several things about each sample and try to figure out why they are different (or nearly the same). This is exploration and hypothesis generation rather than hypothesis confirmation. Look at the following:
Location: What is the central tendency (means, medians, modes, tri-means, trimmed means, etc.)
Spread: How spread out are the distributions. This gets at how well can the distribution be summarized by a single measure of location. You may consider the range, mid-spread, variance, proportion, proportion of the modal category, etc.
Shape: What are the shapes of the distributions? Are they symmetric, normal, skewed, j-shaped, asymptotically exponential, rectangular, etc. What might cause them to be different than what you observe? What underlying process might be responsible for their shape?
Peakedness: Are the distributions platykurtic (flat like a plate), leptokurtic (highly peaked in the middle? Do they have thin or thick tails in the distributions?
Unusual Observations: Are there any observations that are unexpected? Are they extreme, but legitimate observations or are they outliers that were sampled from some population other than the one under study?
Above all, please plot your data.
You should not calculate means and variances of scores. It rarely makes sense.
So you want a test? What is your substantive alternative? How do you judge wins and losses from a correct or wrong decision? Was the experiment designed to achieve a suitable power?
Dear Massimilanio
It is a little unclear to me what exactly you are trying to test. If I have correctly understood, you wish to compare the y scores between males and females, where they have clearly different distributions and the male distribution has no variance.
If this is the case, the bootstrapped 't' will not give a correct "p" value in either version.
Assuming that there is some deterministic reason why males always have y = 1, and that it is not simply a question of your sample being incredibly unlucky, no simple frequentist approach will make sense. So permutations that assume that gender is interchangeable will estimate a variability that does not exist. Adding covariates will not improve the situation with a poorly estimated variance!
I would normally write my results with the clear result that the samples differ completely in their distributions. Most people do not need an inferential statistic when the result is so clear! If a reviewer insists, then I would a) convert the results to a table (y = 1 vs y greater than 1 by gender) and offer a chi-squared with 0 inflation, b) offer a KS test on the distributions c) calculate CI on mean for females (bootstrap is ok) and show that male result falls outside this CI.
Good luck.
Others who have answered your question are exactly correct! It reads like you are hunting for some statistical test to determine whether your groups are different. They are obviously different, and you do not apparently need to rely on any statistical inference to make that determination. It is a fact.
The question is to figure out what to do next. I think that simple description and comparison may be the bet way to lead you to understand what the data is telling you. Don't get bogged down in inferential statistics, stay close to the data.
You can describe several things about each sample and try to figure out why they are different (or nearly the same). This is exploration and hypothesis generation rather than hypothesis confirmation. Look at the following:
Location: What is the central tendency (means, medians, modes, tri-means, trimmed means, etc.)
Spread: How spread out are the distributions. This gets at how well can the distribution be summarized by a single measure of location. You may consider the range, mid-spread, variance, proportion, proportion of the modal category, etc.
Shape: What are the shapes of the distributions? Are they symmetric, normal, skewed, j-shaped, asymptotically exponential, rectangular, etc. What might cause them to be different than what you observe? What underlying process might be responsible for their shape?
Peakedness: Are the distributions platykurtic (flat like a plate), leptokurtic (highly peaked in the middle? Do they have thin or thick tails in the distributions?
Unusual Observations: Are there any observations that are unexpected? Are they extreme, but legitimate observations or are they outliers that were sampled from some population other than the one under study?
Above all, please plot your data.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling