Dear All: I am wondering about the best statistical test that can deal with large data sets. I am profiling plasma microRNAs using a high throughput method and comparing the expression of those miRs in a case-control study. I was wondering if simple t-test or Mann-Whitney test are suitable to deal with this large data set or we can use other more specific test?
many thanks in advance for your help
It depends what you're trying to test. A t-test doesn't really have any upper limit on the size of the dataset, but it's only useful if you're trying to compare two means of a continuous variable (which I assume is what you're intending to do, if miR expression is a continuous variable and you mean to compare the mean miR in cases to that in controls).
It depends what you're trying to test. A t-test doesn't really have any upper limit on the size of the dataset, but it's only useful if you're trying to compare two means of a continuous variable (which I assume is what you're intending to do, if miR expression is a continuous variable and you mean to compare the mean miR in cases to that in controls).
thanks a lot Stephen and Andrey for your help.
Andrey: my hypothesis is that plasma microRNA expression profile can differentiate between disease and control. this should involve profiling of more then 700 microRNAs in plasma to compare their level in health and disease.
Rana -
If you are just looking at a p-value, that can be very misleading. You need a type II error analysis, or some other sensitivity analysis to go with it. "Significance" is a badly chosen word, as happens in statistical (and other) jargon. -- For a really 'big data' set, you would likely get tiny p-values, no matter the test, for your effect size, unless you need to know a tiny effect.
When you can use a confidence interval, that is generally more practically interpretable, though it also is impacted by sample size. (Note that standard deviations for populations are basically fixed values to estimate, but the standard error of a parameter is a function of sample size. Confidence intervals and prediction intervals are changed in width or confidence/prediction level, depending upon that sample size, the test, and standard deviation. Thus, you can use that information to estimate sample size needs to obtain a reasonable "confidence.")
Note the following where the American Statistical Association finally commented on p-values:
Press release for the American Statistical Association:
http://www.amstat.org/newsroom/pressreleases/P-ValueStatement.pdf
Cheers - Jim
If your 'differentiations' consist of yes/no results, then you could estimate the proportion overall, and estimate a confidence interval around the estimated proportion.
- Badges
- Science topic
- Similar topics
- Biological Science
- Biochemistry
- Biomolecules
- Nucleic Acids
- RNA
- Small RNA
- microRNA