Let say my students collected 70 samples from certain population. He found the data was not normally distributed using correct procedures. He tried to removes outliers, anyway the data still not normal. Should he used non-parametric test to analyses the data?
Alias, the analysis can be done in two ways. Like you have said, non-parametric tools can be employed depending on what you are checking in the data. The other alternative is for you to transform the data and check the normality. This is common in geochemical data where non normally distributions are either. Log transform or box Cox transformed depending on what makes the data non normally. I mean the type of skewness.
It is usually applied when you are checking null hypothesis for your data. Null hypothesis means your data are normally distributed. Critical values will tell you whether you need to reject null hypothesis or not. You can use Mann-Whitney U test if you have large dataset. Google it you will find tons of information on this test. For small dataset t-test and SigmaPlot can also be used.
do i have to consider the data are large enough (more than 30), so that data are assumed to be normally distributed, thus perform parametric test? is this assumption valid?
Dear Alias:
While non-parametric methods would work in this case, and transforming data would perhaps also work, non-parametric methods tend to have lower statistical power, and transforming data can also introduce other problems.
A better solution would probably be to use Generalized Linear Models (glms), using the R packages nlme or lme4. GLMs are sort of like ANOVAs, but they allow one to account for violations of assumptions like this, and they do it in a way that is much better than transforming and then using more traditional statistics or going the non-parametric route.
A good primer on nlme, including glms, is Mixed Effects Models and Extensions in Ecology with R: http://www.highstat.com/book2.htm
This book also talks about some of the reasons why transforming data might not be a good idea.
Best of luck!
- James
Dear Alies,
Parametric test always give you robust answer. If your data is not still non normal, you can convert your data using log transformation method / box cox transformation as said by Clement. You can also try square root transformation. Check your data homogeneity of variance using Levene's test. If both the cases your test statistics result is non significant, then go for Non parametric test. In non parametric test data were considered as rank based and loose most of the information of your data so before performing non parametric test think 100 times to make your data fit for performing parametric test.
@ Somidh Saha: there is no reason not to use « central tendancy parameters » if the data is not normal. Modeling them, testing them or building confidence intervals will be harder, but they can be interpretated unless you have multimodal distributions (which probably means you're missing a covariate...).
@ James Mickley: glm can not solve all kind of non-normality problems, they also have strong distributionnal assumptions...
@ Mriganka Shekhar Sarkar: no, parametric tests are not always robust, they can be very sensitive to some deviation to assumptions, for instance skewness...
I agree with Emmanuel, if the data distribution is not highly skewed and one has more than 30 (arbitrary cut-off used in the literature) observations, t-test should work. Also, note that the hypotheses being tested by non-parametric tests are not always the same as 'normal-theory based' tests. I would also like to add that transforming data to achieve normality (such as the log-transformation) also modifies the hypothesis being tested.
Thanks everyone sharing a valuable information for readers outside there.
- Badges
- Science topic
- Similar topics
- Vectorization