To conduct climate change detection, I will separate some climate data into 2 periods: the baseline and the current period. Say, climate data will grouped into the baseline period and the recent period. (Afterwards, I will test normality data and check equality of means).
My question:
Shall I employ Shapiro-Wilk (Kolmogorov-Smirnov) test for each data period ?
OR Shall I employ Shapiro test before I separate climate data ?
I think it depends on your objectives of analysis. If you include group factors in your regression and ANOVA analysis, I think it is necessary to test normality for each data period. Hope it helps!
Dear Muhamad
That depends on what do you want to do and the purpose of Shapiro-Wilk test. I assume you want to check on the normality of your data. SW test is a test of normality in frequentist statistics. Then, will allow to make such diagnosis, and then, proceed with other tests for bivariate and/or multivariate analyses which relies on the normality assumption, e.g. simple linear regression.
If you want later to compare two set of data, the same is considered, if establish a normal dsitribution, you may would like to test mean differences using e.g. a t-Student test.
Finally, please if you consider this answer appropriate, please upvote it using the green up arrow click. Thanks.
Hi Muhammad,
I assume that you will be comparing the distribution of the early period (base period) with the current period (?). If so, and if you will be using a t-test (parametric) or Mann-Whitney test (non-parametric), you should run the normality test on each time period rather than the entire period of record.
Best
Muhamad -
If your objective is "...to conduct climate change detection.." between two "...periods: the baseline and the current period," with continuous data, I presume, then it would appear that you would want to conduct all of your statistics treating those two groups separately.
As for Shapiro-Wilk, I don't know why normality would be important, and many practitioners and statisticians are now of the opinion that hypothesis testing in general is overrated and often misused, as indicated in a letter to the American Statistician I've attached. The problem is that an isolated p-value is a function of sample size, so the practice of using a given threshold - say 0.05 - over a range of applications is not really useful. You need a type II or power analysis, which can be complicated. If you can use a confidence interval, it is far less messy and more interpretable. Say you looked at the point estimate difference between two means for continuous data. You could put a confidence interval around that point estimate. You could see the best estimated confidence level at which zero is still not included in your interval. You don't have to assume normality. You could use other distributional functions to change the factors needed, though that could become more difficult. But what are you going to do anyway? You could transform the data, if that is what you had in mind, but I still recommend confidence intervals for continuous data. You could check out the Chebyshev inequality if needed and if helpful.
Graphical analysis would be a good way to go as well. If you used that to visualize your data, and obtained a point estimate of the difference in means and estimated a standard error for that point estimate, that could be very helpful/useful for interpreting your results.
Cheers - Jim
Article Practical Interpretation of Hypothesis Tests - letter to the...