Hi,
I have a large data sample with 15k rows and I am trying to perform uni-variate normality analysis on the dataset. I have come to understand that normality tests are really sensitive towards large datasets and even slight abnormalities can result in the data being shown as "Not Normal". Going by the Null hypothesis that the alpha value has to be
The graph looks like normal to me but if you have doubts, try several other tests like K-S test and Jarque-Bera test. I prefer the latter as it is specific to normal distribution and it also use the skewness and kurtosis values in the test statistics.
You may want to read the summary of the Jarque-Bera test : Chapter Jarque-Bera Test
The problem you are having is that the hypotheses test is H0:unknown and H1:normal because the test stat(e.g. in K-S test is the max abs diff between the two cdfs)is such that small p values favor H0 in this case. so a small p-value implies normal. See a nonparametric stats book for details. Note that this agrees with your graph as it should. Best, David Booth
Ajit kumar Roy
- But the test indicates that the data is not normal? Is it because of the size of the population or that the test is really sensitive towards big data?- Badges
- Science topic