Up on checking my data (large sample size of 11,008) on histogram, it showed almost normally distributed, slightly skewed to the one side. But the Kolmogorov-Smirnov Test for Normality indicated that my data is not normally distributed. I have transformed my data but still end up with a non-distributed data.
My plan was to use an independent Sample t -test to compare the mean difference in two groups. But the independent Sample t -Test needs a normally distributed data as an assumption.
My question goes like, since the histogram almost indicate a normal curve and I am using a large data set shall I drop the assumption of normality and use the independent Sample t -Test or shall I opt for non-parametric tests?
With a sample size this large, I would not recommend a t-test. Since the standard error will be close to zero, even the slightest difference will be statistically significant. I'd recommend a different approach for testing these data - one centered around the effect size (which is independent of sample size).
Hello Girum,
The normality issue may be avoided altogether by choosing to use bootstrap/resampling solutions, which are available in a number of statistical packages (see links below). Do note that goodness of fit tests are sensitive to sample size (e.g., more power with large N, lower power with small N), just like most statistical tests.
Blaine Tomkins' reply is advising that, because statistical significance with large sample sizes is so easy to obtain (for non-nil differences), that you would be well advised to couch your interpretation more so on whether the observed effect size was large enough to be considered noteworthy.
Good luck with your work.
Bootstrapping example links:
1. SAS: https://blogs.sas.com/content/iml/2018/06/20/bootstrap-method-example-sas.html
2. SPSS: https://www.ibm.com/products/spss-bootstrapping
3. R: https://www.statology.org/bootstrapping-in-r/
4. stata: https://www.ssc.wisc.edu/sscc/pubs/4-27.htm
1-
If you want to convert the data to a normal distribution, this is done using SPSS software with Rank Cases and Compute Variable techniques.
Please see the link below. It is fully explained along with the video and educational text.
https://graphpad.ir/transform-data-to-normal-distribution/
It is important to note that based on the above techniques, the mean and standard deviation of the data remain constant and do not change. But the distribution of data becomes normal.
2-
Why do you want to do the average comparison test as follows?
H0 : MA = MB vs H1 : MA ≠ MB
I think it is better to think about testing the following hypothesis
H0 : PDFA = PDFB vs H1 : PDFA ≠ PDFB
(PDF = Probability Distribution Function)
You can easily test this hypothesis with many statistical software.
3-
You may think this is wrong.
But I personally use Kruskal-Wallis instead of the non-parametric Mann-Whitney test in cases where I have a large number of samples. Even if I want to compare two independent samples. The main reason for this is that the Kruskal-Wallis null hypothesis is not a comparison of means, but a distribution similar to samples A and B.
It's not clear as to how you applied the K-S test: did you apply it to variables or to differences? The key assumption here is that the difference should be normally distributed. But if this does not hold, you can use non-parametric tests (however, in most cases it will still be important to have a symmetric distribution of differences).
Blaine hit the nail on its head. Why should one use a hypothesis test on such a large sample? This is sufficient data to get a precise estimate of the mean difference. You should not be concerned whether or not the data are sufficient to interpret just the sign of the observed mean difference - you should ask if the observed mean difference (given its precision, which is likely very high) is large enough to be of any relevance. This requires an expert judgement, not a hypothesis test.
Hello Girum. There have been quite a number of replies to your question, but none of them, it seems to me, have directly answered your question about whether you can use an unpaired t-test. I'll try to do that. But first, I want to respond to some other things in your question. Italicized sections below are quotes from the original question.
Up on checking my data (large sample size of 11,008) on histogram, it showed almost normally distributed, slightly skewed to the one side.
I hope you were looking at a plot of residuals (i.e., deviations from the group means), not a plot of the dependent variable.
But the Kolmogorov-Smirnov Test for Normality indicated that my data is not normally distributed.
Testing for normality prior to doing a t-test (or some other parametric test) is both silly and pointless IMO.
- https://www.researchgate.net/publication/299497976_Silly_or_Pointless_Things_People_Do_When_Analyzing_Data_1_Testing_for_Normality_as_a_Precursor_to_a_t-test
I have transformed my data but still end up with a non-distributed data.
I think this practice is also ill-advised. Here's another short presentation. It's about OLS linear regression, but bear in mind that an unpaired t-test is equivalent to an OLS linear regression model with one dichotomous explanatory variable.
- https://www.researchgate.net/publication/320617363_Silly_or_pointless_things_people_do_when_analyzing_data_3_Transforming_variables_to_make_them_more_normal_prior_to_linear_regression_analysis
My plan was to use an independent Sample t -test to compare the mean difference in two groups. But the independent Sample t -Test needs a normally distributed data as an assumption.
As noted above, the unpaired t-test is an OLS linear regression model with one dichotomous explanatory variable. The normality assumption for OLS models is that the errors are independently and identically distributed as normal with mean = 0 and variance = sigma^2. But authors often fail to point out that normality of the errors is a sufficient condition, but not a necessary condition. The necessary condition is that the sampling distributions of the parameter estimates (i.e., the model coefficients) are approximately normal--see slides 9 and 10 in the attached PDF, for example. (I say approximately normal because I believe that George Box was correct when he said that the normal distribution does not exist in the real world--see the attached image.) With your sample size (11,008), it is a virtual certainty that the sampling distribution of mean for each of your groups will approximate very closely a normal distribution. And in that case, the sampling distribution of the difference between the means (i.e., of the regression coefficient for Group) will also be approximately normal.
So to answer your question, yes, you can use a t-test if your desire is to test the null hypothesis that mu1 - mu2 = 0 (or some other value dictated by your null hypothesis). Even if you only wish to obtain an estimate of the mean difference with its CI, the output from the t-test will give you that.
One thing you have not told us is the values of n1 and n2. I mention that because the greater the discrepancy between the two sample sizes, the more problematic variance heterogeneity becomes. So you might wish to use a form of t-test that is robust to variance heterogeneity (i.e., Welch or Satterthwaite t-test).
As others have noted, with such a large sample size, it is virtually certain that the mean difference will be statistically significant, and also that the CI for the mean difference will be very narrow. So as noted by others, you will have to use content expertise to determine whether the mean difference is large enough to be practically important or theoretically interesting.
HTH.
Blaine Tomkins , when you say centered around the effect, I assume you don't mean the observed effect (that would be silly). Can you clarify? Also, if the person really is interested in whether the actual effect is near zero, then why not just use an interval rather than a point null. Relevant references below. But in general, yeah, report effect sizes with intervals, use QQ plots to examine deviation from a distribution (not histograms), and realized "all assumptions are wrong" (Tukey, 1986, p. 72).
@article{MurphyMyors1999,
author = {Kevin R. Murphy and Brett Myors},
year = {1999},
title = {Testing the hypothesis that treatments have negligible effects: Minimum-effect tests in the general linear model},
journal = {Journal of Applied Psychology},
volume = {84},
pages = {234--248},
doi = {10.1037/0021-9010.84.2.234},
}
@article{SerlinLapsley1985,
author = {Ronald A. Serlin and Daniel K. Lapsley},
year = {1985},
title = {Rationality in psychological research: The good-enough principle},
journal = {American Psychologist},
volume = {40},
pages = {73--83},
}
Daniel Wright In hindsight, I wasn't very careful with my phrasing. I did not mean a test that literally centers on the effect size (i.e., a single value). Rather, as you suggest, an interval. "A test that measures the effect size and associated error of the estimate" would have been a better way to put it.
Thank you all for your help and brief… I found them very helpful. Just to brief my point a little more.
My intention was to see if there is a significant difference in birth size (birth weight of infants) who are born from mothers who live in households that use electricity and biomass fuel for cooking.
The objective is to see if there is a mean difference in birth weight in grams between the the biomass fuel and electricity users.
The K-S, Shapiro-Wilk’s test and a visual inspection of their histogram, normal Q-Q plots and box plots showed that the infants birth weight among the two fuel type users were not normally distributed with a skewness -0.879 (SE=0.025 ) and Kurtosis of 1.576 (SE=0.05) for the biomass fuel users and skewness -0.806 (SE=0.064 ) and Kurtosis of 1.766 (SE=0.128) for electricity users.
From my data I understood that I have to use A Mann-Whitney U test, but since A Mann-Whitney U test doesn’t exactly specify the how much gram difference and my primary aim was to check the exact significant difference in terms of grams, that is why I have brought the issue.
Once again, thank you all for your help.
Girum Gebremeskel Kanno , that won't be appropriate. You say that your "intension was to see if there is a significant difference in birth size." So you need to define what you mean by significant (e.g., a half gram difference). There are a few approaches that you could take, including the procedures I provided references for. If you have another meaning of "significant difference" please state what it is, and that may change the advise. This gets at the point raised in several of the comments, so these are worth reading. I think the simplest is report the CI of the difference in grams, and as David Morse says if you are worried about the assumptions (big sample, I wouldn't be that worried) use bootstrap methods to estimate this.
The t test does not assume the normal distribution of the population. It assumes the sample mean values are normally distributed. Since your study employed a large sample size, and in line with the Central Limit Theorem that large sample means tend to be normally distributed, the t test could still be valid, albeit non-normal population distribution. That being said, the matter is still debatable among scholars (e.g., Kim & Park, 2019; Kwak & Kim, 2017). Alternatively, attaining normality for non-normal data can be achieved using the Box-Cox transformation or power transformation (Asar et al., 2017; Verma & Abdel-Salam, 2019). Here are the full citations.
Asar, Ö., Ilk, O., & Dag, O. (2017). Estimating Box-Cox power transformation parameter via goodness-of-fit tests. Communications in Statistics - Simulation and Computation, 46(1), 91–105. https://doi.org/10.1080/03610918.2014.957839
Kim, T. K., & Park, J. H. (2019). More about the basic assumptions of t-test: Normality and sample size. Korean Journal of Anesthesiology, 72(4), 331–335. https://doi.org/10.4097/kja.d.18.00292
Kwak, S. G., & Kim, J. H. (2017). Central limit theorem: The cornerstone of modern statistics. Korean Journal of Anesthesiology, 70(2), 144. https://doi.org/10.4097/kjae.2017.70.2.144
Verma, J. P., & Abdel-Salam, A.-S. G. (2019). Testing statistical assumptions in research. Wiley. https://www.wiley.com/en-us/Testing+Statistical+Assumptions+in+Research-p-9781119528418
Good luck,
Girum Gebremeskel Kanno wrote:
The objective is to see if there is a mean difference in birth weight in grams between the the biomass fuel and electricity users.
Emphasis was added by me. Given that objective, and given the huge sample size, I would be very comfortable using an unpaired t-test. As I noted earlier the output from that test will include the mean difference with a CI.
Unless I missed it, you have still not told us the two sample sizes. If they are not close to the same, I suggest (again) that you use a form of t-test that is robust to variance heterogeneity (Welch or Satterthwaite). Can you tell us n, mean and SD for each of the samples? With that info, interested users can do their own t-test.
hello Girume, The Mann-Whitney nonparametric test can be considered a possible alternative to the parametric independent samples t-test when certain distributional assumptions (e.g., normality) are not met for that test.
If the data. is not normally distributed but the sample sizes are large, you can use an independent sample t-test. The t-test is valid for large samples from non-normal distributions