I want to ask about the usage of parametrical and non-parametrical tests if we have an enormous sample size.
Let me describe a case for discussion:
- I have two groups of samples of a continuous variable (let's say: Pulse Pressure, so the difference between systolic and diastolic pressure at a given time), let's say from a) healthy individuals (50 subjects) and b) patients with hypertension (also 50 subjects).
- there are approx. 1000 samples of the measured variable from each subject; thus, we have 50*1000 = 50000 samples for group a) and the same for group b).
My null hypothesis is: that there is no difference in distributions of the measured variable between analysed groups.
I calculated two different approaches, providing me with a p-value:
Option A:
- I took all samples from group a) and b) (so, 50000 samples vs 50000 samples),
- I checked the normality in both groups using the Shapiro-Wilk test; both distributions were not normal
- I used the Mann-Whitney test and found significant differences between distributions (p
@Mateusz Soliński. We can't say much about the significant difference between variables by just looking at two values. We have to consider some other factors. As an example, if we are considering the Z test, it depends on both mean and standard deviation (standard error). So, two central tendency values can be very close, but the test result can be shown as significant. Evan 6.05 and 6.10 can be significantly different from each other due to lower variance.
If consider the two approaches you mentioned, it has to be decided according to your situation. However, the nonparametric test is less powerful than a parametric test. Also, non-parametric tests are better, if there is no way to use parametric tests. On the other hand, for better generalization parametric tests are better. I prefer a parametric test but decide it according to your data.
Another thing is that it is better to represent the results of students' T-test with mean and standard deviation. Then your interpretation will be very easy.
You have 1000 replicate measurements from each subjects. These 1000 values are correlated and they should not be analyzed as if they were independent. So your model is wrong and you should identify a more sensible model. Eventually, the test of the difference between your groups should not have more than 98 degrees of freedom (it should have less, since a sensible model will surely include some other parameters than just the tow means). Having 1000 replicate measurements seems an overkill to me if there was no other aspect that should be considered in an analysis (like a change over time, with age, something like that). If there is nothing else that should be considered, the simplest analysis is to average the 1000 values per patient and do a t-test on 2x50 (averaged) values.
If you had a sample of independent thausands of samples per group, estimation would be mor interesting than testing. You should then better interpret the 95% confidence interval of the estimate (biological relevance) rather than the (in this respect silly) fact whether it is just in the positive or in the negative range.
A significant difference is not necessarily an important difference. With a large amount of data, it can amount to what G.K. Chesterton called a "tremendous trifle" (Chesterton himself was on the large side…)
This is where hypothesis testing once again isn't up to the job. You need to measure effect size.
Incidentally, if your hypertensives included isolated systolic hypertension, then by definition they will have a wider pulse pressure.
And beware : distributions of blood pressure in hypertensives are truncated by the very diagnostic process.
Is there an actual clinical question underlying this? What is it, if so? Just out of curiosity.
- If you have many samples per individual, you might consider a mixed-effects (or hierarchical) model to account for this stratification. ... Or, it may be reasonable to just use the average, or median, or some other summation value per subject.
- Yes, what test you choose should be based on the hypothesis you wish to test.
- Really, don't use a test for normality, like Shapiro-Wilks, to determine what test you use.... It can be helpful to plot the residuals from a model. (Which is simple for a t-test).
you could also analyze systolic or diastolic peaks. maybe you find better distribution or even significant differences
Here you start by saying "My null hypothesis is: that there is no difference in distributions of the measured variable between analysed groups." But it seems that you are only considering differences in location. Some form of probability plot may reveal something important. The apparent presence of outliers on only one side of the distribution is also of some concern.
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Publication