I am currently writing up findings for my masters dissertation and I have noticed that APA often require the t statistic, the p-value, degrees of freedom and sometimes the confidence intervals when reporting a t-test result. Bootstrap, however, only produces a p-value and confidence intervals. I can generate a quasi-t statistic with ease, however I am unsure of which degrees of freedom to use. I used 1000 samples in the bootstrap version, and 29 in the original. Any help would be kindly appreciated!
Strictly I'm not sure it has one. You use the bootstrap to simulate the sampling distribution of the parameter - is in this sense it is more like the z distribution. Personally I'd just report the parameter estimate, p and 95% CI. If one were to associate a df value it would be from the data (29, not the number of bootstrap samples which is just a proxy for how much precision you want in your bootrap).
Hi,
The great thing about bootstrapping is that you eliminate the need for degrees of freedom calculations. Instead of comparing your test statistic to some asymptotic reference distribution (which is in many cases highly inaccurate), you simulate an empirical distribution based on the data you have observed.
You seem to have mixed up some terminology, which may be why you pose the question here. Allow me :)
I'm assuming your original data set had a sample size of N = 29. Under the null-hypothesis, the t-statistic of a one-sample t-test would follow a t-distribution with N-1 = 28 degrees of freedom. That is, if you take an infinite number of samples of size N=29 from a population in which your null-hypothesis is true (i.e., the is no difference/the mean is zero) and calculate the t statistic in each of those samples, you would obtain the t-distribution with df = 28.
Now, instead of using asymptotic distributions, which are very often not appropriate, we can also simulate an empirical reference distribution. One of the methods to do this is the bootstrap resampling scheme. The basic premise underlying the bootstrap method is that we use the observed data as best estimate of the actual population (which is what Maximum Likelihood is all about). So instead of relying on an unknown population, we use the observed data to create a (large) number of new data sets. We can do this in a parametric way (generate new data sets using the ML estimates as population parameters) or non-parametric way (generate new data sets by drawing, with replacement, from the observed data set). These new data sets are often called 'replicates'. Which parameters to specify depends on the test you are doing, so let's not go into that here.
Standard p-value testing looks at the t-statistic in your observed data, and compares it to a null-distribution. I'm guessing you collected 1000 bootstrap replicates, for which you can then calculate 1000 t-statistics. Your bootstrap p-value is then calculated by evaluating how many of those 1000 statistics was greater than the one you observed in your data. That's it. You have your p-value. The number of replicates has nothing to do with degrees of freedom. Even the standard error can be calculated simply as the standard deviation of the 1000 statistics of your empirical reference distribution.
If the observed t-statistic is considered 'extreme', or 'unlikely to occur' in the reference distribution, we reject the null-hypothesis. Usually, belonging to the top 5% is already enough to decide that your observation was extreme. This is the terrible p < .05 convention.
Disclaimer: there is nothing wrong with using a p < .05 if research calls for that cut-off. What is highly problematic is the blind specification of that cut-off without realizing the meaning of Type-I and Type-II errors in the specific study.
Hope this helps
That would not make any sense. It's not a t-test you are doing. If you could assume that delta/se(delta) ~ t, at least approximately, then you would have done a t-test and not bothered with estimating a sampling distribution from a sample of (just!) n=29.
It possibly puts your mind at ease a bit to know that z-tests also don't have something like "degrees of freedom".
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Writing