For some smaller and less know "statistics" often no option to calculated the error or confidence intervals is given. However, this might be obtained by bootstrapping. In addition, both McElrath and Kruschke have used grid approximation as an example in their well written books. While given as an example, I have never seen it being used, although understandably difficult for higher dimensional issues for estimations of single less critical parameters it might be appropriate.
Consider we apply a multivariate analysis and calculate "R" from ANOSIM (see vegan R package vegan::anosim and https://sites.google.com/site/mb3gustame/hypothesis-tests/anosim). Now I am not interested in the testing if the data is compatible with 0, (it non "random" and biased which might be one and the same thing). I want to make some statement on the meaningfulness of the posterior of R by adjusting the likelihood (which is all we do).
The "R" from ANOSIM does not return the error, however we can bootstrap the data and estimate the alpha and beta parameters from beta distribution under the assumption this "R" would be a random variable. Given we know alpha and beta parameters from "R" we could use this as the likelihood introduce the prior and obtain the posterior estimate (see Fig and example). I am just not aware whether this is a reasonable approach or not as there is not much documentation on grid approximation.
Thank you in advance!
I am not clear exactly what you are asking. But you might find use for the discussion in the final chapters of the book@ "Bayesian Theory" by JM Bernardo & AFM Smith, Wiley (1994/2000)..
You idea for bootstrapping may work, but I would suggest just doing a few bootstrap samples anf showing variation in the posterior distributions graphically. In fact, in may be even better just to do a few random simulations from the full model with a single fixed set of parameters, and to get the posterior distributions for these samples ... this would give a direct assessment of the sampling variability of the posterior when the true values are known.
David A. Jones I am very sorry for the late-late response I did not get any message. In any case, my question now looking back on it is horribly written. I meant the following:
We have a ranked statistic called R. This statistic has no known method for obtaining the SE. It is however know it can only be beta distributed as it can only takes values >0 and
I have two separate suggestions.
(1) It seems unnecessary to fit a beta distribution... instead you could just work with the bootstrapped values of R directly. If you were to ignore the "prior", you could just construct and plot a histogram from the bootstrapped values. Doing this essentially just attributes a unit weight to each bootstrapped value. To apply the prior you can just multiply each such weight by the prior and then use an obvious variant of the histogram procedure to produce a plot of the posterior distribution. This might at least provide some form of validation of using a beta distribution if the results are comparable. However, while this would correspond to what you were trying to do with your beta distributions, I believe that you are wrong in trying to treat the bootstrap method as providing any sort of "likelihood" and hence the treatment of results in terms of prior and posterior is wrong. The bootstrap provides an estimated sampling distribution of R for a single unknown fixed version of "truth" ... there is no immediate way of getting the same for other versions of "truth", which is what you need for a likelihood function to use within a Bayes scheme. What you have is simply a way of discounting sample values in the bootstrap version of uncertainty according to what is up- or down-weighted by what you term the "prior". It seems that what you are doing is not standard practice.
(2) You are ignoring the ability of your anosim package to provide a significance test of "no difference". But you can convert this ability to provide confidence intervals for some meaningful non-zero "effect" in the obvious way. You just need to have a parametric way of adjusting one of the samples to become more or less similar to the other. Specifically, you would construct an adjusted pair of samples from the original, containing a given size of adjustment, and then apply the test of significance for no-difference. If the size of adjustment is such that the result says "accept the hypothesis of no difference", then that adjustment is counted as being in the confidence interval. This is standard theory.
However, I can't claim to understand what you are really trying to do. You say "I want to make some statement on the meaningfulness of the posterior of R", but you don't say where the prior distribution is coming from. Notionally, the bootstrapped distribution provides an indication of the sampling uncertainty of the raw estimate of R, but it doesn't assign any particular meaning to R itself. Further, in your first message you mention something about the data being non-random and you will need to worry about the consequences of this. In addition, you twice mentioned "grid approximation" but I don't know the meaning/use of this. Overall, I think you need to get a better understanding of uncertainty for your context.
You might want to find a practical statistician at your university to help with this.
- Badges
- Science method
- Similar topics
- Biological Science
- Ecology