I want to know when to use a specific type of distribution (in general), for example I know the normal distribution and know that according to my data adjustment distribution can choose specific statistical parametric and non-parametric depending on my experimental design . Now I would like to know about other distributions and when I have to adjust my data to these. eg Binomial Distribution, F Distribution, Poisson Distribution. Maybe I'm wrong as I understood it, can you bear me explanations? and basic literature?
A Google search on yields lots of hits that should be relevant. I've included a couple of them below.
There are a couple other points that you should bear in mind.
HTH.
http://www.itl.nist.gov/div898/handbook/eda/section3/eda366.htm
http://pages.stern.nyu.edu/~adamodar/New_Home_Page/StatFile/statdistns.htm
Bruce,
I think your 2nd point is a special case for the normal probability model (LM) but not for others (GLM). As I understood (what may be wrong), the likelihood is evaluated for the prediction (transformed by the link function) based on the linear predictor function, so it is actually all really about the distribution of the response. The key is that the expected value is not (does not need to be) constant, that is depends on the linear predictor function.
As example let's consider a Poisson response (counts):
the likelihood is evaluated using the Poisson distribution, and the parameter lambda is estimated. This is straigt forward if lambda is assumed constant. The relationship of the response to some other predictor(s) (like group, time, treatments, etc.) is modelled by a linear predictor eta = b0 + b1*X1 + ..., and lambda is "linked" to eta through the link function (in this case usually the logarithm) so that log(lambda) = eta or lambda = exp(eta).
So lambda changes depending on the values of the predictor(s), but the likelihood is evaluated for the observed counts (and not for any difference or residuals from observed and expected counts): Y ~ Pois(lambda=exp(eta)). Or am I wrong here?
For the normal probability model the principle is the same. The estimated parameter is mu (and sigma² is considered a nuisance parameter that is considered by using the profile likelihood over sigma²). The link function is identity, so mu = eta. This leads to the fact that the likelihood for Y~N(mu=eta,sigma²) is the same as (Y-Y.hat) ~ N(mu=0, sigma²) because Y.hat = eta (the linear preditor simply and directly predicts Y).
In conclusion: the residuals (Y-Y.hat) for the Poisson model are not Poisson-distributed, but they are N(0,sigma²) for the normal model (and even this won't generalize to the case that sigma² is not constant).
Good morning Jochen. I was certainly thinking of OLS models, where the errors are assumed to be normally distributed with mu=0 and some variance. But I was also thinking of binary logistic regression, for example. It is typically described (I think) as a generalized linear model with a logit link function and a binomial error distribution. I'll have to think about your Poisson regression example.
Meanwhile, let's bear in mind that the Poisson distribution converges on the normal. I think some sources suggest that the approximation is fairly decent if the Poisson parameter is 10 or more.
Cheers,
Bruce
Hi Bruce,
maybe I first should ask: what do you mean by "error distribution" of the binomial model? Is it the residuals (differences between predicted (expected) value and observation? These residuals are between 0 and 1, beacuse the expextation is between 0 and 1 and the observations are either 0 or 1. So to my understanding the residualy can not have a binomial distribution (they can have a beta-distribution, but I am not even sure of that).
What really counts for the model to estimate the coefficients (using a logit link or whatever) remains the data and the distribution of the data. And this is binomial. This is completely similar to the Poisson example: the likelihood is evaluated for the data and with the probability model for the data. For the binomial model the parameter pi (proprotion, expected value) is linked from a linear predictor via the logit function (for the Poisson model it was the log linking it to the rate parameter lambda). And the same is the case for the normal probability model, where the parameter mu is linked via identity (i.e. mu is directly given by the linear predictor) and the likelihood again is evaluated for the data. Because mu and sigma² are independent, the likelihood components of mu and sigma² can be separated through the residuals, so that the model can be separated into an deterministic part (the linear predictor) and a stochastic part (the residuals or "errors" for with the likelihood is obtained from the normal probability model with mu=0 and sigma². This is a special case. This separation is possible only for the normal distribution, but not for other distributions.
And yes, you are certainly correct that the Poisson converges on the normal. And even for cases where the data distribution is not a good approximation, the shape of the likelihood (where the inference is based upon) is usually a sufficiently good approximation (what is generally the case for any data distribution given larger enough sample sizes). So I agree that your original statement is approximately correct in the general case and exactly correct for the normal probability model.
- Badges
- Science topic