There is a couple of error models (probability distributions) where I do understand the derivation in terms of going from simple processes where you have limited knowledge (uncertainty) to the final random variable. For instance, for n Bernoulli trials you get the binomial distribution for the number of "successes" (events). If we go from n experiments to an interval where we think of performing an infinite number of Bernoulli trials with infinitively small success rate, we get the Poisson model. From there, we can ask what to expect about interval times for a given number of events and we get the exponential distribution. Getting the normal distribution can be explained in another route from simple assumptions about errors, e.g. as Herschel showed. Often we deal with strictly positive variables with somehow proportional mean and variance, like shown here:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3019185/figure/F5/
The normal error model is clearly not appropriate here. Some authors use the log-normal distribution, but this requires multiplicative contributions of errors. In many cases I do not find a theoretic justification for multiplicative errors.
However, the gamma distribution has useful properties to fit such data and the fits are usually better as compared to the log-normal. Unfortunately, I can't find an explanation or derivation of this distribution that theoretically explains why (or if and in which cases) this should/could be applicable. The only thing I found is the derivation from the exponential distribution to provide expectations for the waiting times until a given number of events has occured. Is there any theoretical line of arguments to follow that this also models, say, the number of molecules in a solution, or, as in the linked example, the number of carbonyl residues in a molecule and things like this?
I know the the gamma-distribution gives the expectation about the parameter lambda of the Poisson distribution fro a given number of events (i.e. gamma is the conjugated of the Poisson). I hoped to manage going from there but I'm stuck.
Maybe I'm wrong with the gamma distribution at all. I just have the feeling that this provides the best opportunity for a theoretical justification why our knowledge (or uncertainty) about positive variables should be described by the gamma distribution. I know that other distributions are also used to model right-skewed data but to my impression all these distributions are just taken for granted and used because they simply fit well. I want to get this step beyond the simple "because is fits well". I want to get the "beacuse this distribution correctly expresses what we can know about the underlying processes", as I have for the binomial, Poisson, exponential and normal distributions.
Regarding the lognormal, in many applications one can justify it if the central limit theorem is applicable to the logarithm of the variable of interest--see, e.g., http://www.mlahanas.de/Math/Lognormal.htm. I've also seen a lognormal error model derived from a stochastic differential equation for fatigue cracking in metal.
You could also think about the Weibull distribution; this is a distribution of minima, so it applies to situations where you have a "weakest link" model, e.g., the breaking strength of chains.
Here is a paper that derives the generalized gamma distribution (of which the gamma and Weibull are both special cases) from a physical model: http://www.uh.edu/engines/generalizedgamma.pdf. Presumably you could also derive your vanilla gamma from a simplification of this.
Derivations of the gamma from a physical process seem to be less common than for some other distributions. Maybe if you post some specifics about your application someone will have a better answer.
Regarding the lognormal, in many applications one can justify it if the central limit theorem is applicable to the logarithm of the variable of interest--see, e.g., http://www.mlahanas.de/Math/Lognormal.htm. I've also seen a lognormal error model derived from a stochastic differential equation for fatigue cracking in metal.
You could also think about the Weibull distribution; this is a distribution of minima, so it applies to situations where you have a "weakest link" model, e.g., the breaking strength of chains.
Here is a paper that derives the generalized gamma distribution (of which the gamma and Weibull are both special cases) from a physical model: http://www.uh.edu/engines/generalizedgamma.pdf. Presumably you could also derive your vanilla gamma from a simplification of this.
Derivations of the gamma from a physical process seem to be less common than for some other distributions. Maybe if you post some specifics about your application someone will have a better answer.
Hi, I do not have a definite answer. Try this kind of reasoning and see if it applies to your situation: given mean and std deviation, the max entropy solution is a gaussian. Now, the gamma distribution is the maximum entropy probability distribution for a random variable X for which E(X) is fixed and greater than zero, and E(ln(X)) is fixed ( see wikipedia for details, the symbols do not show in this window). Do you have good information on the mean of the log of your random variable in your problem?
I explain the success of Gamma distribution in applications as follows. It is the simplest distribution which has power tail for small values, i.e., self-similarity property, and exponential tail for large values, i.e., memory-less property. More precisely one can say it in terms of limit conditional distributions.
Hello Dr. Wilhelm,
Your question gives me the impression that you actually seek for a physical basis for the use of the gamma distribution in modelling right-skewed random in variables in the positive real line.
I'm not a statistician [yet], I'm a biologist. I feel like I should share your concern about the scientific interpretation of the models we use. But frankly I don't. In my undergraduate's and insignificant opinion, statistical distributions are mathematical devices. All this means is that for some of them (Poisson, Gaussian, Exponential) we HAPPEN to have a somewhat physically interpretable explanation. As you mentioned, the gamma distribution arises when one considers a particular problem regarding the waiting times of a Poisson R.V. In some particular applications, for instance, when we want to know how many minutes we have to wait until the next call is made given that we had 5 calls the past hour, the gamma distribution has a clear physical meaning. Many other times, it is just convenient because of its mathematical flexibility.
In short, what I want to contribute to your thought-provoking question is that, for me at least, the theoretical reason why to use a gamma distribution in some modeling problem has nothing to do with physical interpretation but indeed to its mathematical properties.
Despite my personal opinion, I think you concern about physically-based derivation is rather valid. The article below seems to bear in the same phylosophical orientation and, more interestingly, concerns a related distribution, the generalized gamma.
I hope my comment adds a complementary perspective to the topic.
Best,
http://www.uh.edu/engines/generalizedgamma.pdf
I am not certain I am really answering your question but as a clinical biochemist, who studies biomarkers, I come across right skewed data all the time. The underlying mechanism for explaining the distribution is that the body normally holds most (blood/cellular) proteins within a defined range for health young interviduals. As we age the mechanisms for producing and holding the biomarkers within the set limits are less able and strict such that we see a gradual shift in higher levels. The major right skewing occurs when we have disease and the deregulation,( which is nearly always positive - too much of something as we cant measure negatives in real life) results in huge variable positive numbers - it is like random numbers a let free in short bursts .
However - as an aside comment from a simple biochemist I do not like fitting distributions to such data and always use non-parametrics in my analysis. Not particularly good for modelling processes but....... I think such fitting does not accurately describe the data and underlying process in biology.
I think you check the suggestion of Collins i.e paper " A physical basis for Generalized gamma " that is available in http://www.uh.edu/engines/generalizedgamma.pdf
If distribution is skewed use higher order cumulants to estimate parameters of time series
You seem to have per-conceived ideas about error distributions, and my own opinion is that the normal distribution is an over-achiever, as Mr. Dr. Prof. Bradley Efron proclaims. Not being a normal person, it's hard to defend normality, but is generally the least of our worries.
Call for Papers:
"12th International Probabilistic Workshop" held in Weimar Germany, on November 4th and 5th 2014"
https://www.researchgate.net/post/Call_for_Papers_12th_International_Probabilistic_Workshop_held_in_Weimar_Germany_on_November_4th_and_5th_2014
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics