In Baysian analysis the main idea is that the parameters are not a single value and they are random variables and, so, they have (joint) distribution. The initial distribution of the parameters is called prior distribution, and the conditional distribution of the parameters given the observations is called the posterior distribution.

For a conditional distribution of observations given parameters, if the prior can be chosen such that the prior and posterior distributions come from a same family, then we say that they are conjugate.

Now, when the distribution of the conditional distribution of observations given the parameters be normal with mean \mu and variance \sigma^2, then the Normal-Gamma prior for the random vector (\mu ,\sigma^2) is conjugate; in the sense that the conditional distribution of this random vector given the observations are Normal-Gamma, too.

So, in many cases, (when the above condition holds), the prior distribution for parameters is considered Normal-Gamma.

Finally, the posterior distribution of the parameters can be derived as:

f(\theta|X)=prior(\theta)/f(X)*f(X|\theta)

where X is the vector of observations and f(X) is the marginal density function of X.

Thank you, Heydar, but this is not an answer to my question. You write: "in the sense that the conditional distribution of this random vector given the observations are Normal-Gamma". This is again just illustrating the result. What I want to know is HOW I can come to this conclusion. Actually, I dont care if this is a conjugate. For a flat prior, the posterior has this shape, so it is the shape of the joint likelihood, too. I am just talking about the likelihood. Maybe this is usually parametrized in a different way, and, surely, it is not a probability distribution (like the priors or posteriors), so there may be some proportionality factor.

Again my question: Given a random vector (mu,sigma²) for X~N(mu, sigma²): how do I get the shape of the joint likelihood of (mu, sigma²) (what seemingly is Gamma-Normal)?

It seems that you have a problem in understanding one of the more simple examples of Bayesian inference. I would recommend you to read a standard text in Bayesian statistcs, for example Gelman et al., (2003, Bayesian Data Analysis), otherwise it will be hard to proceed.

Hi Jochen,

Simon Jackman, a Bayesian methodologist, has put up some pretty clear notes on this, which show the likelihood and conditional densities explicitly"

http://jackman.stanford.edu/classes/BASS/ch2.pdf

Slides 15-23 treat the topic in great detail.

I do hope this is what you are looking for.

Fred

Thank you, I will go and read...

I did *not* want to go into Bayesian analysis. I am just stuck with the likelihoods. My problem is that I can only find "Bayesian literature" dealing with the joint likelihoods...

I just want to follow the MLE of mu and sigma2 when both are unknown. Point estimation is simple, but the confidence intervals require to somehow come to the t (mu) and the inverse-chi² (sigma²). So I actually dont care about conjugacy, priors, posteriors and all such Bayesian stuff (not in general, but for this problem). In this Bayesian setting, it seems to me that I need to take a long and complicated way from priors to posteriors via a congugated distribution that always seems to "fall from heaven" (to me) - just to finally use a "reference prior" to end up in the marginal posteriors from which the confidence intervals are obtained (I know: confidence intervals n.e. credible intervals; but both are congruent for the "reference prior" [I think]).

Jochen:

Wackerly et al. in their 7th edition Math Stat text derive the normal likelihood when both mean and variance are unknown on pages 467 and 478.

Al

Jochen,

You asked again, "Given a random vector (mu,sigma²) for X~N(mu, sigma²): how do I get the shape of the joint likelihood of (mu, sigma²) (what seemingly is Gamma-Normal)?"

This question does not make any sense in toto. If you have observations or data, that you believe are from some Gaussian distribution, then the likelihood is just the product of the Gaussian densities at those observations in terms of the unknown parameters (mu, sigma²).

No Normal-Gamma, or anything else is involved. It is not a "joint likelihood," it is just the likelihood.

See the attached equations. Its easy.

Mark Powell

Jochen,

Sorry about the attachment missing.

Mark Powell

@Jochen Wilhelm,

(1) Did you mean why is the distribution of (\mu\hat , \sigma^2\hat) Gamma-Normal?

Are you asking about the joint distribution of ML estimators of \mu, \sigma^2 which are X\bar and S^2?

If this is the case, then the proof can be found in many general statistics text books.

Note that they are independent.

or (2) did you mean why do we choose Gamma-Normal as the prior distribution of (\mu, \sigma^2)?

In this case, the main idea may be the being conjugate with respect to Normal distribution and, also, the simplicity of computations in this case.

Just a quick note that I agree with Mark completely. There is NO need to get into Bayesian analysis; the notes I suggested happen to derive the joint likelihood and tack on a prior, but the prior is unnecessary, and of course not part of classical analysis.

The Bayesian simply simulates from the joint density by going back and forth between the conditional densities (with some priors included). But the FORM of the joint density is exactly as given by Jackman.

The likelihood of a single observation x_i is

exp(-((x_i-mu)/sigma)^2/2)/(sigma(2*pi)^2)

The likelihood of a sample of n independent such observations is just the product of each of n such likelihoods. It does not make any difference to the (algebraic) functional forma whether sigma is known or not. However, if sigma is fixed then the likelihood is constrained to lie along a ridge through the 'hill' defined by this equation. However, this is an additional constraint. The likelihood is unchanged.

Stephen

Correction. The last term should have been (sigma(2*pi)^(1/2)). In other words it is sigma times the square root of two times pi.

Thank you for your help. I think I'm (slowly) understanding...

I understood that the likelihoods for mu and sigma² are independent and that the likelihood for both, mu&sigma² is thus the product. In fact I come to Marks attached formula, but with (2*sigma²) in the denominator of the exponent (instead of sigma²).

The step(s) that I still dont understand (probably I am conceptually wrong here...) is how to use this result to com to the t-distribution: "Cutting" this 2d-Likelihood function at any given value s² of sigma² is a normal density with variance s²/n. Now this just assumes some fixed ("known") value of sigma². I thought (and this may be wrong) that if the uncertainty about sigma² is considered, then all these normal densities for all the different possible values of sigma² should be considered, and that this is obtained by integration over sigma². If I do this numerically, the resulting function is not the t-distribution; it has wider tails. For n->Inf this functions approaches the normal density.

To make this clear I attached the two plots I was talking about plus the R skript I used to make these plots. The colors in "Marginal.png" mean red = t-density with n-1 d.f. and blue = normal density.

I have the vague feeleing that I'm missing a simple stupid thing somewere but I just cant spot it.

A likelihood is not a probability although it has many properties in common with probabilities (or probability densities). You can see this simply by taking the case of a Poisson distribution with a single observation X. This has one parameter. The space of the probability mass function is X=0,1,2..... and for a given fixed mean (lambda) of the Poisson, then the sum to infinity of all the probabilities corresponding to the unknown value of X is 1. However the space of lambda is the continuous interval from 0 to infinity and for given X the likelihood does not integrate over lambda to 1. The attached notes using a different example may help.

Hello Jochen,

First, I think "likelihood" is more for the data than for the parameters, especially if you are not working in the Bayesian framework. What you have for the estimators of the parameters is a distribution function.

No, let's work on your sample of n realisations of iid random variables, assumed to have a normal distribution (µ, sigma) [or (µ, sigma²) depending on the convention you use].

As mentionned in previous posts, the likelihood of this sample will be wrotten as

L(µ,sigma) = product(i = 1 to n) (1/sqrt(2*pi*sigma²)) * exp( -1/2 * ((x_i-µ)/sigma)² ) --- because of the iid assumtion.

As always, you take the logarithm which changes products into sum, easier to differentiate: -ln L = n/2 ln( 2 * pi * sigma²) + 1/(2 sigma²) sum( i = 1 to n) (x_i - µ)²

You then differentiate and says that at the minimum, differential is null.

If sigma² is known, you only differentiate according to µ which leads to the usual estimator of the mean as the arithmetic mean, Xbar = sum(i = 1 to n) x_i / n.

If sigma² is unknown also, you have two differentials, the one for µ is the same and does not involve sigma, hence leads also to the arithmetic mean, whereas the one for sigma² will lead to the _biased_ estimator of the variance as the mean of the squared differences to the mean, (n-1)/n S².

Note that all this does not give any result on the distribution of the estimators, except the asymptotic results general for all MLE estimators.

Then, comes the study of the estimators and the construction of the laws for confidence intervals. Xbar is N( µ, sigma² / n ) as a linear combination of independant Gaussian variables; estimator of the variance is related to a khi-square distribution as a (scaled) sum of squared centered, reduced normal variables (you can find the details in several introductory books on the statistics).

Then, the Student's T law appears when trying to build the confidence interval for µ, since writting its equation involves (Xbar - µ)/sqrt( Variance / n ) which is _by definition_ a Student's T law (well, you must rewrite it slightly, to have the exact definition, but it works). A few steps :

p ( µ in [Xbar - a, Xbar + b] ) = 1 - alpha ==>

p ( µ < Xbar - a ) = alpha / 2 p ( Xbar - µ < -a ) = alpha / 2

p ( (Xbar - µ) / ( S²/sqrt(n) ) < -a / (S²/sqrt(n) ) ) = alpha / 2

...

Similar considerations appear for the chi-square usage in building the confidence interval of the variance.

Hope this helps to clarify...

Stephen, thank you for this very instructive explanation. It is clear to me that likelihood is not a probability. But consider the case of an unknown mu and given sigma²: The likelihood has the shape of a normal density with mean mu[MLE]=xbar and variance sigma²/n. After proper normalization, this (normalized) likelihood can be interpreted as the "probability of mu" (or the sampling distribution of xbar). One can use it to calculate the confidence interval. This interval contains 95% of the area-under-the-curve of the likelihood. I wonder who this works when simga² is unknown, too. Now there is 2D likelihood. It may be normalized to include a volume of 1, turning it into something that may be interpreted as a 2D probability distribution. However, I would suppose that a confidence interval for mu (xbar) should include 95% of the volume under the 2D-surface.

PS: Please note that I normalized the densities of the normal and t in the plots so that the maximum values match the maximum of the likelihood (respectively the integral). If the *shapes* would be similar, the red line would go exactly through the open circles (for n->Inf this is true).No matter how this is normalized: the central 95% of the auc should coincide with the 95% confidence interval, what is the case for the t, but obviousely not for the "marginal likelihood" (if this, what I calculated really *is* the marginal likelihood; I not quite sure here).

> Stephen Senn: are you sure of your definition of the likelihood just as "using the distribution and reverting the role of X and theta", as in your example?

For me, the likelihood is the probability of the data given the parameter and is a conditionnal probability (or density of...). That is, it is p( X / theta ).

Because of this conditionning, your idea of computing the integral for theta varying 0 to 1 of L(X,theta) seems meaningless, since it is a conditionnal probability on theta, and not a probability of getting theta. You should integrate on X with your idea of integrating L(X,theta). This is the reason why you do not have 1.

If you really want to integrate on theta, you should use the "total probability" theorem, hence calculate integrate(0 to 1 ) L(X,theta) * p(theta = theta) * d(theta), hence assuming a distribution on the theta. And this would give the probabilty of X in general.

But this means that the likelihood is NOT the probability of the parameter theta, despite it is indeed a probability, which was may be your idea?

I don't want to integrate on theta. I am merely using the integral to show that the likelihood is not a probability. It is a proof by contradiction since it is a necessary property of a probabilty that it must sum to 1 over the domain for which it is defined so that if a function does not sum to 1 over the domain for which it is defined it is not a probability. I think you are trying to interpret likelihood in a Bayesian way but this was not Fisher's original intention.

Jochen. I don't think that you can get the t this way. The derivation of the t-distribution relies on the fact that the ratio of (x_bar-mu) to sigma_hat, where x_bar is the sample mean and sigma_hat is the sample standard deviation does not depend on the unknown sigma. See Emmanuel's explanation.

> Stephen : this is the point : likelihood is not a probability on theta, but a probability conditionned by theta, so the integral you compute has no reason to be equal to 1, even if the likelihood is a probability.

But may be the problem is also to agree on what "likelihood is (or not) a probability" means... which is ambiguous if not defining the probability of what.

When computing the integration on theta, as you do in your example when showing that the integral equals 0.091, it suggests "likelihood is the probability of theta" (eventually "knowing X"), which is false --- as shown by your computation, but also simply because it is not its definition and would violate several probability theorem.

However, the (usual ? correct ? Bayesian? but I don't think so for the last one since there is no use of the Bayes' theorem anywhere, no prior nor posterior, just a conditionning on the data which is I think the Fisher's idea itself of working conditionnaly on the observed data if I understood well) definition says "likelihood is the probability of the data knowing theta", which means likelihood is indeed a (conditionnal) probability, and is not in contradiction with your computations...

Likelihood is a concept originally defined by RA Fisher in the 1920s. In Fisher's definition likelihood is not a probability. I am using the term in the way originally defined and encountered in classical books of inference. I know that many Bayesians like to think of likelihood in terms of a conditional probability but this is not Fisher's original idea. For example in Statistical Methods for Research Workers, 1925 he states "... the mathematical quantity which usually appears to be appropriate for measuring our order of preference amongst different possible populations does not in fact obey the laws of probability. To distinguish it from probability I have used the term 'Likelihood'. " (pp10-11)

So to sum up. 1. Fisher points out that the concept he is using is not a probability. 2. To prevent a possible confusion of this quantity and probability he proposes to use the term Likelihood. Thus, if contributors to this thread do not distinguish between likelihood and probability, things will get very confusing.

> Stephen Senn : OK, I see the point.

Just for discussion purpose: is there any contradiction between the likelihood defined by Fisher and the likelihood defined as a conditionnal probability? It may be that when Fisher stated that it is not a probability, it was as "a probability of theta" and did not recognized/thought useful to think of the likelihood as the conditional probability...

Because if there is no contradiction anywhere between the two definitions, both can be used and it is really only a matter of "probability, but of what ?" to say if it is or not a probability...

> Nada el OSTA : great number laws only give asymptotic results, hence does not answer the problem. And the >30 /

Emmanuel. I think it is best if the word 'likelihood' is reserved within the technical literature for the meaning Fisher gave it. After all there is a huge literature on this topic including, for example, books just about likelihood by Edwards (1972), Royall (1997) and Pawitan (2001). After all, you can say 'conditional probability' when that is what you mean..When operating Bayes theorem it is possible to use either likelihoods or conditional probabilities (essentially because their ratios become important) and so the terms have become somewhat confused especially in the Bayesian literature. Usually no harm is done, for example you can refer to Bayes theorem in its odds form as posterior odds = prior odds x ratio of likelihoods, and that is perfectly acceptable, but when one starts discussing likelihood specifically in a non Bayesian context it is best to revert to the original definition.

Of course a further source of confusion is that in everyday English likelihood is often used interchangeably with probability.

However, to return to your question re Fisher, he did not confuse the issue. On the contrary it is precisely because he did not like Bayesian inference and sought to create an alternative that he developed likelihood. Thus, he was fully aware of conditional probabilities but that was not what he wanted.

Emmanuel: "I think "likelihood" is more for the data than for the parameters, especially if you are not working in the Bayesian framework.". Sorry for a confusion. Its clear that likelihood refers to our expectation of the data, given the parameters. Further, I absolutely support your question about likelihood conditional probability. I also would like to know more about this. For me, the difference is just the proportionality factor, so that the integral will be unity.

Stephen: see my comment to Emmanuel. You cite Fisher with " the mathematical quantity which usually appears to be appropriate for measuring our order of preference amongst different possible populations does not in fact obey the laws of probability.". I suppose with this "mathematical quantity" the likelihood is meant. However, if this quantity is normalized so that the integral is unity, the rules of parobability calculus can be applied (I think). Did Fisher really see a principal difference here or did he (just) want to avoid that likelihoods are used directly (i.e. without normalization) as probability values? If he saw a principal difference: why? Probability distributions quantify what we expect, and likelihoods also do this (only inversely to the normal questions)...

I am one step further (I hope). I attached this as PDF. My quastions/problems are written in bold face. The document shows a solution, that I unfortunately just found by trial end error (my missing skills...). Now I can specify my problem to connecting this solution to the known way of getting the t-distribution (also cited in the PDF).

I still think my problem is kind of "stupid", caused by my very limited skills in math. I really apologize for this.

Jochen,

The thing that seems to be eluding you is that the likelihood is not for the parameters, it is for the data. It is the concept.

By the way, sorry I forgot the 1/2 term in the exponenent. Here is a corrected normal likelihood.

Mark Powell

Mark, thanks. I already noticed that 1/2 was missing :-)

And: no, I am well aware of the fact that the likelihood is for the data, given some parameters. Besides, this is even important in this context. The aim is to find an estimate for a parameter, and this estimate does depend on the data. The best estimate is at the maxLik, and I was just seeking for a way to get the uncertainty associated with this estimate, based on the same likelihood function that was used to get the estimate. You may also read my last post.

Jochen,

One small correction, the estimate of the joint mode for the uncertainty about the paramters is obtained using maximum likelihood estimation. "Best" would be to infer the entire joint uncertainty distribution of the parameters given the data. Then you can calculate whatever point estimates you find useful.

I cannot tell what you are trying to do or what your problem is in your pdf file with bolded questions. You get a t distribution if you have a hierarchical normal-gamma and marginalize the variance. No likelihood is involved.

What are you trying to accomplish? What problem are you trying to solve? We might be able to help you a bit better if we knew that.

Mark Powell

Mark, I try to understand the meaning of the t-distribution in relation to the likelihood. There must be a link, but I don't see it (clearly enough).

You write: "You get a t distribution if you have a hierarchical normal-gamma and marginalize the variance." Why do "I have" a normal-gamma? Where does it come from? (I think this is related to the likelihood and should somehow be derived from it). Why is the variance marginalized? (I think this mathematical procedure has a "meaning"; it means that we acknowledge the unceratainty in the estimated variance, and this also is related to the likelihood).

The former PDF referred also to a picutre I showd before, but this is now hidden in "previous comments". I attached it again.

This is the R script

> Stephen & Jochen.

I've thought a little bit around that issue of the difference between likelihood and probabability. I think I have some ideas; would you mind to give your advise on what follows?

So, you have a set of observed data (x1,...,xn) realisation of a set of random variables/vectors X1...Xn for which you have a model of (cumulative or density) distribution function. This C/PDF depends on a (set of) unknown parameters, theta, that you want to estimate (and then test may be).

You can always write the [density of] probability of obtaining your observed data with this model. This [density of] probability will be a function of both the (unknown) theta and the (observed) xi : p(observations) = f(xi, theta).

In the Fisher approach, theta is a "usual number". Hence, defining conditionnal probabilities has no meaning (you should have an event to define conditionnal probabilities). Then, f(xi, theta) is somehow a family of function indexed by the parameter theta. Each of these function f_theta0(xi) (for a given value theta0 of theta) is well-defined probability : the probability of the observation for that model "knowing" that theta = theta0, but "knowing" here not related to any probabilistic vocabulary.

But you can also see f(xi, theta) as a function of theta indexed by the xi; this is the likelihood. In that case, for a given set of xi, f_xi(theta), has no probabilistic meaning, hence the "this is not a probability" of Fisher. And one search the theta value that maximizes f_xi(theta) because, getting back to the f(xi,theta), it will be the value of theta that will maximize the probability of the observations we made for the model. By solving f'_xi(theta) = 0, you get theta_0 = g(x1,...,xn) that gives a ponctual estimation of theta. Then Theta0 = g(X1,...,Xn) is the corresponding estimator and you can derive its properties either directly (as in Jochen's question about the mean) or using asymptotic properties.

In the Bayesian approach, theta is a realisation of a random variable Theta, hence one can define conditionning on the event "Theta = theta" and f(xi,theta) is said, as above, to represent the probability of observing xi with the model knowing that Theta = theta, this time with the usual probabilistic meaning of "knowing": f(xi,theta) = p(X1=X1 inter... inter Xn=xn / Theta = theta). You can then use Baye's theorem to revert this and get p( Theta = theta / X1 = x1 inter ... ). I think in that case, you can define two likelihoods:

- the "absolute" likelihood of the data, p( X1 = x1 inter ... Xn = Xn) ;

- the "conditionnal" likelihood, p( X1 = x1 ... Xn = xn / Theta = theta )

with the second one beeing analytically equal to Fisher's likelihood. And in that case, likelihood is a probability, but not the "expected" one: it remains the conditionnal probability of the observations knowing theta. In this context, you have "naturally" a distribution for Theta, hence it answers, but from a completely different point of view Jochen's question. According to the prior selected, the distribution of Theta will change, and if selecting the "correct" prior it can give the same estimator, hence the same distribution, than Fisher's approach. This is in that context that the "normal-gamma" idea appears.

It would mean in that case that in fact the mathematical object f( xi, theta ), which is a multivariate function, has different names accordin...

Nice post!

I think the Bayesian "reversion" of L(theta|data) to L(data|theta) is implicitly used by Fisher, when he states that the theta0 at the max L(theta|data) is the one maximizing the probability p(data), given theta=theta0, what actually is just proportional to L(data|theta).

I also think that a conceptual distinction (Lik vs. cond.p) can only be made when there is a super-layer of abstraction from what the Fisherian likelihood follows.

(End of the previous comment, seems to be truncated :()

It would mean in that case that in fact the mathematical object f( xi, theta ), which is a multivariate function, has different names according to its usage/interpretation:

- f(xi, theta) as a function of xi is a [density of] probability function : the probability of the observations for a given value of theta ; it is really a probability since summing (integrating) on all possibles values of xi will give 1, whatever theta is.

- f(xi, theta) as a function of theta, with theta a number, is the likelihood of the data for a given value of theta, but not a [density of] probability function, as can be seen by summing for all values of theta for these xi that leads to something different of 1

- f(xi, theta) as a function of theta, with theta a realisation of a random variable, is the conditionnal probability of the data knowing theta, and is slightly the same as the first interpretation; it is a [density of] probability when varying x1, not theta.

Emmanuel, I can mainly agree with this, which is essentially what my attached pdf (higher up) explained. Others in this thread have looked at this through Bayesian spectacles. This may or may not be the correct way to do it. That's another matter partially addressed in my paper 'You may believe you are a Bayesian but you are probably wrong'. However, that is not the way that Fisher did it. He was not interested in probability statements about parameters at the time that he defined likelihood. And he DID define likelihood whether we like his definition and concept or not and proved many theorems in connection with it. Thus to the extent to which we discuss likelihood we had better discuss likelihood and not probability. Later Fisher developed fiducial inference for dealing with uncertainty about a parameters but that is another can of worms.

> Jochen : I don't think Fisher's approach is an implicit reversion of the conditionning, because it would assume that theta is a random value to be defined, and if I understood well Stephen's posts, it was not Fisher's idea... But it may lead analytically to the same results. However, for that, one needs to select an appropriate prior (which is required when doing the reversion --- this is why also "implicit reversion" seems strange to me) for that.

Jochen,

There is no link between the likelihood and any distribution. The likelihood works with any model you assume. The t distribution has nothing to do with it.

You asked: "Why do "I have" a normal-gamma? Where does it come from?"

You get a normal-gamma because you choose it to represent your data. If you choose it to represent your data, and if you marginalize the variance, you will find that a t distribution now represents your marginalized data. The mathematical procedure that is "marginalization" is as exactly in the pdf you provided, you integrate out the variance. Marginalization has no relation to the likelihood at all. One is a mathematical procedure, the other is a probability or probability function.

You appear to be very lost with what you are trying to do with probability models. I recommend that you get a good stats book and read up on how probability models and the probability calculus works. Without that fundamental understanding, you won't be able to pose a question where we can help you.

Mark Powell

Mark, could you recommend a book - for a non-mathematician?

Jochen,

Your questions have all been very mathematical. I cannot think of a book that would lead a non-mathematician to the answers you seek.

Probability and stats are almost always taught out of university math departments, for good reason.

Hopefully someone else knows of a reference for you. Better yet, hire a statistician to help you.

Mark Powell

I would recommend "John E Freund's Mathematical Statistics with Applications".I learned my statistics from the second edition (1971) at which time Freund was still the author. I have a copy of the seventh edition which is actually by Irwin Miller and Marylees Miller and dated 2004. I don't know when authorship passed over to them. When I was an undergraduate in the 1970s this was not a recommended book on my course. It was the book I read to provide me with enough background to understand the recommended books on my course!

Thank you, Stephen. I will try to get this book.

In the meantime, i found some mistakes in my script, and in my understaning :-)

I think I know better now why the integral (just) is not identical to the "marginal likelihood", because for the marginal likelihood the individual values must be weighted (-> "conditional probabilities"). I also found that the result of the integral in fact is the t-distribution with n-1 d.f. So this problem is also "solved" by now.

I am now stuck with the fact that the weights are given by n/SS (what is the precision). Why this? Why not by SS/n (variance)? I know and I read that the disitribution of the precision is gamma with shape = n/2+1 and rate = SS/2. I could not find a reference where it is explained why. I further found empirically that f(x = SS/variance|mu) is proportional to the density of a chi-squared-distribution with n+2 d.f.. The value SS/variance is also a precision, just normalized to be 1 for variance=SS. I am still looking for a link between these two findings, but this is just a matter of time.

(SS = sum of squares = sum{(x[i]-xbar)²})

Jochen, Think of weighting two sample means, one based on 20 observations and the other based on 10 observations. Other things being equal would you not want to weight the former twice as much as the latter? But the variance of the latter will be twice (because proportional to 1/10) that of the former (which is proportional to 1/20). So weighting by the variance will give you exactly the opposite of what you want, whereas weighting by the inverse of the variance will agree with intuition.

But maybe I have misunderstood your question

Thank you, this is a good example. There are sometimes these stupid little things that are too clear to be recognized.

But I still have this question: How can one derive ("come to") the gamma distribution as the distribution of the precision? The derivation of the normal distribution is clear (e.g. see attached link), also I know - for instance - how the binomial and Poisson distribution arise from simple assumptions (about the knowledge we can have about "events"). I also see how the gamma is related to the distribution of the parameter lambda in a Poisson process. Further I know that the gamma distribution is a generalization of the exponential distribution as derived modelling the uncertainty about "waiting times". I dont get a (logical) connection between the "waiting time" to the "precision". I suppose that the line of arguments goes via the normal X, the chi-squared Sum(X²) to the gamma of (n/Sum(x-a)²)...

http://www.gtas.dicom.unican.es/files/docencia/tds/Gaussianity_IEEE_SPMagazine_Kim_2008.pdf

Jochen, first of all let us agree that what you are doing has nothing to do with likelihood per se. I presume that what you are asking is "why do Bayesians commonly use the gamma distribution for this purpose?". Well, one answer is that it is purely mathematical convenience. In fact a true subjective Bayesian would strongly reject any suggestion that such a distribution is likely to be appropriate. In fact for variance components in general there have been a huge number of possible proposals. So if you don't find it obvious it's because it isn't.