See "3 Normal-Gamma prior" (page 6 ff) in:

https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf

Jochen Wilhelm Thank you for providing the link. This article suggests that when σ^2 is known, use a Normal prior; when both the mean μ and variance σ^2 are unknown, use a Normal-Gamma prior. I thought that prior distribution should be independent from our current knowledge. In this example, I assume that my previous data were drawn from a normal distribution. My new data are also drawn from a normal distribution. So, I think it is logic to use a normal prior regardless of whether σ^2 is known or unknown. Why should I use a different prior when σ^2 is unknown?

Dear All,

I am currently very busy and I have just time to share some basic remark of very general validity: The prior pd and the conditional pd imply the unconditional pd. The two - prior and conditional - probability distributions are usually interpreted differently: the first one models our knowledge say, about the mean and variance, whence the conditionals describe the randomness of the observed quantity. Let us say that due to physical nature of the bserved quantities we want to keep all conditionals normal (gaussian) with the "unknown" mean and variance. But we do not have ANY bond in choosing the prior pd. Only qualitative features are to be followed by the prior pd.

Obviously, in most cases, even if all conditionals are normal, the resulting unconditional might be (and in general - is NOT) normal. There are some exceptions, like if both, conditionals AND prior are normal with given variances: then the result is also normal (with variances summed up).

The case of fuzzy info about the variance and the mean of the conditionals, we need to model the lack of knowlegde of both: the mean and the variance! Thus, the family of prior pd-s must consist of two-dimensional pd-s. OBVIOUSLY, the marginal for the variance must be concentrated on positive values!

Therefore I feel myself justified to guess (!?, lack of time for more complete studiy) that

the name normal-gamma stands for a product of the normal as the a priori pd for m and the gamma as a priori for v=\sigma^2.

Hopefully this will help.

Best, Joachim

Hening Huang ,

If σ² is unknown, it adds to your uncertainty about µ and thus should be considered.

And no,the prior should not be independent of our current knowledge. It should represent our current knowledge! It is a yet unsolved problem (if it is a"problem" at all) to sepcify a prior that does not represent any kind of knowledge (chosing a model, in a very general way, already depends on some idea about the system under investigation).

The task of the analysis is to use data to modify the knowlege represented in the prior (and it's not possibleto modify something that is not there) - within the space given by the model (e.g. if a parameter is reflected by a gamma distribution, it cannot become negative, no matter what the data is; if a parameter is reflected by a normal distribution,it can become negative even if that makes no sense theoretically).

Jochen Wilhelm Joachim Domsta

Thank you for your replies. So, the key point is, “prior should not be independent of our current knowledge.” This seems unintuitive. Anyway, can I use s² as an approximation of σ² in the Bayesian setting? I know this would deviate from the Bayesian methodology. However, if allowed, this would give a similar solution to that when σ² is known and not need to consider the parameters for the Gamma distribution. Also, the result would be “consistent” with the frequentist result: weighted mean.

Frequentist questions are different than Bayesian questions. I don't think that there is a resonable way to compare these two or to define "consistency".

To my opinion, if your questions are Bayesian, you should do a Bayesian analysis, explicitely formulate your (limited) knowledge about all the model parametes (including the nuisance parameters). If you are concerned about frequentist questions, do a frequentist analysis.

If you have an estimate of s2, than you simply have a maximum likelihood model under a normal distribution.

Why Bayesian?

As a practitioner, I expect a statistical method to give reasonable result (output) for a real-world problem (question), regardless of whether the method is based on frequentist or Bayesian statistics. I don’t think there is a need to distinct between frequentist question and Bayesian question in real-world applications, at least no such need for practitioners. I understand that frequentists and Bayesians may have different views and settings on the same problem (question). However, the validity (or effectiveness) of an output from a frequentist or Bayesian method should not be and cannot be judged by either methodology itself.

You want a result for a question (what I can perfectly understand) - "regardless of whether the method is based on frequentist or Bayesian statistics". But that's simply not possible, because these philosophical frames cannot answer the same kinds of questions. You cannot substitute one by the other. Much has be misinterpreted (to the disadvantage of patients and society) because researchers and practioners wrongly think that there is no difference in these concepts (the most common error is to thin that Baesian questions can be answered with frequentist methods, or that Bayesian methods would give results that have frequentist properties (or a frequency interpretation). Mixing this up, also as a practioner, does more harm than good, IMHO.

---

Don't get me wrong: I am not blaming anyone (particularily not practioners). The literature is extremely confusing and there is no proper education at all. These are important fields where we all must develop and improve (e.g. to change our educational system, teaching statistical thinking not as a separate subject but deeply woven into the curriculum of the empirical scientific subject [medicine/biology/ecology/....]).

Jochen, Statisticians and a few good researchers all understand the difference in the formula for posterior probabilities and maximum likelihood values, as well as the different advantages of Bayesian and Likelihood optimization. And of course by knowing the mathematics are perfectly capable of deciding when to use each one (or even both as I often do). And we certainly don't interpret the results the same.

If some researchers do, then it is up to them and their reviewers to correct that rather than having the hubris to believe that statistics and the philosophy of statistics is understandable without higher level theory and training.

I also suggest that that you read AW Edwards on Likelihood where he describes how Bayesian posterior probabilities reduce very often to maximum likelihood. Also read Joseph Felsenstein on the same topic with reference to phylogenetics..

Read also Ziheng Yang (Molecular Evolution, A Statistical Approach, 2014) on the pitfalls of priors.

Furthermore, as I understand this particular problem, the prior does not represent a distribution from which the prior is estimated but rather a measured standard deviation. If that is incorporated as a prior then the Bayesian expression with reduce to a conditional likelihood expression. So Bayesian analysis is not necessary although Bayes theorem, used by all statisticians of any ilk, is part of the conditional likelihood.

Jochen Wilhelm

I agree with you that “The literature is extremely confusing and there is no proper education at all. These are important fields where we all must develop and improve (e.g. to change our educational system…” I’d like to copy two paragraphs of my comments in another discussion “Why are we still teaching t-distribution?” here:

“It has been my impression for years that if something associated with the application of statistical methods goes wrong, practitioners are blamed for misusing statistical methods, or for misunderstanding or misinterpreting statistical concepts (e.g. p-values). But why statistics textbooks contain those statistical methods and concepts that are faulty or easy to be misunderstood (e.g. confidence intervals)?”

“I think it has been accepted, after many years of debate, by many statisticians and practitioners that t-tests, or hypothesis testing in general, and confidence intervals are faulty tools for statistical inferences. Indeed, our university statistics classes have taught these tools for over 80 years (since 1930’s) and will continue to teach. But few textbooks point out the limitations and shortcomings of these tools so that students naturally assume that these tools are valid. Note that a prescription drug always comes with a description and a warning about the potential side-effects of the drug. Analogously, a statistical inference method (tool) should be taught with a warning about its limitations and shortcomings, and potential misleading results. Also note that statistical methods are not the law of nature and statistics is a methodology.”

Statisticians at a Masters or Doctoral level are taught the "limitations and shortcomings" of modeling natural stochastic processes during years of study. These are measured by first optimizing the models that best describe the process.

For decades statisticians have also recognize the terrible complexity of these stochastic processes and that statistical models can never perfectly describe natural processes. But these models and their associated measures of significance or confidence and power are all we have to make any sense of the natural world and the "laws of nature". And fortunately they do provide insight to those who ask the right question in the first place.

Statistical analysis is a rich science and comprises a large number of really useful tools. The mathematics is not easy and so understanding the pitfalls is extremely difficult for those who typically get just a year of study. That is too short a time to learn the nuances of analysis. So it is not surprising that many very bright practitioners are confused and often misinterpret results.

But this is not the fault of the instructors who must convey their years of training to a necessarily short course that most researchers assume has prepared them for all sorts of analysis. It is rather their responsibility to seek advise from statisticians with years of training. Without consulting a statistician with years of training results will often be nonsensical.

Hening Huang

“I think it has been accepted, after many years of debate, by many statisticians and practitioners that t-tests, or hypothesis testing in general, and confidence intervals are faulty tools for statistical inferences.“

I don't subscribe here. It's not that the tools are bad. It's their mindless and inappropriate application and the unwarranted hopes and wishes that researchers project into these tools. A hammer is a good tool, but for a particular (and possibly very useful and sensible!) task, but not for many other tasks (like driving a screw, cutting paper, or sawing wood).

To cite Douglas Adams: "We demand rigidly defined areas of doubt and uncertainty! "

I couldn't agree with you more. As a statistician I am appalled at much published statistical analysis and the conclusions derived from them.

To cite the other Douglas Adams that I always have at the beginning of my lectures. The key is to pose the question that is being asked of the statistical analysis:a well formed hypothesis and the corresponding null. The answer (42) alone is not enlightening.

“ I checked it quite thoroughly,” said the computer, “and that quite definitely is the answer. I think the problem, to be quite honest, is that you’ve never actually known what the question is.”

Hitchhiker’s Guide to the Universe

Douglas Adams, 1979

John Tukey

"The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data."

and:" It's better to solve the right problem approximately than to solve the wrong problem exactly".

And I'll end with RA Fisher:

"To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of."

Oh and of course Ernst Meyer echoes those of us who are very found of the covariance matrix (Σ) that is calculated from the data and underlies so much statistical analysis:

"The mean is a mere abstraction. The most interesting parameter in the statistics of biology is the variation"

Jochen Wilhelm and Patrice Showers Corneli

I greatly appreciate your comments. I also like a famous quote by Tukey (1962): “Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.”

Let’s come back to my original question (that is often encountered in real-world measurements). The problem of interest is restated as follows. Assume that a measurement gives n observations y1, y2, ......yn. The data are drawn from a normal distribution: Y~N(μ,σ²). Both μ and σ are unknown. We have the sample mean and sample variance from the measurement data. On the other hand, based on historical data, we have prior knowledge about the measurand that can be represented with a normal distribution μ ̃~N(y_prior, σ_prior^2). We want to incorporate the prior information into the current measurement to obtain: (1) an estimate of μ, and (2) the standard uncertainty of the estimate. My approximate frequentist solution to (1) is simply the weighted mean of the prior mean and the sample mean. Accordingly, it is easy to obtain an approximate solution to (2). My question is: “What are the Bayesian solutions to (1) and (2).” By the way, the Bayesian solutions in Murphy’s paper(https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf) with the “Normal-Gamma” prior are not the answers for this problem.

Hening Huang

"We want to incorporate the prior information into the current measurement to obtain: (1) an estimate of μ, and (2) the standard uncertainty of the estimate. My approximate frequentist solution [...]"

Estimation problems are inherently Bayesian questions. There is no frequency solution for these kinds of questions. The frequency is about the data. It is based on a hypothesis about µ (and/or σ²) under whilch the relative likelihood of the observed data is judged (based on data we didnot observe but would expect under the given hypothesis). There is no way around Bayes if you want to judge what you can reasonably know about µ (and/or σ²) given the observed data.

Jochen, Yes you are quite right. The only question is the quality of the prior which in Hening's case seems very good.

Instead of replacing the standard error in the posterior distribution why not use the know mean and standard deviation as the prior. You hint at using the parameters as a weight, which is exactly what Bayesian methods do.

Specifically the likelihood with its unknown parameters is weighted by the prior with the known parameters.

That way you are essentially multiplying the likelihood by the prior. It is simply a conditional likelihood (with a known prior. Please see attached. I could not figure out how to format mathematical statements in ResearchGate so attach a revision of one of my teaching lessons.

Patrice Showers Corneli

Thank you for your suggestion. I looked at the attached file and noticed that your formulation is essentially the same as that in the section 2 of Murphy’s paper, “Normal prior”, in which, σ² is assumed to be known. I am interested in a solution to the case that σ² is unknown.

Ah! Of course, I read the question wrong.

This makes it a more interesting question as I would have appreciated if I had properly read the question. Not sure of the exact answer, but with the mean known and the standard deviation unknown I would guess that the answer would be similar to what I wrote but with a gamma prior rather than a normal prior distribution.

Will have to think about that.

Do you mean the following Murphy, which discusses the subject but without a specific solution in sections 4 and on?

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=FA91E26DCD144D02AE4FB038BE225CB0?doi=10.1.1.126.4603&rep=rep1&type=pdf

Anyway it is a wonderful question perhaps not quite straightforward.

I appreciate the excuse to get back to my favorite part of statistical theory - that of the relationships among all the well-known distributions, normal, chisel, gamma, Beta, inverse gamma, etc. Just elegant stuff.

Yes. It is the paper that discusses a number of priors. However, it seems that none of the formulas is for my practical question.

Let me suggest the following:

STEP 1.

replace the prior pd by a nonintegrable JOINT PRIOR weight function g_o(m)\times 1(\sigma), with the domain R \times R_+ , where g_o is the prior of m;

conjecture A: then for the gaussian g_o the joint weight W should become a function of x_{average}, s^2, m, \sigma and n, only.

conjecture B: Probabbly, Indepentently of the prior pd g_o, the result will be intergable wrt m and \sigma.

STEP 2.

if the second conjecture is true (hopefully it is!), then apply Bayes to obtain the posteriori probability density function for m and \sigma as follows: by dividing W by the integral of W wrt m and \sigma over R\times R_+.

conjecture C: the resulting JOINT POSTERIORI pd of (m,\sigma) [which s dependent on the prior of m AND the sample], in case of normal prior N(m_o, \sigma_o) of m should be determined by x_{average}, s^2, m_o, \sigma_o and n; and the margine of the posterior variance=\sigma^2 should be somehow related to inverse gamma.

Joachim

Erratum: the first conjecture is

conjecture A: then for the gaussian g_o the joint weight W should become a function of x_{average}, s^2, m, \sigma and n, only. AND of course on m_o and \sigma_o.

.

what's so wrong with the good old suggestion of Jeffreys ?

http://bayesoft.com/articles/normalpostmusigma.htm

.

Dear Fabrice Clerot ,

congrats for supplyig the right link. Indeed, the Jeffreys algorithm performs the scheme suggested in my reply. I am happy that my conjectures are consistent with those known since 1939 (or earlier:).

Best regards, Joachim

In fact, there has been a debate on the validity of the Jeffreys priors among Bayesians. D’Agostini (1998), a leading proponent of Bayesian methods in particle physics, argued that “…it is rarely the case that in physical situations the status of prior knowledge is equivalent to that expressed by the Jeffreys priors, …” He further stated, “The default use of Jeffreys priors is clearly unjustified, especially in inferring the parameters of the normal distribution, ….” Moreover, we do have prior knowledge about the measurand for the problem of interest, so the Jeffreys priors are not applicable.

D’Agostini G 1998 Jeffreys priors versus experienced physicist priors: arguments against objective Bayesian theory Proceedings of the 6th Valencia International Meeting on Bayesian Statistics (Alcossebre, Spain, May 30th-June 4th)

As far as I understand this refers to the `transformation invariant Jeffrey's prior` which is not the vague improper prior suggested above

If you have a specific prior knowledge there is nothing to prevent you from building your own specific prior

In this practical problem, I do have a specific prior knowledge that is appropriately represented by a normal pdf. My current measurement gives n observations that are assumed to be drawn from a normal distribution with unknown mean and unknown variance. It seems that, after the discussions, there is still no simple, analytical Bayesian solution to this practical problem. In addition, it seems to me that in the Bayesian setting, the selection of a prior cannot be “objective”. Especially, the selection is related to the knowledge about the current measurement. For example, when the variance of the current measurement is known, a normal prior is selected. When the variance of the current measurement is unknown, a gamma-normal prior pdf is selected, which overrides the known, “objective” normal prior. But this gamma-normal prior cannot resolve the problem considered.

May I ask what is sooo.wrong with the approach in the link I supplied above.?

Just curious, maybe I completely misunderstood your question (my apologies), maybe I'll learn something !

The link you provided (http://bayesoft.com/articles/normalpostmusigma.htm) is a solution to a different problem: no prior information is available. Not to mention that this solution is arguable according to D’Agostini (1998), “The default use of Jeffreys priors is clearly unjustified, especially in inferring the parameters of the normal distribution, ….” In the problem considered, we have prior knowledge that is represented by a normal pdf.

.

1- Agostini's comment is about the (mis)use of the "transformation invariant" prior (aka "Jeffreys' prior : p(s) = sqrt(det(I(s))) with I the Fisher matrix and s the parameter) ; the link above is about an approach proposed by the very same Sir Harold Jeffreys for the normal when the mean and the variance are unknown, but this approach is different from the so-called Jeffreys' prior. This is not to imply that this approach fits your needs, but just to make clear that Agostini's comment does not apply to this approach.

2- You say that "we have prior knowledge that is represented by a normal cdf" ; do you simply mean that your parameter to be estimated is distributed according to a normal law with unknown mean and variance or do you mean that you have prior knowledge that the (unknown) mean and variance of the (normal) distribution of your parameter are distributed according to a (bivariate ?) normal law ?

.

Let me point at the core difference between the "usual" Bayes' and the Jeffreys' proposal.

1. usual Bayes uses prior info in the form of a FINITE weight, reducible without loss of gnerality to probability distribution, called then prior pd.

2. Jeffreys' algorithm - also possible to be named Bayesian - uses the prior info in a form of INFINITE measure, which is a product of

2.1. (obviously finite) normal pd for the mean and

2.2. (obviously infinite) Lebesgue mesure on positive reals R+ = (0, oo).

3. In the case under considerations the resulting a posteriori info is a finite weight and it can be for further needs used in the "usual" Bayes' as a prior probability distribution.

Hopefully, this explains the acceptability ofJeffreys' formulas when there is NO INFO A PRIORI ABOUT THE SIGMA.

PS1. One can use any other algorithm, since statistics is based on our "impression" that we have about existence of some probabilities. In every case the results of the statistical inferrence are subject of a CONVENTION BETWEEN USERS; the mathematical derivation of algorithms is only a tool to transform principles into concrete formulas. But the principles (e.g. maximum likelihood) are our free choice supported by some elegant consistency with our image.

PS. Moreover, in every case when applying the probabilities to a single event, we are completely lost wondering why the things do not happen according to our expectations. The answer is: For a single event no probability calculus can help predict the result.

Fabrice Clerot

Regarding your comment #1, in his paper, D’Agostini (1998) essentially invalidated the t-distribution that is derived based on the Jeffreys' priors, which is the same result described in the link you provided. I don’t see the difference.

Regarding your comment #2, let me describe a real-world problem. We (hydrologists) have established a ‘rating curve’ about the relationship between river discharge (Q) and water level (or stage, H) at a hydrological station based on (many years) historical data, i.e. Q=f(H). The rating has uncertainty in terms of standard deviation. Suppose we went out yesterday to make discharge measurement at this station using an acoustic Doppler current profiler (ADCP). We made four transects of ADCP measurements (typically), which gave 4 observations of Q, under a steady flow condition. We took the sample mean as the measured Q and calculated the sample standard deviation as the ‘standard uncertainty’ of the measured Q. Concurrently, we measured the water level H and obtained Q(rating) based on the rating curve. We assume that both ADCP measured Q and the rated Q are valid (in fact, the rating curve may have a “shift”, but shift is not considered here and detecting “shift” is a different problem). The question is “how to combine the current ADCP measurement with the rating information (i.e. the prior knowledge about Q based on historical data)?” I think this is a “usual Bayes” problem. Note that the true discharge at the station is unknown. Otherwise, we do not need to make measurements. In addition, I don’t think a bivariate normal model is involved in this problem. By the way, it is strange that, to my knowledge and based on the discussions here, no statistics textbooks provide a Bayesian solution to the problem considered that is extracted from the real-world problem.

.

reread D'agostini's note : it basically states that the use of non-informative priors (such as the priors proposed by Jeffreys) "does not feel right" when the domain expert has some specific prior knowledge ; this is a possibly useful reminder but at the same time this is close to obvious as "non informative" priors are there to stand as priors when no prior information is available to the statistician !

.

the conclusion of the note is double-edged : on the one hand it reminds the practioner that "non informative" priors should not be used if the practitioner has prior knowledge and is wishing to leverage this knowledge ; on the other hand, it reminds the practitioner that embedding such prior knowledge into a prior is not an easy task and often comes at a high price, a price possibly so high that the practitioner is not willing to pay

.

so far, so good : if the practitioner has prior knowledge she values a lot and is ready to pay the price of technically embedding this knowledge into a prior and run the computations, there is **absolutely** nothing to prevent her from doing so

.

in your case, if i understand correctly, you have sequential data ; this setting illustrates the above dilemna, in a slightly different form : in such a sequential updating setting, the bayesian mechanics works very smoothly when you use conjugate priors ; your posterior at time t+1 (which will also be your prior at time t+1) has the same form as your prior at time t so updates have the same form at any point in time

for the normal with unknown mean and variance, this leads to the so-called "normal-gamma" prior

.

this is a conjugate prior, not a non informative prior ; it can be made "less informative" by choosing "vague enough" hyper-parameters ... at the cost of requiring more samples to see an effect of the data on the posterior

.

in a sense, conjugate priors are "worse" than non informative priors from D'Agostini's point of view as they do embed "some kind" of knowledge (they are not "non informative") and this knowledge may not be in agreement with the practitioner's prior knowledge

.

when using a different prior (non conjugate) in a sequential setting, the posterior will have "some form", and the next posterior "some other form" and so on, which makes the bayesian updating possibly very heavy from the computational point of view ; anyway, this is the price to pay to embed some specific knowledge into the prior (beyond the basic knowledge you can embed in the conjugate prior by choosing the values of the hyperparameters)

.

Fabrice is certainly right that a normal gamma is distribution and therefore is far from uninformative.

Furthermore you started out saying that the mean and standard deviation are known to be normally distributed

Can someone explain to me why Bayesian Statistics uses non-informative priors if they are really non-informative. Why not simply use the maximum likelihood term which predominates and leave the information-less prior behind.

The mean of a normal has no information if not accompanied by a standard deviation.You remark earlier that you have an unknown SD but later say that you gathered the means and SDs previously. The mean is effectively just a scaler which may be precise or may be very imprecise if the CI is very large. You have no probabilistic information if you use the mean alone, Without that knowledge, how can you possibly use it to weight the posterior probability? You remark earlier that you have an unknown SD but later say that you gathered the means and SDs previously.

I am confused.In Bayesian phylogenetic analysis of sequence data, "non-informative" data really are informative and can badly bias the results. In fact, of course, any badly chosen prior, badly biases the results. Use reliable prior knowledge if you have it.

I find the description of the problem confusing.

Information is nothing that exists objectively for itself, or is an attribute of something (like of data). Information is an emergent feature from the interaction of a signal (data & statistical model) with a reciever (scientific model).

The scientific model is a complex building, with reference to the statistical model, but neccesarily built on what scientists feel relevant, based on what they know.

Bayesian analysis takes data to change the state of knowledge. It does not apply when there is "no knowledge". If there is some knowledge, then this will define the information content of the data, and that needs a prior. There cannot be an "non-formative" prior, as any prior carries some idea about the model.

The likelihood is not about the model, it's about the data, expressed as a function of the parameter values used in the model. Only when the different possible parameter values are weighted, the appropriately weighted (and normalized) likelihood will merge the information from the data into a changed version of the weights. These weights are a probability distribution over the parameter space. We distinguish it as "prior" and "posterior" depending on the fact if the information from some current set of data is not yet or is already incorporated.

Using equal weights for all parameter values will result in a posterior that is just a normalized likelihood. However, such a weight function ("prior") is a very particular model of our knowledge, just like any any other prior, too.

The information we have is the sample data. That is all we ever have. It is information nevertheless and we use likelihood to try out parameters and associated distributions to find the combination that maximizes the likelihood of the data. This I know very well really. I also understand conditional likelihood very well and understand all else that you have said including how a posterior probability reduces to a likelihood expression. And so I am quite familiar with and use Bayesian statistics.

But in this whole discussion, I have heard that the the distribution is known to be Normal, that the mean prior is known (derived from earlier data) but that the SD is unknown: hence the use of a gamma prior distribution for the SD. Then I read that the SD was estimated afterall from the earlier data that gave rise to the knowledge of the mean. If that is so, then those values represent the normal distribution that constitutes the priors. So then I have no problem with using Bayesian analysis. Either way the prior appears to be a well-described. So what is the problem? None of this is particularly difficult under the described assumptions and is in many papers (cited already) and books.

Use the priors if you feel comfortable with them. If not then do a likelihood analysis based in the data rather than doing a Bayesian analysis with uninformative or poorly understood priors that could possible bias the outcome.

And please state the question more consistently.

The purpose of measurements is to gain knowledge about a quantity of interest. Prior knowledge about the quantity may or may not exist. It is intuitive that, if prior knowledge exists and available, it is ideal to incorporate the prior knowledge into current measurement. If prior knowledge does not exist or unavailable, we have to reply on current measurement only. In this essence, prior knowledge is independent from current measurement. The selection of a prior pdf should not be affected by the current data (knowledge) and should be determined based on the available prior information only. In the problem considered, we have known, and only know, the values of two quantities: mean and variance based on our historical measurements, which naturally leads to the selection of a normal prior. In contrast, the "normal-gamma" prior requires knowing the values of five quantities, three of which are not available. Thus, from practice point of view, the "normal-gamma" prior is inapplicable and the solution based on which has no practical meaning.

The problem considered in our discussion is very similar to the example given in A. W. F. Edwards’ classical textbook: “Likelihood”. Suppose we possess many years’ records of the atmospheric pressure in London at noon, April first. The series is long enough so that we can define the parameters of the probability distribution as normal with mean 760 and standard deviation 10 mm of mercury height. Also, suppose we make a measurement of the pressure today that gives a value of 770 mm of mercury height, with a normal distribution of error having a standard deviation of 1 mm. The question is: “how this information from the current measurement is to be combined with the prior information from the historical measurements?” Using the “Method of support” that is another format of the Bayesian theorem, Edwards obtained the posterior probability distribution that is normal with mean 769.90 and standard deviation 0.995 mm. In this example, the standard deviation for today’s measurement is 1 mm. However, it is unclear where this standard deviation of 1 mm comes from; it may be estimated from today’s measurement or it is an “subjective” assumption. Note that Edwards’ solution is based on the normal prior, not the "normal-gamma" prior. I think a normal prior is more practical than a normal-gamma prior. So, my question is: “Can the Bayesian approach relax the known-variance requirement for using a normal prior when the variance is unknown but estimated from data, instead of using the unpractical normal-gamma prior?”

By the way, sorry about the confusion I may have caused. The problem considered does have a specific prior information as indicated in my original post, like that in the Edwards’ example.

I am very, very familiar with Edwards and that book the source of a great deal of my knowledge. He has very interesting and not necessarily flattering remarks on modern Bayesian analysis per se. I am away from my library at present but seem to remember that he uses Bayes Theorem in the sense of a conditional likelihood.

Patrice Showers Corneli asked: " Can someone explain to me why Bayesian Statistics uses non-informative priors if they are really non-informative. Why not simply use the maximum likelihood term which predominates and leave the information-less prior behind. "

The above question with respect to the question of this thread is meaningless, since the prior is not NON_INFORMATIVE, but - as the questions says: we have prior info about the mean and no infor about the variance. Hence, the case is really described by some substantial prior information about the pair (m, sigma), thogh, just about m.

One can say the case is really noninformative only then,

if there were no information about both m AND sigma.

I realize this and i said this. I said that I don't see what the big deal is about the original question. If the prior of a normal distribution is a good one, then with the mean, one has all the information that is needed and using a prior should be absolutely straightforward. But that original post said nothing whatsoever about non-informative priors.

From the original question:

> When σ^2 is unknown, I expect to have a similar solution, i.e. simply replace σ^2 with its estimator s^2 in the posterior mean. However, I cannot find this solution anywhere. My question is: “does this solution exists?”

If anyone is interested in a development relating to the subject discussed, please see: "A new Bayesian method for measurement uncertainty analysis and the unification of frequentist and Bayesian inference" available on ResearchGate:Preprint A new modified Bayesian method for measurement uncertainty a...

My major concern about the Bayes Theorem in continuous form, i.e. Eq. (35) in the preprint, is that it is nothing but an operation of using a prior (distribution) to “modulate” the likelihood function to give a posterior distribution.  To my knowledge, however, there is no rigorous mathematical proof in the literature that the Bayes Theorem in continuous form is a valid operation.  It seems that it is largely considered as an axiomatic operation.  As I demonstrated in the preprint, the Bayes Theorem in continuous form violates “the principle of self-consistent operation”, so it is methodologically flawed.  However, the Bayes Theorem in discrete form, i.e. Eq. (34) in the preprint, is a self-consistent operation because it entirely operates on probabilities, so it is methodologically valid.  Importantly, the Bayes Theorem in discrete form can be derived, rigorously, based on the “product rule” and conditional probabilities.

Update: the paper "A new Bayesian method for measurement uncertainty analysis and the unification of frequentist and Bayesian inference" was published in Journal of Probability and Statistical Science, 20(1), 52-79. https://journals.uregina.ca/jpss/article/view/515.