The ignorance of prior is sometimes called diffuse, uniform or non informative prior. It opposite is called informative. Sometimes, prior is constructed from the data themselves, by using the data to estimate the parameters. Thus, distinguishing the parameters is important for many reasons.
This means that you have assumed that extraneous information are correct which is not known with certainty. To me a common practice is to test the information for it validity, before estimation ; if the hypothesis that the information criterion/ stochastic restriction is true is accepted, the restricted OLS estimator is used, and if hypothesis is rejected the unrestricted OLS estimator is used.
If extraneous information is incorrect, an estimator incorporating information or pre-test estimator that is sometimes incorporates this information , will be biased. This complicates the decision to incorporate extraneous information because the reduction in variance from incorporation might be more than offset by the bias introduced.
In Bayesian analysis everything has a distribution, so say an estimate is made of a single parameter eg a variance- which is summarizing a distribution, then this too is thought to come from a distribution which has a variance, this is a hyperparameter. It is a property of the prior distribution,that is before you use the data, whereas the posterior distribution is the result of combining the prior distribution with the information contained in obtained empirical data. As you say you can impose a lot on the estimate by specifying a tight, narrow prior (with a small variance), or at the other extreme a flat or uniform prior, which does not strongly influence the results. In Bayesian analysis you have to specfy this prior distribution.- - it is your belief about the world before exploiting the data.
I have tried to provide a gentle and applied introduction to the rather advanced techniques for Bayesian regression model in a chapter in this, and I discuss and demonstrate the effects of different priors on the results for the posterior distribution of a regression parameter:
Book Developing multilevel models for analysing contextuality, he...
@ Kelvyn Jones , since incorrect extraneous information create bias and trade off variance and bias (usually through the MSE criterion) become a question of central importance in this context, how then can we then can this be resolved?
As always by thinking! And in practice by using a variety of priors (based on expert views or the literature or you reasoning) and thereby seeing how robust your results are. There is no automatic road to the truth, and you have to get into the Bayesian paradigm to appreciate the full impact of this. Again in practice, the Bayesian calibrated modes I use have good MSE properties through shrinkage ( and shamefully I often just use default priors)
Article Using Shrinkage in Multilevel Models to Understand Intersect...
and the papers in
This means that you have assumed that extraneous information are correct which is not known with certainty. To me a common practice is to test the information for it validity, before estimation ; if the hypothesis that the information criterion/ stochastic restriction is true is accepted, the restricted OLS estimator is used, and if hypothesis is rejected the unrestricted OLS estimator is used.
If extraneous information is incorrect, an estimator incorporating information or pre-test estimator that is sometimes incorporates this information , will be biased. This complicates the decision to incorporate extraneous information because the reduction in variance from incorporation might be more than offset by the bias introduced.
https://medium.com/.../comparison-among-hyper-parameter-optimizers-cd..
- Badges
- Science topic
- Similar topics
- Philosophy
- Philosophy Of Science
- Reasoning