The Bayesian theory of measurement uncertainty was first introduced by Weise and Woger in 1992. Since then, a large amount of papers on the Bayesian approach for estimating measurement uncertainty have been published. Importantly, the GUM is under a major revision and the revised GUM would follow a Bayesian approach. The Working group 1 (WG1) of the Joint Committee for Guides in Metrology (JCGM) is responsible for the GUM revision. According to the JCGM-WG1, “The first committee drafts (CDs) have been circulated to the JCGM member organizations (MOs) and the NMIs at the end of the year 2014. More than 1000 comments were received and the feedback on CD JCGM 100:201x was largely negative. The working group is working on responses to the criticisms received and elaborating a strategy for the development of a revised GUM.” According to Meija and Ellison (2017), “…, the working group acknowledges that the proposed GUM2 has failed to adequately communicate the rationale for revision of the GUM……. Replacement of the existing Guide is not envisaged in the short term”.
However, there are objections to the revision of the GUM based on the Bayesian statistics. For example, Willink and White (2011) stated, “It is our view that the GUM should be revised, but not according to the Bayesian philosophy.” White (2016) argued, “…that the move away from a frequentist treatment of measurement error to a Bayesian treatment of states of knowledge is misguided. The move entails changes in measurement philosophy, a change in the meaning of probability, and a change in the object of uncertainty analysis, all leading to different numerical results, increased costs, increased confusion, a loss of trust, and, most significantly, a loss of harmony with current practice.”
Some limitations and weaknesses of the Bayesian approach were addressed by Willink and White (2011), Attivissimo et al. (2012), Giaquino et al. (2014), Giaquino and Fabbiano (2016), White (2016), and Willink (2016). Willink and White (2011) addressed philosophical, computational, and performance issues associated with the Bayesian approach. One of the issues is that, “For the Bayesian, uncertainty lies in the experimenter’s mind and probability measures the degree of belief about a hypothesis. One of the major consequences of this view is that a measurand is no longer represented in the analysis as if it had a single well-defined value.”
It is true that Bayesian approaches become popular in recent decades in many fields of science. For example, “Bayesian methods are increasingly being used in the social sciences, as the problems encountered lead themselves so naturally to the subjective qualities of Bayesian methodology (Jackman 2009).” Indeed, Bayesian bias or Bayesian adjustment to observation with a prior thought is naturally acceptable in social sciences, because in which there are essentially no objective truth (so ‘bias’ is not an appropriate word there). If we admit, or assume, that true values exist in measurement science, the Bayesian approach may not be appropriate for estimating measurement uncertainty.
Various subjective views you mentioned and huge amount other existing ones can be unified/accepted/clarified/avoided ... by adopting decision making view on Bayesian methodology (estimation serves to some purpose: you act according to its results) ,
see
@BOOK{Sav:54, title = {Foundations of Statistics},
publisher = {Wiley}, year = {1954},
author = {L.J. Savage},},
@BOOK{Fin:73,
title = {Theories of Probability: {A} examination of foundations},
publisher = {Academic Press},
year = {1973},
author = {T.L. Fine},
address = {New York, London},
}
This view essentially state that when you are acting under uncertainty and incomplete knowledge you have to describe incompletely described quantities by probabilities. They are, before acquiring additional knowledge (from books, experts or measurements), inevitably subjective. Good news is that a new knowledge (say measurements) can correct it "objectively". In a finite sample situation the result is always the mixure of subjective and objective information. Asymptotically, if the used model is identifiable and data informative, you get "objective" frequentist answer.
Personally, I have started to understand Bayesian paradigm via asymptotic analysis
@incollection{BerKar:97
author="L. Berec and M. K{\'a}rn{\'y}",
title="Identification of reality in {B}ayesian context",
booktitle={Computer-Intensive Methods in Control and
Signal Processing},
publisher={Birkh\"auser},
pages="181--193",
year=1997,
editor="K. Warwick and M. K{\'a}rn{\'y}"
}
Essentialy: you are dealing with a set, typically parametric, models and reality
from which you get measurements does not fall into your model set. The only
think you can hope for that you will find the model which is the closest to the reality.
The asymptotic analysis outlined in \cite{BerKar:97} mimicks and generalises the results in
@ARTICLE{San:57,
author = {I.N. Sanov},
title = {On probability of large deviations of random variables},
journal = {Matemati\v{c}eskij Sbornik},
year = {1957},
volume = {42},
pages = {11--44},
note = {({R}ussian); in Selected
Translations Mathematical Statistics and Probability, I, 1961, 213--244},
}
All this implies that the prior distribution expresses your belief that a specific model in this set is the nearest one to reality, Bayes rule corrects this and asymptotically finds that model which is the nearest to reality while inducing Kullback-Leibler divergence as the adequate proximity measure.
Even in this favourable case you will never know what is the distance of your model to reality: you will know how far you are from the best projection in your model set ... you cannot hope for anything better as it is impossible to measure quantitatively any distance to u k n o w n reality (people in image processing call it ground truth) ...
Indeed, there is a long-lasting debate between frequentists and Bayesians. “The controversy spans in an enormous body of books, papers, and web pages, with a vast number of examples, demonstrations, counter-examples and refutations” (Attivissimo et al. 2012). “The philosophies behind the two approaches are so different that seems impossible to find enough common ground and understanding to enable a probable debate to occur” (Willink and White 2011). However, both frequentists and Bayesians agree that a statistical method should be judged by the result which it gives in practice (Jaynes 1976; Kempthorne 1976).
So, let’s consider a simple example that is often encountered in practice: estimating unknown mean and standard deviation from a sample, i.e. a multiple observation, drawn from a normal population. Here we do have “subjective” assumptions (information): normal distribution with unknown location and scale parameters and “objective” information: a sample. According to the frequentist approach, it is straightforward to use the sample mean as an estimate (mean-unbiased) of the unknown location parameter (mean) and a bias-corrected sample standard deviation s/c4 as an estimate of the unknown scale parameter (standard deviation). Furthermore, the Type A standard uncertainty (SU) is s/(c4√n) (e.g. Huang 2018). On the other hand, according to the objective Bayesian approach, the posterior distribution of the measurand is a scaled and shifted t-distribution. Consequently, the Bayesian estimate of the unknown location parameter is also the sample mean, but the Bayesian estimate of the unknown scale parameter is √(n-1)/√(n-3) s. Furthermore, the Bayesian Type A SU is √(n-1)/√(n-3) s/√n. However, based on our normality assumption, the “true” sampling distribution of a multiple observation is also normal, not a scaled and shifted t-distribution. That is, the true sampling distribution is distorted by the Bayesian transformation (Huang 2018). The distortion is so severe at n=2 or 3 that the Bayesian standard deviation even does not exist. In addition, the Bayesian Type A SU does not follow the -1/2 power law, the physical law for the relationship between the measurement precision and the number of observations (Huang 2018). Therefore, the Bayesian approach for inferring the unknown scale parameter or the Type A SU is invalid for this practice example.
In principle, Bayesian method is a transformation. Transformation distortion is inherent in Bayesian approaches, resulting in biases in inferences. Again, Bayesian bias may be naturally acceptable in social sciences or decision making, because in which there are essentially no objective truth (so ‘bias’ is not an appropriate word there). However, measurement is a science of physics, in which, there exist objective realities that are independent of our minds or knowledge (Huang 2018) (although this assumption itself is “subjective”). Thus, the Bayesian approach may not be appropriate for estimating measurement uncertainty as demonstrated with the above example.
Some references cited in my previous discussion:
Attivissimo F, Giaquinto N and Savino M 2012 A Bayesian paradox and its impact on the GUM approach to uncertainty Measurement 45 2194–2202
Giaquino N and Fabbiano L 2016 Examples of S1 coverage intervals with very good and very bad long-run success rate Metrologia 53 S65-S73
Huang H 2018 Uncertainty estimation with a small number of measurements, Part I: new insights on the t-interval method and its limitations Measurement Science and Technology 29 https://doi.org/10.1088/1361-6501/aa96c7
Huang H. 2018 A unified theory of measurement errors and uncertainties, Measurement Science and Technology https://doi.org/10.1088/1361-6501/aae50f
White D R 2016 In pursuit of a fit-for-purpose uncertainty guide Metrologia 53 S107–24
Willink R 2016 What can we learn from the GUM of 1995? Measurement 91 692-698
Willink R and White R 2011 Disentangling classical and Bayesian approaches to uncertainty analysis Measurement Standards Laboratory, PO Box 31310, Lower Hutt 5040, New Zealand
It is a misperception that, “… the use of prior information can only decrease the uncertainty associated with the measurand and will never increase it”. It is well known that Bayesian approaches inherently produce biased estimates. Because of the Bayesian bias, the decrease in “uncertainty” is misleading.
In principle, either the subjective or objective Bayesian approach can be regarded as a procedure of ‘transformation’ because the samples in the original sample space are transformed into the posterior sample space through the Bayesian transformation procedure. I revealed that the transformation distortion due to either the frequentist t-interval procedure or objective Bayesian procedure (also leads to the t distribution) causes a large bias in the estimated uncertainty with respect to the z-based uncertainty for small samples (Huang 2018). The objective Bayesian estimator of μ is the same as the sample mean that is unaffected by the objective Bayesian transformation; only the associated uncertainty is affected (biased and less efficient). However, the subjective Bayesian transformation affects both the estimated measurand value and the associated uncertainty.
I recently noticed that there has been a debate on the validity of using the Jeffreys non-informative priors among Bayesians. D’Agostini (1998), a leading proponent of Bayesian methods in particle physics, argued that “…it is rarely the case that in physical situations the status of prior knowledge is equivalent to that expressed by the Jeffreys priors, …” He also stated, “The default use of Jeffreys priors is clearly unjustified, especially in inferring the parameters of the normal distribution, ….” and, “I am much more worried by the attitude of giving up prior knowledge to a mathematical convenience, since this can sometimes lead to paradoxical results.” An important result of using the Jeffreys non-informative priors is the t-distribution. However, the t-distribution based inferences may lead to paradoxical results because of the t-transformation distortion. Some paradoxical results have long been known, e.g. the Ballico paradox (Ballico 2000, Huang 2016).
Ballico M 2000 Limitations of the Welch-Satterthwaite approximation for measurement uncertainty calculations Metrologia 37 61-64
D’Agostini G 1998 Jeffreys priors versus experienced physicist priors: arguments against objective Bayesian theory Proceedings of the 6th Valencia International Meeting on Bayesian Statistics (Alcossebre, Spain, May 30th-June 4th)
Huang H 2016 On the Welch-Satterthwaite formula for uncertainty estimation: a paradox and its resolution Cal Lab the International Journal of Metrology 23 20-28
The unification of frequentist and Bayesian inference is possible, at least in measurement uncertainty analysis, please see: "A new Bayesian method for measurement uncertainty analysis and the unification of frequentist and Bayesian inference" available from: https://www.researchgate.net/publication/344552280_A_new_Bayesian_method_for_measurement_uncertainty_analysis_and_the_unification_of_frequentist_and_Bayesian_inference?channel=doi&linkId=5f7fd8a5458515b7cf71d5ec&showFulltext=true
- Badges
- Science topic
- Similar topics
- Engineering
- Computer Engineering
- Computer