I'm not aware of a lot of work (or anything at all rigorous) which shows a need to *decompose* different types of uncertainty. Understanding different types of uncertainty (the aleatoric/epistemic distinction is an example) is important in order to capture the main sources of uncertainty.

The problem with failing to capture and report uncertainty is not strictly "accuracy", it is"precision". The ability of your risk assessment to relate in any way to a "real world" amount of risk is limited by how precise and repeatable the risk assessment is. Risk assessments often massively underestimate their own uncertainty.

For numerical simulations, typical uncertainty that is often not captured includes:

- the assumptions inherent in the type of simulation (e.g. which things are actually simulated)

- the faithfulness of the particular model

- the validity of the simulation within its range of application

- the applicability of the simulation to this particular situation

- the trustworthiness of the simulation tools (including management of data)

Most of these can't themselves be quantified. That is, you can't assign them "uncertainty numbers", and put those numbers back into your simulation to come out with a final "uncertainty number" (oversimplifying here for effect).

Thank you Andrew for your answer.

A discussion of uncertainty decomposition into aleatory or epistemic can be found in:

Armen Der Kiureghian, Ove Ditlevsen:

"Aleatory or epistemic? Does it matter?"

Structural Safety 31 (2009) 105–112

There are several papers that K. Mishra and K. Trivedi have written on this topic. See for example the paper in int symp on soft rel eng. another in aiaa journal.

Let me know if you want copies of these papers

I would like to have the copies of those papers.

In general, it didn't seem to me that a common word usage for the same type of uncertainty exist.

For example, in the document:

"Guidance on the Development, Evaluation, and

Application of Environmental Models"

Council for Regulatory Environmental Modeling

U.S. Environmental Protection Agency

EPA/100/K-09/003 | March 2009

the types of uncertainty are defined as:

Model framework uncertainty

(function of the soundness of the model’s underlying scientific foundations)

and

Data Uncertainty

(arises from measurement errors, analytical imprecision, and limited sample size during collection and treatment of the data used to characterize the model parameters).

So far as I think, there is an common meaning on one side between Model framework uncertainty and the Epistemic Uncertainty.

On the other side, a overlap between the Data Uncertainty and the Aleatoric Uncerainty exist.

Here is one paper on the topic. Terminology may differ but the ideas and techniques in the attached paper stand.

It is important to note that the reason Der Kiureghian and Ditlevsen separate the different types of uncertainty is for risk communication, not for risk assessment. The purpose in separating is to make clear which types of uncertainty you choose to reduce or not reduce. My previous answer ignored this purpose. I agree that talking about different types of uncertainty can be helpful. In my public talks on risk, I often make the point by asking people what's the probability that the dice in my hand will roll a "6". When they answer "1 in 6" I point out their faith in my honesty, and ask them to consider how _sure_ they are that the odds are "1 in 6".

I maintain that if you are trying to quantify risk, it isn't important whether your uncertainty comes from "inherent randomness of the universe" or "things we don't know". If you're trying to communicate risk, in particular if you're advocating actions to reduce risk or to better understand risk, the distinction may be helpful.

The Trivedi approach which folds the epistemic uncertainty back into the probability calculations has a different purpose which (personal opinion here) runs contrary to the transparency goal of Der Kiureghian and Ditlevsen. It requires a number of assumptions which _themselves_ are unquantifiable epistemic uncertainties. The risk is that by quantifying some of the epistemic uncertainty, you give a false impression that this is all of the uncertainty.

Andrew makes a good point. True that models are an abstraction of reality and as such will have a lot more sources of uncertainty that may not even be quantifiable. Some of these are sometimes put under the umbrella of modeling errors, numerical errors (truncation, round-off etc. for analytic-numeric solution), sampling errors (especially for simulative solution). Nevertheless, those errors that can be quantified should be. That is my two cents worth.

Andrew and Bastian

Let us work further together on this topic.

Yes - just for clarity, I agree with what Kishor Trivedi says. I did not mean to imply that there was anything incorrect or unhelpful about quantifying what could be quantified.

(The amount that can and should be quantified is a matter of ongoing debate. I've just read an interesting paper by John Downer called "Disowning Fukushima" with an interesting sociological perspective on risk quantification.)

Instead of converting epistemic uncertainty into aleatory uncertainty, there are approaches (e.g. evidence theory) to treat them separately and propagate the hybrid uncertainty into the final system level risk/reliability index.

Attached is one paper with Enrico Zio using the simulation approach for the hybrid uncertainty treatment in distributed power generation system model.

We have recently developed another approach to deal with hybrid uncertainty based on universal generating function.

It will be nice to compare and contrast the methods I describe in my papers with the ones Yan-Fu Li and Enrico Zio describe. I can attache several other papers that I have on the topic.

After a quick look at Kishor Trivedi's paper, I think it is a very good idea to contrast the methods. In Kishor Trivedi's work, the epistemic uncertainty is modeled by probabilistic distributions and the classical probability tools are used to compute the expectation and variance of the final reliability index, given the parameters follows some probability distributions.

On the other hand, in the papers of Yan-Fu Li, we use fuzzy number to model epistemic uncertainty and the UGF/simulation to compute the final reliability index, which is essentially a random-fuzzy number.

It would be really interesting to compare the two methods.

So how do we go about it? Do you have the cycles or a student to do this?

I will also check with my student.

Dr. Trivedi. I can devote some cycles on this. Do you have any plan?

It may be better to communicate by direct e-mail? Mine is [email protected]

Nice to see, that the question lead to some interessting discussion and active knowledge exchange. I hope you will inform us according to your findings. Would be nice to get some information.

At the moment, I´m very busy due to the fact that I have to finisch my PhD until the end of the year. In general, I would like to contribute to the discussion but it is not so easy for me.

Nevertheless, I will follow this discussion. I would like to thank all of you for your contribution.

Bastian

yes if we do something, we will keep you and Andrew updated.

In the paper by T. Aven (University of Stavanger, Norway):

"Interpretations of alternative uncertainty representations in a reliability and risk analysis context", Reliability Engineering and System Safety 96 (2011) 353-360

Imprecise (interval) probability

Possibility theory

Evidence theory (Dempster-Shafter theory. Theory of belief functions)

Fuzzy probabilities

as alternaive representations are introduced and discussed.

I´d like to mentioned some deffinitions of the types of uncertainty (based on Oberkampf, W.L; Helton, J.C; Joslyn, C.A; Wojtkiewicz, S.F; Ferson, S.: "Challenge problems: uncertainty in system response given uncertain parameters", Reliability Engineering & System Safety, 85(1-3)2004:11-19):

epistemic uncertainty

(also refered to as

model uncertainty,

reducible uncertainty,

subjective uncertainty,

state-of-knowledge uncertainty)

aleatoric uncertainty

(also refered to as

parameter uncertainty

variability,

irreducible uncertainty,

inherent uncertainty,

stochastic uncertainty)

An overview of the existing uncertainty quantification methods is also presented in:

Riley, M.E.; Grandhi, R.V.

"Quantification of model-form and predictive uncertainty for multi-physics simulation"

Computers & Structures 89(11-12) 2011:1206-1213

Aleatoric or epistemic?

Does it matter?

presentation on the paper of

Kiureghian and Ditlevsen, 2008

http://www.ibk.ethz.ch/emeritus/fa/education/Seminare/Seminar08/PhD_Seminar_Sandomeer.pdf

Here is my two pennies worth about the problem.

In my opinion, what matters the most is whether the mathematical model you use to describe your uncertainty or information is adequate or not, in the sense that it describes it accurately.

As far as I understand it, Aleatory uncertainty usually designates the uncertainty arising from the fact that you want to describe the behaviour of a (large) population in which individual properties vary (in time, space, etc.). In this case there is little doubt that probability theory and limiting frequencies are the right tools to model it accurately, provided you have a sample large enough.

Epistemic uncertainty, on the other hand, means that your knowledge about some quantity/value is incomplete. It may concerns a population (in which case the sought probability characterising this population may be incompletely specified, e.g., given by an expert opinion) or the property of a single individual (e.g., the weight of a given apple provided by a quite imprecise balance scale). In this latter case there is a large body of literature questioning the fact that (precise) probabilities are always adequate to describe this uncertainty, see among others:

- Peter Walley, Statistical Reasoning with Imprecise probabilities, chapter 5 (The importance of imprecision)

- Decision-making process: concepts and methods, chapter 3 (Formal representations of uncertainty)

and authors following this line of thought have proposed models to complement or generalise (not replace) classical probabilities. It is interesting to point out that this question is already present in works of J.M. Keynes or G. Boole, so it is not really new.

So, to answer your question, my opinion is that the difference does not really matter (or, to be more specific, matters less) if you consider that any uncertainty can be modelled by probabilities, but does matter if you think (e.g., agree with P. Walley and others) that there are cases where your information may be insufficient to identify a probability, or cases where even assuming the existence of a unique "ideal" probability makes little sense (i.e., uncertainty about the property of a unique situation/single individual).

P.S.: as a side remark, I would also point out that the subjective/objective distinction is different from the epistemic/aleatory one. You may have a subjective source (an expert) giving you information about aleatory uncertainty (the failure rate of a family of components), and you may have an objective source giving you information about epistemic uncertainty (e.g., balance scales providing only information in terms of kilograms will give you an objective reading not changing from balance scale to balance scale, yet an item will have a specific weight, so your uncertainty about it will be epistemic).

CHAPTER 1:

"Philosophical and Theoretical Bases for Analyzing and Modeling Uncertainty and Ignorance" Bilal M. Ayyub

in:

"Applied Research in Uncertainty Modeling and Analysis"

edited by:

Nii O. Attoh-Okine

University of Delaware, Newark, Delaware, USA

Bilal M. Ayyub

University of Maryland, College Park, Maryland, USA

©2005 Springer Science + Business Media, Inc.

eBook ISBN: 0-387-23550-7

My opinion reflects practical experience with large and complex risk assessments. In practice it is wise to employ both categories of uncertainty from the design of the modeling system, through the analysis of its results. The benefits are not only to aid in communicating the analysis results, as was put forward by Andrew Rae, but also, to aid in clearly defining each uncertain quantity in the analysis. Risk analyses often model the evolution of a system over time, in which events can disrupt or alter the system's state. In these circumstances the idea of aleatory uncertainty is naturally associated with the uncertain future of the system (i.e., the occurrence of the events), and epistemic uncertainty reflected in the uncertain response of the system to the events. For example, IF an earthquake occurs (aleatory uncertainty regarding time, magnitude, ground motion amplitudes, etc.), how does material X respond (epistemic uncertainty regarding material properties, etc.). This structure can be a great aid when sorting out the definitions of uncertain variables and identifying the appropriate data to estimate each variable's distribution.

The terms 'aleatoric' and 'epistemic' errors are what used te called (more simply, but possibly less accurately) as'random error' and 'model error' - the latter saying simply that any model is no more than an approximation to the reality.

In the field of hydrogeology, there has been a lot of discussion (and argument) about the differences between aleatory and epistemic uncertainties, and their relative importance.

In my mind, aleatoric uncertainty is the uncertainty in parameters of a model, whereas epistemic uncertainty represents uncertainties about whether the given model and its inherent assumptions are correct or not.

As a relevant example, in hydrogeology, many modelers assume that the subsurface consists of a set of layered geologic formations, and that the governing equations for confined groundwater flow can be used, which assume constant water density, constant viscosity, etc... Aleatoric uncertainty in this context would be the uncertainty in the parameters of the model (e.g., the conductivities of the geologic units), and these uncertainties can be reduced by collecting data from pumping tests or other hydraulic tests. Epistemic uncertainties, in my mind, result from the fact that our model of layering may not be exactly correct (the aquifer could contain discontinuous, "patchy" heterogeneity), or that the governing equations for groundwater flow may not exactly apply (e.g., the groundwater could experience significant temperature gradients, which could influence density and viscosity).

Historically (at least in my field) there has been a lot of work on quantifying aleatory uncertainty, and less work on quantifying epistemic uncertainty. Parameter estimation and parameter error covariance propagation are standard tools these days (especially, the linearized approaches), but the tools needed to do some form of epistemic uncertainty analysis (e.g., multi-model comparison, Bayesian Model Averaging, Bayesian Model Selection, etc.) are computationally intensive and not used as often.

Personally, my feeling is that there is a bias towards showing that your aleatory uncertainties are small in research, and this is generally achieved by researchers through choosing very simple models (those with only a few parameters). This has the effect of "hiding" a lot of uncertainty in epistemic uncertainty and not acknowledging the fact that simple models can be very wrong.

I´m fully agree on your statement:

"In my mind, aleatoric uncertainty is the uncertainty in parameters of a model, whereas epistemic uncertainty represents uncertainties about whether the given model and its inherent assumptions are correct or not."

In my research about the modelling of concrete structure, i also use the uncertainty analysis considering the aleatoric and epistemic uncertainty. First of all, I choose serveral models for representing the same phenomena. This could be called pre-selection of models based on engineering knowledge. Afterwards, I´d like to quantify the difference between the model prediction.

On one side, I quantify the influence of probabilistic input parameter on the output prognoses (aleatoric uncertainty). On the other side, I compare the model`s prediction based on deterministic input parameters. This can be seen as an average "error" of the models based on their underlying assumptions.

The combination of both uncertainty sources is possible using the total uncertainty. Based on this, it might be possible to compare the prediction of several models considering both, the aleatoric and epistemic uncertainty.

In any definition aleatory uncertainty cannot be reduced (like the 1 in 6 for a "perfect" die). Epistemic uncertainties sometimes (but not always) can be reduced by more "research" improving the information position (like sorting out whether mr Rae has the habit of cheating on his students or is an honest man)

It isn't quite true that aleatory uncertainty can't be reduced - it just requires a change in reality rather than a change in information. My students can improve the epistemic uncertainty by _researching_ whether I have a habit of using a loaded die. I can improve the aleatory uncertainty simply by _using_ the loaded die.

In human exposure and risk assessment, "aleatory uncertainty" refers to inter-individual variability whereas "epistemic uncertainty" refers to lack of knowledge regarding the true value of a quantity or a true distribution. Personally, I do not find the term "aleatory" to be useful in this context, but it is widely used (perhaps incorrectly). In this context, it connotes that there can be inter-individual differences that are real even if we do not have the mechanistic models (at this time) to explain the underlying source of the differences. Even if we could later develop mechanistic models, the differences are still real. In contrast, epistemic uncertainty refers to our inability to precisely specify distributions of interindividual variability because of random and systematic errors in data or inference methods (including models).

The policy implications are: aleatory "uncertainty" (a misnomer, in my opinion) deals with the certainty that different people have different exposures and risks. Inter-individual variability is a critical factor in making decisions regarding whether a regulation or action is adequately protective of public health in terms of protecting sensitive or highly exposed persons within a population of interest. Inter-individual variability cannot be reduced, but given better understanding of factors that lead to inter-individual variability, a population can be stratified into more homogeneous subpopulations for purposes of analysis and policy. For example, it is common in exposure and risk assessment to focus on a sensitive subpopulation such as children, those with a pre-existing disease, or a particular socio-economic group that may have higher exposure and risk than other subpopulations.

Epistemic uncertainty pertains to decisions regarding what margin of safety to use when deciding whether a regulation or action is protective of public health, given lack of knowledge regarding the true exposures or risks. Furthermore, quantification or identification of epistemic uncertainty is a useful basis for identifying key uncertainties and determining research priorities.

These policy implications are well discussed in a National Research Council report "Science and Judgment in Risk Assessment" (1994).

The World Health Organization has recommended a tiered approach to exposure assessment that includes probabilistic analysis in which variability and uncertainty are simulated distinctly as the highest tier. (Guidance on Characterizing and Communicating Uncertainty in Exposure Assessment, WHO, 2006)

Cullen and Frey (1999) have provided a taxonomy of variability and uncertainty and an overview of methodological approaches in the context of exposure assessment (Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing with Variability and Uncertainty in Models and Inputs, Plenum Press).

To Frey: aleatory in my "context" means what it says: the uncertainty waht the next throw on with a die (alea iacta est) will be

to Rae. The only way to reduce the aleatory uncertainty throwing a die is making sure 1 face comes up always. Otherwise you change the distribution over the outcomes and make the probability of the oucome different for each outcome (ie not uniform as with a perfect die), but not the uncertainty about the next throw. If you use loaded dies the students have to observe you for a while (before you are thrown out of the casino). That would tell them if the die is laoded. But they still could not predict the value of the next throw with certainty (unless as i said one face comes up all the time, which is easy to establish)

To Ale: I am reporting that others use the term "aleatory" as I described in the context of a particular field, not that I agree with the term as it is being used in that context.

To frey: fair enough

Hi

I and my research team, after two decades investigation, proposed a new method for approaching the natural phenomena, called " the State Based Philosophy (SBP)". In SBP we derived simple, explicit, and exact solution for most of natural phenomena (change of state of systems). For example the fragility curve for a nuclear power plant which conventionally is developed as the result of more than 500 000 dynamic nonlinear analyses, is expressed by a simple equation.

In the proposed SBP every thing is exact, and all uncertainties (epistemic and aleatory which is inherent in conventional methods) have no place!

For more information look for state based philosophy and Persian Curve in the open literature and in the researchgate as well.

The aleatory uncertainty is as the result of tests. It can be reduced to some level by providing better conditions, to be more in comply with the nature, for the tests.

On the other hand, the epistemic uncertainty is related to the philosophy behind the formulation! It can be reduced by change of philosophy. The change of philosophy is possible only by upgrading of your knowledge from nature. For example, nothing may done in reducing the epistemic uncertainty in analysis of structures, which are clearly are not elastic. Another example is in analysis of fragility curve for assessment of safety of systems. The fragility is defined, wrongly, in terms of the hazard intensity. Nothing can be done except modifying the wrong understanding. The presence of epistemic uncertainty is in the current state of the most areas of human knowledge. The only way is revision of the current state of the art of human knowledge.