The scientific method proceeds from observation to induction to deduction. That is the ideal. Deduction is rarely reached, that is, a situation where the logical propositions encompass the complete universe of a condition. Nevertheless, the inductive part of science requires knowledge in order to make an induction.

You pose the the problem of 'one must express the limited, or almost vanishing, information available by means of a probability distribution' under the condition of ignorance.

There must be information to pose a question that leads to an investigation. The frame work of the question implies a boundary or limit. An investigator must have a concept of the problem and the mechanisms or suspected mechanisms involved. The concept and the mechanisms bound the problem. Indeed, the investigator expects a result in given bounds and requires a result in given bounds in order to conduct the investigation. Otherwise, the distribution is uniform from minus infinity to plus infinity. It is the expected result that determines the prior distribution. (Note: This applies to frequentist and bayesian methods. You must have a prior, else you do not have a proposal to proceed.)

The requirement for a prior is essential to decision making. You must have information to investigate for a decision and you must have a distribution about that decision level. The distribution about the decision restricts the confidence in the decision.

Posing a question poses the distribution, otherwise the investigation cannot proceed.

Possibly you can go orthogonal to the question of degree of belief in a particular hypothesis, instead using the information theory formula. The information is the sum of the p of each possibility times the log of (1/p). This is a scale that has a maximum at the uniform distribution (probability distributed equally over all the possible events that you are considering). I have found it more useful to use this scale to compare possible questions to ask (is question A expected to reduce uncertainty more than question B?), so it may not address the original need that motivated the posting of this topic.

Franco, hi. It is normal in inductive statistics that we have N measured values of some variable, but have no idea of the probabilities of each one. Laplace asumed that in such situation it is sound to asign each event the same probability 1/N. That Laplace Criterion is employed as a first step in all inferences I know to estimate media U. It works better when dataset is big. Besides U as a data inner parameter, other ones like deviation has been designed for parametric models. If we knew distributon a priori, we would have perfect knowledge and there would not be the need of some kind of induction -or inference, or regression- as you prefer to name the process. I believe Laplace criterion is sound because we have no better option to start -unless somebody knows a better one and kindly shares it to us- Thanks, emilio

Emilio, the Laplace criterion also leads to the normal distribution and the mean as estimate of the expected value. The mean is in fact the result if each of the observed values is considered equally likely.

However, we must still keep in mind that the probability of the data is not equal to the probability of the hypothesis under which the data is thought to be generated. Here, a uniform (Laplcian?) prior is often used but it surely is not so often reasonable.

Thanks, Jochen for your remarks which I share -except the last one about the hypothesis- because the task is to analyze dataset as it comes, without questioning how it was gathered. Each value of data set gives a number, but does not bring a ticket sayng "my frequence is 0.25 and is different from other ones". Even more, we have no way to know if the estimated U media is bigger or smaller than the real one: this effects the deviation estimated and the inference as a whole. This may be simulated if we design an algorythm of CDF suposed to be an exact referent to generate a random sample. Actually, I am working a paper about it to be published next year. The spanish version is ready, but the english one takes longer. If it is of your interest I may send a private copy of the spanish version (14 pages). Regards, emilio.

Emilio, I clearly disagree here: "because the task is to analyze dataset as it comes, without questioning how it was gathered." - this is not statistics! We have to assess the information provided by the data, and this requires to know how the data was gathered. Anything elese feels like data-torturing...

Jochen, my position is that measurement problems are certainly important, must be solved aside and the responsibility to do this belongs to researchers that produce data and metrologists that work measuring devices and precision technics. Statistical analysts receive their datasets to analyze and interpret them, asuming they were obtained with precision and honesty. A good research must combine both aspects without mixing them, I think that way and see no reasons to change it. In anycase I welcome your kind disagreement. emilio

Dear Emilio, surely a statistician should assume that the data were "obtained with precision and honesty". That was not th epoint I was making. The analyst (be it a statistician or the researcher) should know quite a lot about the whole story, about the kind  of data (not its quality or precision) and the kind of information it can provide to judge what statistics might be sensible and what and how much one thus can learn from the data (recall that "learning" requires a context).

Jochen, I agree with your "kind of data" comments; it is always good that researchers know about both aspects. My point is that in certain point researcher must say: data is good enough, it is time to analize its statistical behaviour and other theoretical aspects related to main question of research in the defined learning context. We know that sampling is effected by randomness, but also by the limits of the model employed -which also adds some premises-. So we have two sources of uncerntainty. If we add the measuring precision one then we will have three sources of error to consider. In general I observe that we may approach our points of view. What I hardly oppose is the truncation of data to fit the model if we have the option of changing the model. Thanks, emilio

Emilio

As Jochen has said, data is not simply data. A good example is from the RG question

https://www.researchgate.net/post/How_do_you_transform_a_non-normal_set_of_data_into_a_normal_distribution?_tpcectx=profile_answers

in which it is asked how to transform a non-normal data set into normal. Jochen pointed out that the data set appeared to have a detection limit. Were it the case that there was a detection limit, the data quality is poorer just above the detection limit so that the data must be heteroscedastic. There can be several problems in the collection of the data that make regions of the data less reliable. This has nothing to do with the honesty or competence of the investigator. All methods of data collection must be known to the data analyst and the potential problems the with the data. An analyst can simply consider the data as set of numbers (like those at the end of a chapter in a text book).The result will be completely objective. The analyst will have let the data do the talking. Nevertheless, it might not be all the data has to say.

Joseph. To understand you better, may you give an example of size N=20 dataset made of positive values with detection limit problems? Randomness distorts small datasets, I admit, but some distortions come from applied methods and modelling premises. I fear some analysts blame data for their problems instead of checking their methods and premises in depth. Thanks for your comment. emilio

Emilio

An example is at the question I indicated, but I will add it here. The problem is not the shape of the distribution as much as the applied methods as you suggest. There are disparate practices for handling data near the detection limit. Some replace the detection limit and below with zero, some replace with half way between the detection limit and zero, and some provide the measured value. 

The problem is the uncertainty in the values. Some definitions of the detection limit are at a minimum uncertainty of 60% to100%, others may be as good as 30%. These are the minimum uncertainty and account for only the uncertainty of the detection limit model. The total uncertainty is larger. Based on most detection limit models, a value that is 10x the detection limit is about 10% uncertain. 

Data taken near the detection limit (any thing less than 10x the detection limit) have increasing uncertainty with decreasing magnitude. Analysis must take this into account. The attached data have no indication of the uncertainty.

My contention is that you cannot start with the data without knowing how and why the data were collected. Knowing how and why you have a form of a prior.

You state, " I fear some analysts blame data for their problems instead of checking their methods and premises in depth." I agree.

Joe

I waited a week to understand if the meaning of my question was correctly understood. I thank all of you who have replied, but I think you did not catch my question.

I preferred to use the word “ignorance” to stress a limited level of information that obliges the experimenter to declare that he/she is not able to estimate a value for the expectation of the distribution of the disperse values he/she gets, This is more common that you can suspect in experimental science.

In these cases, the way-out is different depending on the preferred statistical frame. Limiting myself to the probability frame, the most common solutions are: for the frequentist frame, to assume a uniform distribution with zero expectation because a non-zero value cannot be estimated; for the Bayesian frame to assume a uniform prior distribution for minimal conditioning the posterior distribution determined by the actual available data. As you see, I am not talking of issues concerned with the limit of detection nor with the number of observations.  The above may looks a sufficient solution. However, it is not, because the uniform distribution cannot extend from minus to plus infinite, but must be truncated.This means that there is a minimum information required also in this case. If a minimum information is required, can we give for granted that the uniform distribution is a ‘neutral’ solution, or  could it be better modelled?

I admit that it is not a an often-placed question, but, in my opinion, the uniform distribution is too often misused, However which other more correct solution can be used in these cases of basical ignorance about a possible probability distribution?

I agree that the uniform distribution is often misused, even abused.

I do not agree that the experimenter is not able to estimate a value for the expectation of the distribution. An experimenter believes, knows, expects, or predicts the outcome of the experiment, otherwise there is no basis for the experiment. Expecting an outcome does not eliminate the possibility of a surprise. A surprise calls for an evaluation of the methods, then if the surprise is real, an evaluation of the expectation of the experiment. Whatever the result, it may be easier to use a uniform distribution in the initial analysis. Nevertheless, the uniform distribution must be bounded by the reality of the the experiment and its products.

It may also be easier to perform analysis using a truncated distribution. The truncation must be within physical bounds dictated by the experiment and its products.

What other distribution can be used that is more correct in the face of ignorance? What ever distribution that will allow ease of analysis within the physical bounds of which the experimenter is not ignorant. It is then the job of the experimenter to determine the posterior distribution.

Thanks Joseph and Franco (I could not download the file). I observe that uniform distribution is one more parametric-distribution asumed a-priori, so it is very easy to abuse with that tool. The information gathered is highly limited by the measuring device; however it contains some information only valid for an small fraction of possible events, so the analysis is incomplete, restricted to a tiny region and doubtful, but it can be made if the researcher informs its limits from the start. This is a case of poor measuring plus poor data analysis that conduce to poor quality inferences and conclusions. emilio

Sorry for my absence from the discussion, but I was busy organising and attending the AMCTM 2014 Conference (look at my newly added contribution).

Obviously one does not expect to have to use a uniform distribution for the measured influence quantities of an experiment. Limited to that, I agree with your comments.

However, the issue is that almost universally you have also a number of influence quantities that you do not necessarily directly measure, pertaining to the category of those that are responsible of systematic effects. If they are estimated to influence the result, their precision is also relevant for the resulting value of the measurand. In the normal procedures their effects must be "corrected".

It happens that the information about some of them may be so limited that you can estimate only a likely maximum range for their effect.

A very frequent case in chemical-physics is that you must obtain the value of a measurand being a pure substance, in the presence of impurities affecting the value of the measurand. You can have either the case that you do not have a reliable or complete assay for your specific sample, but, at best, only a batch evaluation or literature data; or, that you to not have enough information about the sensitivity of your measurand value to each of these impurities (or isotopic concentrations), or that the literature information is insufficient. In these cases, it is usual to avoid to make a correction, meaning that you assign the value zero to the correction, and associate to it a component of uncertainty computed from a uniform distribution between estimated maximum bounds for the overall impurities or more specifically for some.

In a few cases, you know that the effect of all impurities have the same sign, e.g. all lower the measured value. In this case, the procedure is one-sided. Often this case is overlooked and the previous one is applied instead.

The above also means that, in general, a uniform distribution is bounded by an estimation of a maximum range of variation arising from the instrumentation of from physical or chemical facts (e.g.,at least,  zero-full scale range of an instrument).

Is the simple uniform distribution always the best solution?

But think now about the Bayesian frame, where, at least in metrology, the non-informative prior (i.e. indicating equi-probability of all values) is used and abused. It means that NO prior information exist. Should not this mean that all the information comes from the actual data treated? But this fact would coincide with the way the frequentists treat the information (and are criticised for by the Bayesians). So in this case, should not the results of the Bayesian treatment need to coincide with that of the frequentist treatment?

Incidentally, if instead the influence of the data treated on the posterior probability distribution is negligible, that should mean that the prior information is dominant. However, since it is taken as an unproved hypothesis (the prior), would not it mean that the assumption behind the prior remains unproved by the new mesurements?

Franco, "the non-informative prior (i.e. indicating equi-probability of all values)" -which is widely applied as a first step to deal with samples- was suggested by Laplace. It permits to estimate the expected media of sample and to estimate a non-parametric distribution for the sample. The problem is that parametric models use this expected U to build other distribution structures and forget conveniently that they used Laplace criterion, to save their a priori parametric models that appear as "posterior distributions"  and do not fit well in graphs with datasets.

You mention too some cases of signal noises and precipitations of matter (which gives two possible states); they require different methods to solve their particularities. I am speaking only of simple univariate positive datasets.

So, in my opinion, the Laplace criterion is sound -given our initial state of ignorance and uncertainty- and I repeat: it  produces a prior distribution that has nothing to do with the posterior distribution found with parametric methods.

Well, it is my vision and I may work it out with real samples. Thanks, emilio

Emilio, consider you estimate a "concentration" c, that can take any positive value. For convenience let us consider that there is also a known upper bound, for instance by the solubility constant. Let us call this upper bound U.

Following Laplace, as I understood, a prior on c would have a uniform density of 1/U.

But since you do not even have an idea about the order of magnitute of c, this may be an unreasonable prior. When c is tiny, say in the range U * 10-10, it would be a concernible difference if I look in the interval of  (3...4)*U*10-10 or at (4...5)*U*10-10, what is almost wiped-out with a uniform prior in (0...U). If I have no information about the order of magnitude, I have to expect a value in an interval of a given size d with higher probability, the closer the interval is at zero.

A proper derivation shows for that for d->0 that the probability (density) exponentially increases for c->0. Thus, a uniform prior on the log scale would be resonable. But I have not found the connection to Laplaces principle of insufficient reason.

Jochen, let me pose another example somewhat more precise. Supose it follows the exponential shape CDF, K>=(x)  =  U# * (0.01x^(0.01-1)) taken from a Paretian theoretical distribution for descending order of variable. Supose the mean U#=1000 and an upper bound of the measuring device, B = 30# (# are real units). It is a highly dispersed distribution like your example.

If you make N measurements you will find that 17% of the measurements hits the upper bound of the instrument and 83% of them are valid measurements.

From analysis it is also observed that the mean of valid lower 83% measurements gives U(0.17

Emilio, the field of applications is so wide that it is apparently difficult to make an example understandable to all.

In my example there is no "precipitation of matter" nor "signal noises". I was talking of impurities affecting a substance whose properties you must evaluate as an ideal pure one, so you need to estimate the effect of the impurities, a case that should be quite general.

I do do not feel stopped to think about the issue only because Laplace suggested a certain use of a uniform distribution: I always maintain a critical mind on any recipe. Recalling the use by Laplace is not, for me, a reply in itself. There are many statisticians unhappy for the ample misuse of the uniform distribution: Jochen made a case.

In all instances, if a Bayesian has no prior information, the situation becomes identical to that of a frequentist, because the posterior will only be based on the actual sample.

Modeling ignorance is, in my opinion, generally more difficult and contentious that modeling existing information, also because is 'complete ignorance' actually possible?

Franco. I accept your correction about precipitation and noises.  But I have no reasons, neither information to think about your critical mind.

One thing is to have some prior information, another one is to consider it as a premise to build a particular model of a dataset. I use Laplace Criterion as a premise because obtained results and graphs fit much better to datasets than other methods, and because so far I have not found better alternatives. If someday anybody gives me a better one in fitting and general isomorphism with dataset, I would be glad to accept it instead of Laplace premise.

The question deals with modelling (partial) ignorance not total one. Once we recognize that frequences related to dataset are unknown, we must face this problem to develop some approach as a model solution. The existence of registered memories, measurents and datasets means that there is prior information, but it is not sufficient to start the process. The missing piece is related to frequences and distribution of chances.

Why do we take the effort to do it? Because humans need to interpret the world that surounds us and transform it into actions, justifications and explanations. Thanks, emilio

Franco, one year later I have a proposal of a model where you know U(mean), and the two extreme values maximum and minimum Kx, Kn, obtained from sample. It may be applied to small sample with higher risk of uncertainty. The point is that when we have only two extreme datapoints, the media becomes U=(Kx+Kn)/2, just the middle point, so U is equaly spaced from Kn and Kx. But if you have more data points, U changes and may be placed at different intermediate points. This case requires a different model from the uniform distribution.

I include here the 10 pages paper that I prepared for an event in november and was accepted. Summary is:

Summary

Title: Parametric induction applied to small samples –Method and model with inverse function-   Author:         Emilio Chaves  Country: Colombia, South America

It works the proposed model of title, using media as unique parameter plus two data extreme values: the maximum and minimum ones. It answers common question: How can we interpret small samples at the beginning of research? Main advantages are: 1) it separates dimension from structure; 2) it has inverse function and is coherent. It recommends the help of high level mathematitians to: 1) Evaluating use of proposal in education; 2) Not using any parametric model without inverse function; 3) Searching comprehension and rigour in applied statistics; 4) Declaring if methods that apply Normal distributions are useful or not. This article sustains they are not, so they should be eliminated from curriculum.