In undergraduate (and some graduate) statistics courses required for a degree in many of the sciences, it is not uncommon to find examples like height or weight following a definition of a “continuous variable” or a continuous variable that is normally distributed (height and weight of a population is a fairly common example for a variable that is normally distributed). The most common statistical tests used in (null hypothesis) significance testing assume that the variable in question is continuous (otherwise, there wouldn’t be an issue of how robust e.g., ANOVA or t-tests are to violations of normality assumptions). Let’s assume that human birth weight (currently or all humans- past, present, and future) is actually normally distributed or that it is simply continuously distributed (a far weaker assumption). The number of human births, even if we consider all human births that were or will ever be, is countably infinite. Therefore, there is some one-to-one function that can map all birth weights that were or will ever be to the set of rationals in the unit interval, and no possible one-to-one mapping from the set of all birth weights that ever were or will be to the interval [0,1] (the required interval for every continuous probability distribution). It cannot be that birth weights are normally distributed, but there is a more far-reaching issue here. Consider the probability that a randomly “picked” number from the unit interval will be rational. That probability is 0, because even though the rationals are “dense” (they satisfy the incorrect definition of “continuous” given in many an introductory statistics textbook that continuity means there are an infinite number of values between any two values in the set), they “fill-out” a negligible “amount” of the unit interval (they have measure 0). Thus whenever we say that some variable like “weight”, “height”, etc., is normally distributed we are asserting:
1) There is no interval of possible values this variable can take in which an irrational number doesn’t appear
2) The condition that between any two values there must exist infinitely many other possible values is wholly insufficient (alternatively, between any two values there are infinitely many rational numbers AND infinitely many rational numbers)
3) If we remove all rational values from the set of all possible values this variable can take (alternatively, if we remove all the rational points along the x-axis under the normal curve of this variable), what is left over is essentially the same (we have removed an “amount” of measure 0).
Given that often we treat as normally distributed variables that are actually far more clearly “discrete” than those like all present, past, and future birth weights, to what extent are we justified in doing so? Alternatively, to what extent are we justified in using as a basis for hypothesis testing or statistics more generally a formulation of probability theory that isn’t measure-theoretic (i.e., one in which the distinction between continuous and discrete variables is dismissed as artificial and unnecessary)?
Dear Andrew,you are seriously making a point which i think we need to look into critically
I will say Yes.But then you have raised an important issue that need serious suggestions in the last paragraph
There is nothing like a norml distributed variable in nature, just as there is nothing like a circle in nature. Or even an electron. These are all models. They are very useful, because they sketch or clarify some characteristics in some handy way where we can build further ideas upon or to make forecasts that are correct within a defined frame of circumstances/conditions and with a defined precision.
The normal distribution must further not be confused with a frequency distribution. It is a probability distribution and therefore is a tool to quantitate a "state of mind" rather than a "state of nature". There has been a long battle on this philosophical aspect but I consider it more or less "sloved" today. It is clear that probability statements are reasonable only when they are oriented at observed frequencies (as Aristotle said: "the probable is the frequent" - but this was in the context of poetry). But this is just a mapping between "mind" and "reality" based on "common sense", there is nothing like a direct, physical connection.
Continuity and infinity are also just mental concepts. There is nothing continuous or infinite in reality, and still these concepts prooved to be useful in great many applications.
The usefulness of any scientific model is defined (relative to its purpose) by its practical applicability. As Tukey said: "All models are wrong, but some are useful", and usefulness is a gradual thing. The (mental) concept of the normal distribution turned out to be very useful for inference. This is the justification for its use. In some cases it is obvious that this model is quite bad, and still it may be useful. It may be that other models would be better but they would also be more costly, so the question is: do I want to invest more (money, time, energy, etc) in using a "better" (probability or inferential) model or is it enough for my purpose to use a "worse" model and save time, energy, and money? This depends all on the purpose and on the required precision and accuracy of the inference.
http://plato.stanford.edu/entries/probability-interpret/
http://www.laeuferpaar.de/Papers/
https://www.rug.nl/research/portal/files/14565567/thesis.pdf
Dear Jochen Wilhelm:
1) With respect to "real" infinities, although this remains a seriously contested philosophical and metaphysical point, I think the denial of actual infinities is untenable given modern physics.
2) It was Box who said "All models are wrong, but some are useful".
3) Granted that continuity and so forth are mental concepts (and ignoring that there is no other type of concept that wouldn't make this description redundant), so are statistics, probability, statistical significance, and more or less the whole of concepts from applied mathematics, data analysis, and so on. Yet somehow these "mental concepts" are fundamental for the advances of the sciences and have been developed rigorously (and increasingly rigorously) for the past few centuries (as you are well aware). Simply asserting that concepts like these are idealizations, abstractions, concepts, etc., doesn't negate the fact that so are things like rigorous probability, elementary probability, and statistical theory, nor does it somehow remedy reliance on a probability theory that one violates so fundamentally as to accept as true the formal ("mental concept' or no) definition of continuity when this is assumption is violated in a way as to make the "area under the curve" equivalent to the same area were all the actual values the variable can assume removed.
Simply put, regardless of whether or not continuity and so forth are "mental concepts", unless you can suggest how we can integrate finite sets or even countable sets to make sense out of the formal basis for the foundations of the entirety of continuous distributions, then how could it matter if they are "mental concepts" or not? Also, granted that the "normal distribution" is just a "mental concept", what makes it better than a discrete distribution for variables that are clearly not continuous, let alone normal?
Dear Andrew, thank you for your corrections.
I think we absolutely agree that in practice is does not matter if anything is a "mental concept" or not. But is matters, to my opinion, for the question that has been asked: "Can things like weight or height be normally distributed?" - there the answer is "no" and the fact that the normal distribution is a model (concept, idealization, abstraction) is the justification for this answer. I also wanted to stress the point that probabilities are not frequencies (the measure-theoretic foundation of the whole probability calculus is purely mathematical/philosophical and does not give any hint about any possible interpretation of what probability is). Probability calculus is a tool that can be very efficiently used to model and derive expectations, and it can be calibrated (to be practically useful) using frequency distributions of actual observations.
It is not the values of measured/observed heights or weight or any other variable that is normally distributed. Saying so is a simplification in the wording we use. The probability distribution is a formal way to make our expectations about such values mathematically tracktable. Actually we would need to say: "The normal probability model is appropriate (according to the requirements) to formalize our expectations of values from variables like 'weight' or 'height' (given a precise scope and operationalization)". The appropriateness depends on our aims and needs. It may be appropriate for some applications but not for others.
I did not want to put this as a kind of a negative answer and wanted to emphasize that making models (concepts, idealizations...) are not only useful but also absolutly neccesary. All our understanding, our thinking, and our communication is based only on such models, even "reality" is a model/concept, something we only shape in our mind, and this is not restricted to a scientific point-of-view. It becomes philosophical (and in this way mathematics is a philosophical discipline) when we try to make these concepts logically coherent, and it becomes scientific is we use data to justify, fit, or value such logically coherent models.
Dear Joachim:
"I did not want to put this as a kind of a negative answer and wanted to emphasize that making models (concepts, idealizations...) are not only useful but also absolutly neccesary. All our understanding, our thinking, and our communication is based only on such models, even "reality" is a model/concept"
Well put, and I agree completely.
"The probability distribution is a formal way to make our expectations about such values mathematically tracktable."
Herein lies a subtle issue. We know from history and the cognitive sciences that our expectations, and indeed our judgments regarding probability and most of logic are false (that is, the human mind tends to reason erroneously with respect to logic and probability, and thus informally "expectation" is mathematically unsupportable and any sufficiently rigorously defined probabilistic expectation is bound to be incompatible with expectation as colloquially defined). In other words, expectation here is an idealization, or the kind of conceptual abstraction you refer to. Yet mathematically and pragmatically we can only employ said abstractions to the extent they are rigorously and consistently defined, and the definition of continuity satisfies both requirements, is required for even the ancient interpretation of integration as "area under the (in this case distribution) curve", and will fall apart completely if this concept fails to hold as required for continuous distributions.
Expectation, in terms of statistics and probability, is rooted in physical concepts from measurements of physical system or physics itself (whence comes notions such as probability densities and so forth). These are more intuitive in this respective domains, yet still require the same formal requirements of continuity to be meaningful. Simply extending the properties of continuous variables to those that fail to meet fundamental requirements, idealized and conceptual or no, removes the foundations for using continuous distributions or even defining them as such either as opposed to discrete or as opposed to ridding our frameworks of the distinction and adopting a measure-theoretic framework in which this artifice is removed.
Mathematicians in general are sympathetic towards the constructivist perspective, but we don't in general tend to accept Kronecker philosophy.
Dear Andrew,
"In other words, expectation here is an idealization, or the kind of conceptual abstraction you refer to." - yes, that's the point. In fact, we use probability caclculus to guide us from simple cases, where our intuition is transparent, to more complicated cases where our intuition easily fails.
"Yet mathematically and pragmatically we can only employ said abstractions to the extent they are rigorously and consistently defined,... " - I am not sure if I oversee something here, since you make that point so explicitely. I completely agree and I never wanted to give the impression that probability calculus was not rigorous or consistent. What actually is neither rigorous nor consistent is the "frequency interpretation" of probability, in which probability (of an event) is defined as a physical constant that can be estimated by the (relative) frequency of observations of this event.
I did not want to get snookered in Kronecker philosophy or anything related by saying that probabilities are "things in our mind". It was not intended to be taken so literally, and I agreee that is it not even relevant (in any practical sense) to consider. It seemed to be relevant for the question. As an analogy the question could have been "Can things like photons and electros be particles?". My answer would be similar: No, because "particleness" is a concept, and it is very useful for many purposes. But is is wrong to see an identity of concept and the physical object. The electorn is neither a particle, nor a wave, nor a quantum-mechanic psi-function (or whatever). These are all just concepts, like "shadows in our mind" that fit to the data we consider.
Dear George Stoica:
Even in elementary probability theory (the non-rigorous sort that doesn't incorporate measure theory), there exists a distinction between continuous and non-continuous variables (distributions). If what you say is true, then there is nothing better than what we have in even undergraduate level mathematics: no support whatsoever for any idiotic "approximation" of a continuous distribution (a fair coin toss is as close to continuity as are integers, and the rationals demonstrate how completely inadequate any infinite and infinitely dense set is when it comes to approximating continuity). Luckily, however, even in elementary probability theory and in elementary statistics we actually DO have something "better to put instead": finite probabilities. Is this wholly inferior to measure-theoretic probability? Sure. Is it better than the ludicrous notion of "approximating" infinity let alone an uncountabley infinite set? Yes.
Andrew, I am not sure if I completely understand what you said, but to me your answer seems too harsh. There is nothing infinte and nothing infinitisemal in nature at all. And yet the majority of extremely good models are based on calculus and are using the idea of continuous measures. I do not see a major concern in defining expectations on continuous measures, too, as long as one is aware that too sparce information may reveal a "granularity" that may not be ignored without considerable draw-backs (pretty much like the step from classical to quantum mechanics).
Dear Joachim:
Strict interpretations of quantum theory (both collapse and non-collapse) require physically realizable infinities. Negative temperatures (empirically realizable) require those "hotter" (in a thermodynamic sense) than any of infinitely positive temperatures. According to some, time and space are infinitely divisible.
Far more importantly, integration (required for any continuous variable) is impossible if Riemann integration is used on any countably infinite set, and Lebesgue measure of any countably infinite set is 0. There are infinitesimals strewn throughout modern physics, and infinities as well. I'm not sure what your basis is for the contrary claims. But thanks for the constructive criticism. It is always good to hear from those such as you and Dr. Stoica to refine my thoughts (however insignificant they may be).
-Andrew
I think perhaps the question "title" detracted a bit (or more than a bit) from the opinions I was interested in. It's not as if I am asserting that discrete variables can't be usefully approximated by assumption of continuity. Hence this portion of my original question: "Given that often we treat as normally distributed variables that are actually far more clearly “discrete” than those like all present, past, and future birth weights, to what extent are we justified in doing so?" I am interested mostly in others' opinions to THIS question, not really whether or not variables like height or weight can be normally distributed. In particular, I am interested in treating Likert-type scales as continuous.
The answer is provided by quantifying the ``discretization effects''. One way of doing that is by computing the identities that the moments of the discrete distribution are found to satisfy and comparing them with the identities, for instance, that the moments of the normal distribution satisfy. For example, it's known that the moments of the normal distribution satisfy the identities =(2k-1)!!^k, for all k, for one variable and similar identities can be found for more variables. (The generalization to applications in quantum field theory is known as Wick's theorem.) Therefore, if the moments of the experimental distribution satisfy these identities, within the precision of the experiment, then it's not possible to detect ``non-Gaussian'' effects. If not, it is. Similar identities hold for the moments of any distribution that can be reconstructed from them (i.e. which is normalizable and all of whose moments exist) and fully characterize the distribution.
Is the question about "things" or "measures of things"? This is a crucial distinction here because:
1) You cannot measure anything with infinite precision (mostly for practical reasons: life is short, there is no time to write down a number with infinite precision, and nobody would give us a grant for doing this, nor would we get a permission from an ethics committee for a study requiring participants to stay on the scale for ever [literally]). So all measures are, in fact, discrete, and any empirical distribution will always be different from the normal distribution. In case of height, the measurement is typically rounded to the closest cm or half cm. In the case of weight, the measurement is often recorded by a computer, so it can have several decimal places. For curiosity, I had a look at a recent measurement using BC-418, - the weight was recorded with one (1) decimal place. I'm sure this could be done much more precisely but even if we used all of our computer's memory, the result could still not be an irrational number.
2) The distribution of actual weight or height is a different matter. It makes sense to suppose that the actual weight and height of most objects would be an irrational number. (Why not? How would one object?) So the actual weight or height can be continuous and can be normally distributed. However, you will never really know.
:)
Kenn,
it might be interesting to look a bit deeper. It is clear that distributions of actually measured things is never normal, because measurements are of a finite precision.The values do not span a continuous range on the real line, but the normal distribution is a continuous model. And finally, if we do have observed values (with whatever precision we like to recor them), they are present, whereas other, unobserved values are not present. As a continuous model, the normal distribution does not assign a finite probability to any of these observations, so if the distribution of observed values would be normal we would need to state that we actually did not observe these values, because this is an impossible event. That clearly makes no sense.
But it leads over to the definition of a probability distribution. The probability distribution is a distribution of a random variable. A random variable is a quite technical term and means a function that is defined assigning a real value to any outcome in the sample space. If the sample space is uncountable infinite, the random variable is continuous and its distribution can be described by a continuous distribution function or density. The point here is, that there is no restriction on how this function (the random variable) has to look like. We have to decide what kind of function mapping outcomes to real values makes sense, practically. We may or may not map irrational numbers, the random variable does not care, because the densities for unmapped values are zero by definition, and any finite interval of the real line will still contain uncountably infinite mapped values when the sample space is continuous.
Take-home message: we do not talk about the outcome (the "attribute") but about the values we map to the outcome according to a particular rule (the random variable).
So the fundamental question is not whether or not the frequency distribution of observed values is normal. The important questions are:
- how do we map outcomes to real values (e.g. will we use the weight, or the logarithm of the weight, or the square-root, or the product of weight and age etc.)?
- what is a suitable probability model / distribution model to model the random variable?
The data are not the model. It is a mistake to attempt to reify the model. When a model is chosen to represent the data, additional uncertainty is introduced. The uncertainty of the model is not the uncertainty of the data. A statement of the model uncertainty for the data uncertainty is over-confident.
Restrict birth weight to full-term, live births. A normal distribution may be acceptable for representation of the data for the purposes of a particular investigation. The actual weights cannot be normally distributed, because there cannot be weights of zero or negative values. Indeed, it may not be possible to have a live weight of one-half the average value. Certainly, the birth weight cannot not exceed the maternal weight. Nevertheless, a normal distribution may be adequate and acceptably accurate for many purposes. The actual uncertainty will increasingly deviate with distance from the mean.
Continuous models have problems of specification and logic. Leibnitz introduced the infinitesimal with his calculus. The Greeks held the infinitesimal concept and found many logical paradoxes. Leibnitz introduced the derivative, a quotient of infinitesimals. Cauchy and others showed infinitesimals as impossible. Klein and others decided there was a way to keep infinitesimals so we could continue doing calculus.
We have very fast computers. It is now possible to perform all calculations using discrete models. We have only to decide what is small enough.
Jochen, I agree about your two main questions:
1) - how do we map outcomes to real values (e.g. will we use the weight, or the logarithm of the weight, or the square-root, or the product of weight and age etc.)?
My answer is to use weight, or any unit employed, only in positive unities. If there are negative values and positive values in same sample, the real mean U will be reduced. I guess that is one of the reasons why physics researchers created positive absolute temperatures (T+273) C.degrees.
2) - what is a suitable probability model / distribution model to model the random variable?
I think there are models (parametric and non-parametric) to map positive random univariate cases, that may be worked with calculus (via derivating, or via integrating). I proposed a plausible continuous model to fit small postive datasets limited by 2 extreme values; once we know the real media value of sample. (It asumes that each dataset value Ki has equal probability following Laplace premise -so each Ki is an average of an small interval-).
For the case of non parametric model, it is possible to build a model applying Lorenz curve, asuming also equiprobable values; then we get an Old Square Method polynomic model for that Lorenz Curve. Then derive it to get the cummulative distribution function in medias vs. probability P, CDF. The final real equation is K(P) = U* CDF in real values vs cummulate probability P.
I do not trust normal asumptions in general, because it starts from an inner theoretical pdf density function hard to imagine and to understand. The models I proposed to replace it also have precision fitting limits and uncertainties, and that is common in statistical induction, so I must be prudent and open to critics too.
OK. Hope it helps to Andrew Messing search for opinions about question.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis