It is the law of large numbers. The two first factors you mentioned will change randomly from a trial to another so their effects will be balanced.

Do not confuse probability distributions and frequency distributions. Probability distributions represent a "state of knowledge" we have about a system. This essentially has absolutely nothing to do with actual outcomes. However, our state of knolwedge can be better or worse. A comparison with the actual frequency distribution (i.e.: data) can tell us if (and even how) we can modify or state of knowledge to have more sensible expectations.

Think of a coin lying on the table. One side is up. As long as you do not know which side is up, your most reasonable "state of knowledge" is expressed by a Bernoulli probability distribution with p(head)=p(tail)=0.5. It has nothing to do with a coin toss! The only relevant thing is that you have not enough information to say with certainty which side is up.

If you consider a coin that is tossed in a particular way, not very high, with well-known starting conditions, you may prefer one side over the other. Note than here you know many things more than before, although the interesting outcome remained the same as before (side showing up). You can think about a reasonable degree of preferrence, make some edugated guess. If you have the opportunity to observe some of such tosses you can use the observed frequency distribution to "learn" and modify the probability distribution.

If you have complete knowledge about everything (all conditions) and correct physical models to describe the entire toss of the coin, you can predict the ountcome with certainty. The Bernoullid distribution will have P(head)=1 and p(tail)=0 (or the other way around). There is no need to talk about probabilities, there is no uncertainty anymore, the entropy of the available information is zero. This is the other extreme case.

So taken together: P does not really depend on the variable or the outcome(*). P essentially depends on your state of knowledge you have about the system.

(*) Well, sure somehow is does depend. The domain of the outcomes is given (statistically the sampling space), and probabilities should only be above zero for events within this domain. And some basic rules allow us to provide a reasonable kind of distribution from very simple assumptions (in the coin example is was the Bernoulli distribution).

Thank you Sirs.

Jochen Wilhelm Sir. I was saying that is there any imposition by any theory/law of statistics that ensures that for large number of trials(also how large?) the outcomes MUST follow say a Bernoulli distribution? Why doesn't it happen that somebody tosses a coin 1 lacs times & gets p=0.8 & q=0.2 (say)?

Mhamed Ali-El Aroui Sir, is ther any such law of large numbers, please share.

Also what ensures that there will be 2 outcomes only (because we are not using any physics here that the coin can't rest at its side(neither head nor tales))?

Occurrence of head & tail in coin tossing experiment is equally likely means if u go on tossing coin larger number of times then the proportion of occurrence of head and tail will be tend to be half -half.

As far as I understood Your are trying to test how many tosses have to be done in real experiment to identify that Your coin is close to "ideal" coin? Probably this will help (please see attachment)?

Achal Singh sir, my question is why is it so?

Actually it also crucially depends on something else, to wit Heisenberg's uncertainty relationship, which randomizes the outcome regardless of all other factors.

John Barrow cites the example of a billiards game: if you took theoretically infinite pains to always apply the exact same force and momentum with the exact same cue to the exact same spot on the exact same balls, Heisenberg's uncertainty would lead to a totally new game after the very first hit , and the spatial uncertainty attached to the location of the ball would be bigger than the width of the billiards table after the 12th cue stroke.

This would equally apply to individual coin tosses.

Thank you very much H Chris Ransford Sir, you actually got my question.

Thank you Albert Batushansky Sir.

I think still the root of the confusion is about seeing "probability distribution" and "frequency distribution" interchangeably.

The probability distribution summarizes our expectations, given insufficient information. The aim is to have as reasonable expectations as possible, and statistics provides methods to derive what "as reasonable as possible" is, given any available information.

Our knowledge about the side of a coin showing up when we would observe the coin can be represented by a Bernoulli distribution. It has nothing to do with repetitions or the law of large numbers. The distribution follows from the given conditions (knowledge) that there are only two possible, mutually exclusive values that could be observed (head or tail). The distribution does NOT say that is "is" equally likely that head or tail is up. The distribution has a parameter, P, that gives our relative expectation of one of the two possible results. Assinging P(head)=0.5 simply means that we do not prefer any of the two outcomes. This parameter choice reflects the highest possible ignorance, the distribution with P=0.5 has the highest entropy and represents the least specific knowledge.

If we have some particular extra knowledge, we may consider a different value of P, favouring one of the sides. P can have any value between 0 and 1, it is not fixed at 0.5. The optimum value depends on what all we know about the system.

In cases where a large number of similarily different systems can or will be observed, it turns out that the relative frequency of the observations is the best choice for the probabilities, defining what we should expect from yet unseen systems or experiments.

A coin toss and a billard game can be done to end up with always the same identical result. Heisenberg will only matter when the unavoidable microscopic uncertainties can evovle to impact the macroscopic system (e.g. in a billard game with zero friction so that the ballr run for an infinite time, until all balls eventually fall in the holes). In this case the uncertainty about the result is as large as the uncertainty about the quantum behaviour (i.e. it is maximal) and have no reason to prefer any possible outcome (well, it's a little more complicated in a billard game, but for a coin toss, being tessed infinitively high, the most reasonable P to assign is 0.5..

If soneone tosses coins and I have to find out what I should expect about the result for the next of his coin tosses, I would look how often head came up previousely. If this was very often, say in 80% of the cases, a reasonable expectation would be P(head)=0.8 and P(tail)=0.2. The reason might be that he is tosisng the coin in some special way so that it happens more often that head comes up. Or is was simply a strange series. If I have reasons to beliefe that he can flip the coin quite coordinatively, forcing one side to come up, I would set P(head)=0.8. If I know that the peroson is not able to do such tricks, I have no more useable knowledge than befor and would remain with not preferring any of the sides. But still, if the person managed to have 80% heads in a series of 1000's of flips, I would consider it a better strategy to set P(head)=0.8 to express my expecation for the next flips.

It all depends on the knowledge and not (or only very vagely and indirectly) on the properties of the coin or the tosser.

Thanks

Maybe your heading question should be reformulated as "Is there a correspondence between probability of an event & its actual frequency?". In its initial form, your answer has a trivial answer: It is about the correspondence between the argument of a function and its value in that argument. In my formulation, the answer is given by central limit theorem (in its simplest form (Bernoulli), the relative frequency of an event converges toward the probability of that event", which explains why over the long run the actual relative frequency comes to approximate the mathematical probability with such an accuracy. Probability is a (mathematrical) measure. It measures events, provided that they belong to a certain mathematical structure (Boole algebra). In this measurement, the physical factors are ignored, as we cannot account for them on one hand, and on the other hand that the physical evidences that would favorize an outcome or another are "simetrically balanced" or "there is no evidence which to favorize an outcome or another". Probability measures the "possibility" of an event happening, but it is not the only measure we can design for this purpose (see the classical interpretations of probability). It measures events in an ideal structure (where the physical factors you mention are ignored). Without the "equally-possible" idealization, the mathematical probability does not make sense. Somehow, the difference between probability and actual relative frequency has the nature of the difference between probability and possibility, between theoretical model and the investigated physical system, between abstract and empirical. In certain interpretations of probability, the stabilization of relative frequency over the long run is given as an axiom and raise philosophical questions on the concept of probability. It is very likely to have one head at each two tosses in average, but no physical law would prevent the coin to land heads 1,000 times in a row. Still, this never happened. If this would happen, over a longer run (even though infinity) this relative frequency will decrease so that to match the mathematical probability.

Catalin, I think you make all this much too complicated, presumably because you say that "the relative frequency of an event converges toward the probability of that event". This implies that "probability" is something (somehow physically) existing that could be estimated or measured by relative frequencies. But I think that this is a misconception that leads to many problems. Probabilities are measures of "mental structures", not of "physical structures". However, it would make sense if you turn it around and say that probabilities are sensible or reasonable when they are similar or equal to relative frequencies. Or, even better: probabilities express our expectations about (lon-run) relative frequencies. Both statements are applicable only for cases where one can think of frequencies at all. But there are many cases where this is not the case and where we still can (and do) assign probabilities. I then often hear people talking about "theoretical populations" with some "theoretical properties" and the data being some "random sample" for such "populations"... all this sounds pretty much like metaphysics or "statistical alchemie" to me.

Jochen, probability can only be mathematical and it is standard to use the word "probability" in sens of the Kolmogorovian account. When I say "probability" I refer to the ideal mathematical probability as a measure on a Boolean field of events with the three Kolmorogovian axioms. There are other (scientific) interpretations of probability which still preserve its character of measure or at least pseudo-measure (for example, obeying only finite additivity). Among them the frequentist interpretation, which, as you noticed, requires collectives and not singular events. The only interpretation not requiring collectives and allowing the asignment of probability to singular events is the propensitic probability. It is not clear to me what you mean by "mental structures", however I said probability is applied to mathematical structures, not physical ones. Probability has a special epistemic status among the mathematical concepts when it comes to application to the real world and we must consider all its interpretations (including the frequentist one) for gaining insights about its nature and how it really models the uncertainty.

Yes, Catalin, I see what you mean and I essentially agree. But I disagree to the point-of-view that the Kolmogorovian axioms tell us what probability is (I understood you this way, maybe wrongly). They just set up the rules how to treat probabilities mathematically. The frequentistic interpretation, in contrast, is an attempt to declare what probabilities are, but it has major draw-backs. I think I is eventually not really helpful to follow this interpretation, but to calrify that in those cases, where frequencies may be obtained, an assignment of probabilities is optimal when the assigned values are as close as possible to the frequencies.

These discussions all sound very complicated. I have also wondered why the odds for a balanced coin are 50-50, in terms of Newtonian mechanics. I submit a very simple symmetry argument. For any initial position, velocity and physical environment, the outcome is theoretically predictable, although horribly complicated to figure out (I don't understand what Heisenberg has to do here; this is not quantum mechanics). However complicated these calculations are, I think we can assume/agree that initially reversing the position of heads and tails in a perfectly balanced coin (keeping all other initial conditions unchanged) will reverse the outcome. It is then reasonable to assume that for every initial condition, the position of the head and tail sides are equiprobable, from which the overall 50-50 result follows. Clearly with any sort of bias (e.g. loss of equiprobabilities for head and tail initial positions) the overall odds will not be 50-50.

Marc, odds are always 50-50 according to the definition of "odds" in a probability field where all physical aspects are ignored (or, if you prefer, they are "simetrically balanced" for the two faces of the coin). If you succeed in quantifying the physical conditions of the coin (add also the medical condition of the one who tosses it, as a hand tremor per a specific face up might change the outcome), you will get a different probability field and perhaps different odds. Again, we can only talk about probability in mathematical terms. And physical conditons cannot be accomodated for a coin tossing, unless someone proves the opposite.

So, well, at the end it is again only about what we KNOW about the toss, about the INFORMATION that we can use to predict the result. Ignoring physics boils down the problem to pure geometry, and there is no information other than symmetry we can use. However, knowing and considering the physics - together with the starting conditions - may help us to give a better guess.

http://www.youtube.com/watch?v=7KFsZ6-1wiA

http://www.youtube.com/watch?v=7KFsZ6-1wiA

Thanks all for giving valuable inputs. Please tell if there is a theory f LARGE NUMBERS that suggests that for large no. of occurrence of an event the frequency of outcome converges to the probabilistic value?

Mohd, more precisely the theorem states that for a sequence of independent experiments performed uder indentical conditions, the relative frequency of an event (that is the ratio between the number of occurences and the number of experiments done) converges toward the mathematical probability of that event (Bernoulli).

Thank you. But how do we take (what is the criterion) identical condition given the fact that so many factors affect an outcome(please refer my question). Also, what is the AUTHORITY, or mathematical tool rather, that ensures that the theorem holds good. I mean like the numerous physical laws that are followed by bodies, elements are supposed to be imposed by God or Nature, what ensures that a theorem shall be followed by numbers? Is it merely by using Mathematical induction or sampling or any other statistical tool?

Catalin, it would be nice to get a reference. I doubt that there exists a proof for this. Especially for "indentical conditions". If an experiment is repeated under identical conditions, you will always get the same result. The key is that the conditions are *not* identical, that they vary (within certain limits) and that we do not know the exact conditions. The probability distribution reflects this "lack of knowledge" in the experiments/physical context. Another dimension opens for sampling strategies, when the "experiment" is to take a sample from an existing, defined, fixed population (of items or subjects). Then, identical conditions would mean take the *same* sample over and over again, what again will give always the same result. Again the key is that we have a variability in the sampling process so that we can not know what particular sample will be taken. Going further, if one assumes (!) a particular frequency distribution for the sampling, then one can derive frequency distributions for the observed variable.

Jochen, Identical conditions refer just to the setup of the experiment and not to the physical condition of the trial. Particularizing to your sampling example, this would be to pick the sample according to the same criteria, but different samples each time. As for references, I would point to:

http://www.britannica.com/EBchecked/topic/477530/probability-theory/32776/An-alternative-interpretation-of-probability (popular presentation)

http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter8.pdf (from central limit theorem to Bernoulli;s)

http://www.mhhe.com/engcs/electrical/papoulis/graphics/ppt/lect2a.ppt

http://www.springerreference.com/docs/html/chapterdbid/60792.html

http://statweb.stanford.edu/~cgates/PERSI/Courses/Phil166-266/IV-Frequency2013.pdf (bernoulli's theorem in epistemological view).