Suppose that an urn contains r red balls and w white balls. If n balls are removed from the urn without noticing their colors, then the probability, p, of drawing a red ball from the remaining balls in the urn is the same as the probability of drawing a red ball before removing the n balls; i.e., p = r/(r+w). How can we generalize this or relate it (for example) to information theory?
If you want to generalize your observation you first need to formulate your observation in a more mathematical way. For instance one may say that the probability of red is independent of the number of balls removed, or one may say that the probability of red equals a mixtures of conditional probabilities.
From my perspective, information theory does not (and cannot) enter the problem until you formulate a decision you are attempting to adjudicate. One such decision (but certainly not the only one) is to decide if of the next ball drawn will be red or white. For that question, the prior information is Lp=ln(r/w), which remains constant for successive draws. Based on you observations of intermediate draws, you can formulate the information contained by that data, i.e. Ld (here, since you chose to throw that data away, Ld=0). The total instantaneous information bearing on this decision for any draw is Ltot=Lp+Ld. The optimally informed decision is to choose red as the answer if Ltot>0, white if Ltot
Dear John and Peter,
Thanks so much for your answers.
I am, in fact, asking about any mathematical formulation for my observation that may be help in developing a mathematical model. For example, in terms of information, removing the balls from the urn without observing their colors has zero information. Then, my question is: how we can measure the information contained in the given event when we compute the conditional probability?
Have a nice day.
Mohamed
Mohamed,
1. Just to be precise, the approach I passed along utilizes the discriminating information measurement scale, not the more traditional Shannon (or entropic) scale; I find the former (with its clearly defined instantaneous information measures) to be more readily adaptable to generalized problems.
2. In general, to compute the information content of data for a binary question H1/H0 (such as will the next ball be red (H1) or white (H0)), one formulates the probabilities p(d|H1) and p(d|H0) for the data (d) to be exploited and identifies the LLR=ln(p(d|H1)/p(d|H0)); then Ld=LLR evaluated for the actual observations.
3. For the particular problem you have posed, I strongly suspect (but have not worked it out in detail) that the information provided by the data will adjust the total information to provide precisely the same value as if you reevaluated the prior information for the subsequent draw; that is, on the second draw, I would expect Ltot=ln((r-1)/w) if red was drawn the first time and Ltot=ln((r/(w-1)) if white were drawn the first time. Note that if the total number of balls in the urn to start with is unknown, the reevaluated prior information from draw to draw remains unchanged, so that the information provided by observing the previous draws is zero, even if you choose to record the results of those draws [this makes sense since the draws are statistically independent events - but it is also possible to formulate problems where the draws are not statistically independent]. It is only by knowing the value of your n [either as a single value or as some distribution of possible values] that the events become statistically relatable.
John
John,
Thank you so much for your good setting and explanation of the problem. I really appreciate it.
Mohamed
- Badges
- Science topic