In the attached paper page 206, the author gave an example when the prior and posterior probabilities can provide invalid probabilities.
The example is as follows
Let us suppose that X1 corresponds to a rare but important target event like, e.g., some medical pathology. We are going to collect the cases supporting this event very accurately. However, we are not going to collect information about all the “healthy” cases as X0. In the medical data sets we can rather expect the 50:50 proportion between positive and negative examples. It does not mean, however, that Pr(X1) should be estimated as 1/2. It is questionable whether the posterior probabilities Pr(X1|E) should be derived from such data using estimation with |E| in denominator – it is simply difficult to accept that |E| is calculated as the non-weighted sum of |E ∩ X0| and |E ∩ X1|.
But really I'm unable to get the idea so can anyone elaborate on this example.
Hi Ahmed,
Page 205, last 2 lines are important if you want to understand what is said in the Example 3 p. 206.
The posterior probability P(X1|E) depends on E. Now, the way I understand the example is this one. Suppose you have an unknown probability of having an illness X1, that is, the prior of this event is P(X1). Now, to make things more concrete, suppose P(X1)= 0.01.
Doctors do not see all people, because many people are healthy and do not need to see a doctor. The event "been seen by a doctor" is E. Among those seen by a doctor, about 50% are healthy, so P(X0|E) = 0.5, and the other 50% have the target illness, P(X1|E)= 0.5
You see that if you were to compute P(X1) from equation (4) and based on the data the doctors have (i.e., P(X0|E) = 0.5 and P(X1|E) = 0.5), the result you would get for P(X1) would be wrong.
To get the right result you must also know what is the proportion of the people who go see a doctor with respect to the people as a whole, i.e., to know what is the proportion between E and non-E (non-E being the proportion of people who do not go to see a doctor, so that E + non-E = 1).
Let me know if my explanation helped.
Best,
SCM
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Probability
- Probability Theory