What is the relationship between the two? As I understand, 95 confidence interval means the chance that people can get an independent result (e.g., mean or correlation) falling in the CI is 95%.
But the problem is that this 95% CI may include values of r that is not very significant. For example, in one study, you get a r of 0.12 with nearly 600 sample size. The r is significant and the CI might be [0.04, 0.20]. If you rerun the study and find a correlation of 0.045 (which is clearly not significant) with independent samples. So do you think you have replicated your first study? What about another case when you get a correlation of 0.039?
On the other hand, can we reject a published result if a replication study finds a result outside the 95% CI(we get the 95%CI from the published result)?
I want to add one point:
The fact, that in the long run about 95% of such intervals will barcket the population value ("TRUE" value), does explicitely not mean that that there is a 95% chance that this one particular interval of this particular study/experiment does bracket the TRUE value. The frequentists interpretation (!) is a long-run interpretation and does not make any statements about single cases. Thus, having a "significant result" (at 95% confidence / alpha=5%) still does not at all tell you something about the TRUTH based on your particular result. It is just that it gives you a rule to make a binary deciscion (e.g. accept or reject H0). Nothing said or implied about the particular result, and nothing said about the chance of making a correct or wrong decision in this particular case. Just if you always base such decisions on this same strategy, not more that 5% will be based on a wrong rejection of H0. Helpful in a long-term error control, helpful in screenings, but not helpful in judging an individual, particular result.
Jan, correct me I am wrong here.
Note: Frequentists see the limiting frequency behaviour as a definition of probability. To *my* opinion, this is not a definition, at most it is a resonable demand or requirement of "good choices for probability values". We assign probabilities to events, based on our knowledge. Reasonable assignments fit to long-run observations of the corresponding relative frequencies (Dutch-book argument). However, assignments are possible also where no long-run frequencies can be determined (practically or as a matter of principle). Hence, "probability" is a measure of knowledge (or ignorance, "lack of knowledge") - it is not a measure of something physically existing. The match of probability values and corresponding long-run frequencies just expresses the fact that we reached a minimum possible amount of ignorance (depending on the state of measurement/observation; using additional observations, e.g. from covariables and/or using better instruments may reveal that we can further reduce our ignorance/improve our knowledge, what will be represented in accordingly modified probability distributions).
I think that the (unfortunately very popular) frequentistic interpretation causes many such misconceptions, problems, irritations, confusions as for instance is seen in these question posted here (and in many others as well!).
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics