Actually, neither of those definitions is correct. The correct interpretation of a 95% confidence interval is: if you repeat the experiment infinite times (i.e., if you take infinity random samples from the population), 95% of those samples' CIs will contain the population mean. There is some R code at the end of this post which you can run to view simulations showing that this definition is accurate, whereas the definition you mentioned (that 95% of the sample means fall within the interval) is not.

Note that in many cases it is safe to use the real definition of a CI to infer the "classical" interpretation you mentioned. That is to say: if, out of inifinity random samples of the population, 95% of those sample CIs contain the population mean and 5% of the samples don't, then it is usually reasonable to assume that the sample you got in your experiment is more likely to be from the 95% of sample CIs that do contain the population mean rather than from the 5% that don't. But it's definitely not the same as saying that 95% of replication means will also fall within the CI (if your experiment sample happened to be un-representative, e.g. drawn mostly from one tail of the distribution, then actually very few replication means will fall within your observed CI, as you can observe from the simulations below).

Also note that the purpose of a CI, at least in applications I am familiar with, is not "to determine if the sample is representative of the population"; rather, it is to form a reasonable estimate of where the population mean might fall, given an observed sample. This could, in turn, be used to see if the sample is unlikely to have come from a certain population (e.g., performing a hypothesis test by checking whether the CI doesn't include 0 is the same thing as asking whether the sample is unlikely to have come from the null distribution [which is a population with mean 0]) but there's nothing about the CI itself that inherently tells you how representative of the population a certain sample is.

I highly recommend the Hoestra et al. and Belia et al. papers (which I mentioned and linked in my above posts), they have good explanations of the interpretation of CIs.

####################

### CI SIMULATIONS ###

####################

# Create a normally distributed population

population

I don't have a good answer (I haven't tried teaching this in a formal setting yet), but the following papers might be useful:

http://link.springer.com/article/10.3758%2Fs13423-013-0572-3

http://www.apastyle.org/manual/related/cumming-and-finch.pdf

I also made this tutorial, which might come in handy (although it might be more advanced than what you need for your students, or might not be relevant to the kinds of research designs you are discussing):

http://www.mypolyuweb.hk/~sjpolit/confidenceintervals.html

Explain to them that the question for the confidence interval is to determine if the sample is representative of the population. If we reject the null, we then say, with confidence (e.g., 95%), that the sample is NOT representative of the population. Now that there is statistical difference, we need to determine if there is a practical significance, and that answer requires discussion with stakeholders and colleagues.

In the classical interpretation, we say that there is a 95% chance that the population mean falls within the interval (i.e., fail to reject)

In the contemporary interpretation we would say that if we took 100 random samples from the same population that 95 of those sample means would fall within the interval (again with a fail to reject (aka retain the null)

Actually, neither of those definitions is correct. The correct interpretation of a 95% confidence interval is: if you repeat the experiment infinite times (i.e., if you take infinity random samples from the population), 95% of those samples' CIs will contain the population mean. There is some R code at the end of this post which you can run to view simulations showing that this definition is accurate, whereas the definition you mentioned (that 95% of the sample means fall within the interval) is not.

Note that in many cases it is safe to use the real definition of a CI to infer the "classical" interpretation you mentioned. That is to say: if, out of inifinity random samples of the population, 95% of those sample CIs contain the population mean and 5% of the samples don't, then it is usually reasonable to assume that the sample you got in your experiment is more likely to be from the 95% of sample CIs that do contain the population mean rather than from the 5% that don't. But it's definitely not the same as saying that 95% of replication means will also fall within the CI (if your experiment sample happened to be un-representative, e.g. drawn mostly from one tail of the distribution, then actually very few replication means will fall within your observed CI, as you can observe from the simulations below).

Also note that the purpose of a CI, at least in applications I am familiar with, is not "to determine if the sample is representative of the population"; rather, it is to form a reasonable estimate of where the population mean might fall, given an observed sample. This could, in turn, be used to see if the sample is unlikely to have come from a certain population (e.g., performing a hypothesis test by checking whether the CI doesn't include 0 is the same thing as asking whether the sample is unlikely to have come from the null distribution [which is a population with mean 0]) but there's nothing about the CI itself that inherently tells you how representative of the population a certain sample is.

I highly recommend the Hoestra et al. and Belia et al. papers (which I mentioned and linked in my above posts), they have good explanations of the interpretation of CIs.

####################

### CI SIMULATIONS ###

####################

# Create a normally distributed population

population

Stephen Politzer-Ahles has it right.

There is a subtle difference between a confidence interval and its associated random interval. For example, a 95% confidence interval for the mean is one instance of an associated random interval.  The random interval has a 95% of containing the mean, not the confidence interval.

Both the mean and a confidence interval are fixed.  They do not vary. Either the confidence interval contains the mean or it does not.  There is no probability that can be logically assigned to it.  On the other hand, there can be a probability assigned to a random interval containing the mean.

As explained below, I believe that the issue comes down to understanding the difference between the "mean" \mu and the "sample mean" \bar{X}.  The sample mean is a random variable, while the mean is a fixed number.

For a known variance \sigma^2, and a normally distributed random variable X, the random interval associated with a confidence interval for the mean is [\bar{X} - z\sigma/sqrt{n},\bar{X} + z\sigma/sqrt{n}], where z is a fixed number that depends on the confidence level (e.g. 95%) and n is the sample size. This interval is random because it contains the random variable \bar{X}.  Once you assign an instance of \bar{X}, the interval is no longer random.  It either contains \mu or it does not.

For teaching confidence intervals in an introductory statistics class, one might consider the following outline.

(1) Explain what a random variable is, (2) Explain what a random sample is, including what IID means, (3) Define \bar{X} in terms of a random sample (4) Explain the difference between \mu and \bar{X} (\bar{X} estimates \mu, \bar{X} is random, \mu is not). (5) Explain what a confidence interval is (for the mean).

There are of course several steps in between to explain, but these are essential from my perspective.  It may be that doing something less will lead to the sorts of confusion already mentioned.  On the other hand, it may be too much for an introductory class, depending on the ability of the students involved.  Ultimately, it's a judgement call.  

By something less, I mean the approach taken in the following reference:

Orkin, M. and Drogin R., "Vital Statistics", McGraw-Hill, 1975.

The approach I've suggested above is more in line with the following:

Meyer, Paul L., "Introductory Probability and Statistical Applications", Addison-Wesley, 1970.

Both are introductory, but assume different levels of mathematical experience and/or ability; the latter being at a higher level.

Thank you all for your valuable contributions.  More thanks to Stephen Politzer-Ahles for  unpacking the intricacies inherent in the meaning of the CI and for the CI simulations; and to Gregory L. Wilson for the  5-point advice.

I have been in charge of an introductory course of statistics for many years and your point (about CI) is one of the trickiest I encountered.

After many attempts, I ended to present not just the CI but rather three intervals : the "betting interval" (crude translation from the French : intervalle de pari), the "confidence interval" and the "bayesian interval" (without trying to compute them all explicitly, but the two first may be).

To make myself clear : suppose a normal setting. The betting interval is the interval you may compute to contain an observed mean with a given probability, if you know the theoretical values of mean and variance. The confidence interval is the interval which covers the true unknown value with a given probability, may be deduced from the former by elementary calculus around inequalities. The bayesian interval is the interval  actually *containing* the true unknown value with a given probability.

Doing so the students may figure out that they are in fact different probabilities involved (or rather probabilities of different events). That may help them.

Two concluding remarks : 1 - the students may then understand how and why the confidence interval is suited for testing, 2 - this leaves open the question of the fiducial interval/probability which is a point I am still striving to understand.

May be you mush use the grafk probability function illustration. CI depend on Ha or H1.

My colleagues explained above the exact  meaning of the CI, and  no need to repeat it.

I found from my teaching experience that CI is a very difficult topic for the students.

I would suggest first of all to teach more about point estimates, which is also

not very easy for the students (but possibly more important). Ilya Gertsbakh

Thanks Ilya,

Students, I believe, would have first covered frequency distributions, measures of central tendency and measures of variation of univariate data.  Point estimates was not in this course covered in detail, but  I  required students to generate different shapes of distributions and decide which point estimates (e.g. mean, median) would be  appropriate for such a distribution. (The second course in  my Department treats estimators in more detail).

The CI gets more challenging at the intro stage I think   because it is grounded on the theory of probability, which theory students are still developing.

I think that my words would be not have a great value after all Professors said. But I want to ask if the concept of sampling distribution of the mean could help to explain the confidence interval concept? 

I saw a simple numerical example of a give population of numbers {1, 2, 3} and we have to get all samples of size 2 (with replacement). It shows that central limit theorem is applicable.

We could then ask students, what is the probability to get a sample of 2 from this population with a mean equal to the population mean? The probability is 50 %. And using that sampling distribution is normal, what is the probability that another sample mean value could be resulted? An example could be found in this link: http://onlinestatbook.com/2/sampling_distributions/intro_samp_dist.html

Is this would help to explain the confidence interval after that introduction??

I want also to refer to a book titled "Even You Can Learn Statistics". I hope it could help. https://books.google.com.eg/books?id=yYfMVq2FK5QC&redir_esc=y