Dear Andy,

Would you kindly provide a bit more detail. Assume that we are scientists, but we can be from any field. Or perhaps the only person who has the skills to answer the question is an expert in Nordtest? I also found it a bit odd that the question uses proficiency testing data (from school children?) and then goes on to quantitative testing in molecular biology.

Nordtest.info is a website that could be relevant, but there is also something like this: http://prev.pt-conf.org/proceeding/05_Magnusson_UM_O_P0109.pdf

Maybe look at it this way:

1) I have an underlying distribution. It may be Normal, Uniform, or some weird multimodal curve.

2) I have a population that is a sample from the underlying distribution.

3) I have a sample that is some value less than the total population. Frequently considerably less than the total population.

This is where most people stop. They get a mean and standard deviation from the data and run statistics. However, there is uncertainty about both the mean and standard deviation calculated from these data.

Your question goes to the next level.

4) Here is where you are testing the experimental design, or lab. I have replicated #3. I now have a mean and standard deviation for the mean in #3. I also have a mean and standard deviation of the standard deviation in #3. I can do statistical analysis of these factors.

Of course there is error in these estimates too. I could replicate #4. .... and so forth for as many levels as I want. I haven't figured out why I would want to go past #4, or what the results would really mean.

The problem is that there are usually insufficient resources to address #4 in an experimental context. I use simulations. So I take a sample and calculate a mean and standard deviation. Based on those values and the expected distribution I can use a random number generator to create a population and another random number generator to sample that population. Of course the results are only as good as my estimate of the underlying distribution. Errors generated as a result of using three replicates to estimate the underlying distribution will be magnified many times with this approach.

If you ask about acceptance criteria for PT results: In Bioanalysis of Dioxins and PCBs in Feed and Food, we are using 20% for target standard deviation and calculate bioassay-scores in the same way as calculate z-scores for GC/MS-results (where 10% is used as target standard deviation). In other words, we allow more tolerance for the bioanalysis.

If you ask about acceptance criteria when estimating ubias using results from PT studies: When used for MU assessment, the assigned value together with its uncertainty (which may be relatively high due to different methods applied by the various participating labs) may sometimes lead to an unduly high contribution to the laboratory's MU, making it desirable to define a maximum acceptable value for uCref. We require that uCref shall not exceed 30% of RMSbias of the value reported by the participant:

uCref / RMSbias ≤ 0.3

Hope this helps, Johannes