Folks,
I have 15,000 address strings which I have written a piece of code which assigns them to broad geographic region e.g. American address assigned to North America. I would like to check the accuracy of my code however to do this I must manually check the strings. To do this I've taken a sample (SRS). I set a margin of error of 5%, a CI of 95% and a response distribution of 50%. Therefore, I assumed that I would assign the correct region half the time.
The issue here is that I am much more accurate then I anticipated. Of my sample of 375 372 (99%) where assigned to the correct location. This means I am unable to calculate lower and upper confidence intervals.
Is there any other types of formula I could use when your response distribution is skewed to a highly positive outcome? Or is the only remedy to simply increase my sample size?
The only way I can verify if something is correct is by manually checking therefore the need to draw.
Hello Sean,
If I understand your query correctly, you planned your sample presuming the largest possible variation with respect to classification success (correct classification 50% of the time), but your overall classification success was in fact much higher (99%) for your sample.
First, congratulations on such a high success rate!
With respect to the sampling, however, the result simply means that you most likely over-sampled, taking far more cases than absolutely necessary to estimate the population value with your target precision (+/- 5%, with 95% confidence). Your obtained precision will simply be higher, as a result, if the true (population) success rate is closer to 99% than to 50%.
This in no way precludes you from generating a CI for your parameter estimate (which, I presume, is the population rate of correct classification). It just means that the upper limit can't be beyond 1.0, and your obtained CI will have a much smaller width than +/- 5.
Good luck with your work!
There are different methods to calculate the confidence interval of a proportion. Which method are you using and why do think it fails in this case?
I think the Clopper-Pearson method would be okay to use in this case. One simple way to use this method is the binom.test function in R. The following code could be run at https://rdrr.io/snippets/ if you don't have R installed. In the output you can ignore the p-value.
binom.test(372, 375,
0.5,
conf.level=0.95)
### 95 percent confidence interval: ### 0.9767998 0.9983472 ### sample estimates: ### probability of success 0.992
Skewed data are the norm rather than the exception in many areas of the geological sciences. I typically default to nonparametric methods to estimate measurements of central tendency and dispersion, but in many cases, it is necessary to use other methods, especially if confidence intervals are of concern. I assume that your skewed data either are or approach a log-normal distribution. You might consider the following:
http://jse.amstat.org/v13n1/olsson.html
- Badges
- Science topic
- Similar topics
- Psychology