I have data having over 20000 rows.
1) I am going to use a tool to test how many error rates this tool will calculate in this data.
2) And then, I will randomly choose some data and check how many error this data have. (data will be checked one by one not by the tool in 1))
3) And I want to compare the error rates of 1) and 2) to validate the tool I used in 1) to check if that tool correctly calculated or not.
But, I don't know how much sample size is proper to validate. The data is pretty big and take too much time to check one by one in 2) . Is there any methodology to decide the sample size in this case?
Hello Jiwon,
The glib answer is, the more (cases used), the better!
Your question has to do with the precision of any estimates of the rate of false positive and false negative results from your tool. To estimate a requisite sample size for this scenario, you'd need to: (a) specify the expected rate of each type of error; (b) specify a target precision or tolerance for the estimated value and target confidence level; (c) use a simple binomial model for estimation.
Example: suppose you thought that false positive rate (for your tool) would be as high as 20% (p = .20), and false negative rate as high as 15% (p = .15). Suppose you wanted your estimate to be accurate within +/- 1% (.01) with 95% confidence.
The web site (http://epitools.ausvet.com.au/content.php?page=1Proportion) computes the recommended sample size to achieve this precision as 5964 cases (for the false positive estimate) and 4781 for the false negative estimate. Of course, if you posit different rates or desire a different precision, your sample sizes will change.
Good luck with your work!
Hello Jiwon,
The glib answer is, the more (cases used), the better!
Your question has to do with the precision of any estimates of the rate of false positive and false negative results from your tool. To estimate a requisite sample size for this scenario, you'd need to: (a) specify the expected rate of each type of error; (b) specify a target precision or tolerance for the estimated value and target confidence level; (c) use a simple binomial model for estimation.
Example: suppose you thought that false positive rate (for your tool) would be as high as 20% (p = .20), and false negative rate as high as 15% (p = .15). Suppose you wanted your estimate to be accurate within +/- 1% (.01) with 95% confidence.
The web site (http://epitools.ausvet.com.au/content.php?page=1Proportion) computes the recommended sample size to achieve this precision as 5964 cases (for the false positive estimate) and 4781 for the false negative estimate. Of course, if you posit different rates or desire a different precision, your sample sizes will change.
Good luck with your work!
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size