Interesting question, but I presume you are actually asking how many bootstraps you NEED to perform (since you can do as many as you like). It is important to realize that whatever the number of bootstraps made, the final precision is still completely determined by your initial sample size. Bootstrapping is one method to assess a statistic computed from a sample. Very often we are interested in the accuracy of that statistic when used as a point estimate of a population parameter, Bootstrapping means we temporarily substitute the empirical probability distribution induced by the sample for the probability distribution defined by the population. The repetition of taking a bootstrap sample just replaces the otherwise very teady calculations with that empirical distribution that would be required to assess your initial statistic. Bootstrapping is not per definition taking a large number of bootstrap samples, that is just the practical way it is performed (i.e. by Monte Carlo-simulation from the empirical distribution). To judge upon how many bootstrap samples you should take requires (as other contributors here also have said) consideration of for instance how large the variance of your statistic (assuming the empirical distribution is temporarily the true one) is, how skew its sampling distribution is etc. etc.

It can be variable depending upon many factors. However, in practice, 1000 iterations can be performed. For statistics students, when NPC, SE etc. are explained, 1000 iterations usually give an account of how, given every other condition constant, it can be interpreted in reference to the assumption of normality.

There are some other specialized tools that have used different iteration numbers. For example, Standardized Low Resolution Tomography Analysis (sLORETA) recommends using 5000 iterations because they don't use assumption of normality for their analysis of Statistical Nonparametric Mapping (SnPM). However, to analyse the difference of R between two surface Laplace maps (difference between map1 vs. map2 and pool1 vs. pool2), just 200 iterations have also been used as found in the literature.

So, as I said, it is variable and depend upon many factors at hand.

Hope this helps...

Ashwini is right. 1000 replication will be sufficient.

You mean, 1000 replication whatever the number of elements of my data?

I know b =(1:1000))

I am not sure I understand what you want to do. Monte Carlo will already sample your parameter space sufficiently, so I see no need for bootstrapping the "one" dataset you got. Monte Carlo processes could easily give you 1000s to millions of datasets.

I think more 1000s that good

The rule of thumb is 1000. However, for a sample size N there are only (2^N - 2) possible sample combinations (assuming you ignore the original sample and the null sample), after which your new replications are simply repetitions and probably shouldn't be included. Hope that helps.

Maximum of 1000 replicates is more than sufficient.

Cheers.

I think it depends on the complexity of the "statistics" you want to calculate from the samples. But 1000 is good enough.

JL Foulley, University of Montpellier II, France

The total number of bootstrap samples of size n out of a n data sample is simply the number of n combinations out of 2n-1 items . This is a particular case of the so called Bose Einstein statistics giving the the number of ways of distributing n particles among the g sublevels with here g=n.

So, I recommend to people that thinks that 1000 repetitions is enough, to read

Pattengale, N., Alipour, M., Bininda-Emonds, O., Moret, B. and Stamatakis, A. (2009) How many bootstrap replicates are necessary? Research in Computational Molecular Biology. Lecture Notes in Computer Science, Springer Berlin, Heidelberg, pp. 184-200.

With 1000 repetitions many times you do not get the proper answer if you have a large enough number of variables.

Interesting question, but I presume you are actually asking how many bootstraps you NEED to perform (since you can do as many as you like). It is important to realize that whatever the number of bootstraps made, the final precision is still completely determined by your initial sample size. Bootstrapping is one method to assess a statistic computed from a sample. Very often we are interested in the accuracy of that statistic when used as a point estimate of a population parameter, Bootstrapping means we temporarily substitute the empirical probability distribution induced by the sample for the probability distribution defined by the population. The repetition of taking a bootstrap sample just replaces the otherwise very teady calculations with that empirical distribution that would be required to assess your initial statistic. Bootstrapping is not per definition taking a large number of bootstrap samples, that is just the practical way it is performed (i.e. by Monte Carlo-simulation from the empirical distribution). To judge upon how many bootstrap samples you should take requires (as other contributors here also have said) consideration of for instance how large the variance of your statistic (assuming the empirical distribution is temporarily the true one) is, how skew its sampling distribution is etc. etc.

Have a look at this paper:

Davidson, R & MacKinnon, J G (2000) Bootstrap tests: How many bootstraps? Econometric Reviews 19(1): 55-68.

I agree with Hugo Volkeart above, that your problem is not too clearly defined.

The minimal number of bootstrap samples you should take is dependent on the statistic you want to bootstrap and what aspect of the statistic you need to analyse and interpret.

For instance, to get confidence intervals for the mean, 100 bootstrap samples would be sufficen, for a confidence interval of the standard deviation you need more, like 500. But for a statistic like the cost-effectiveness ratio I use 50.000 since the underlying distribution of the statistic is Cauchy (because the denominator can be zero, the function contains a discontinuity).

If one is interested in the complete distribution of the statistic, not in its confidence intervals only, a large number of samples is needed to get a proper and precise impression (This occurs if one wants to determine outlyers).

But so what? To calculate the statistic on a large number of bootstrap samples usually takes only a few seconds in R.

(note that Sean Cloustons calculation of the different number of bootstrap samples (see above) is incorrect, because it does not take into account (a) that the number of elements of the boottrap sample is always N; (b) since you sample with replacement, the same element of the original sample can occur several times in the bootstrap sample.

For more on the subject see:

Advising on research methods: A consultants companion

by Adèr, Mellenbergh and Hand (2008).

(website: www.jvank.nl/ARMHome)

The book has been indexed for Google books

(there is a link to Google books on the website, type BOOTSTRAP in this case as a search term).

Herman Adèr

Its not clear what is the Output of your MonteCarlo Simulation is. Apparently you are producing a series of data from that process, that you call a Sample. (Eventually one data per each Trajectory of the Simulation) If you could broaden your explanation of your Process, perhaps you could get a more Useful Suggestion.

Treatment of Data Series as for Input or Output to/from a MonteCarlo Process is Dellicate.

Simple random sampling by purposive?

Try 1000; 10000; 100000 and observe the results plotted. This helps you decide

In R is very fast

what is ln R?

R is a freely downloadable, high quality statistical package, derived from the Splus package.

Look at: http://www.r-project.org/ to get hold of it. It has a wide range of additional packages, not all statistical,

including one for bootstrapping (boots)

Hi,

I usually do 10,000 replicates but actually you could do many more depending on how long it takes on your computer.

But I also have a bias estimator that shows me how far off (my simulated mean) is from say the mean given by your original sample. I usually use Python to do the simulations or SAS.

ok bye for now.

we can perform infinite time bootstraps on a sample of N elements.

Could anyone give an advice for bootstrap analyses using R to phylogeny?

for genetic phylogenies, I would recommend you RAxML; you can perform many bootstraps in a very fast way (faster than using R)

Thank you, Leocadio!

All the best, Vitor

In general, I believe you can do as many resamplings as you like from a dataset without concern about exceeding N. That said, applying bootstrapping to Monte Carlo derived dataset surprises me and makes me think you may want to reconsider. Typically Monte Carlo analyses themselves involve repeated samples from random number generators and generate an often very large number of output observations, so it stikes me as odd to take some or all of these and then build up a bunch more by resampling. You might rather want to just up the number of observations created by Monte Carlo process you are using.

The number of bootstrapped samples that you can generate from a sample of N elements is more a question of computing power rather than a statistical question. If you are thinking about this as a statistical question, then perhaps your concern is that you don't create the same bootstrapped sample twice. There is no way you could guarantee that you didn’t create the same bootstrapped sample twice – that could occur by chance. However, the number of possible unique bootstrapped samples from a sample of size N is equal to N! (N factorial or N * (N-1) * (N-2) * … * 2 * 1). This number gets large very quickly. For example, 12! is equal to 479,001,600 – or nearly half a trillion possible combinations, but 6! is only equal to 720, while 7! is equal to 5,040, and 8! is equal to 40,320. So if you have a really tiny sample, such as N=6, you might want to limit the number of bootstrapped samples to, in that case, 720 so that you could say the expected number of times that each possible bootstrapped sample would occur would equal 1. Of course, there would undoubtedly be some duplicate samples and some that did not occur, but with any number of samples over 720, you would be guaranteed to be generating duplicate samples, which would not contribute any new information to the analysis. However, with a sample size of 10 or more (over 3.5 million possible samples), there would be a relatively low probability of duplicate samples, even if you created 100,000 bootstrapped samples.

Dr Vanner/I do not agree with you: the number of different possible boostrapped samples from a data set of size N is not N! but C(N,2N-1) number of combinations of N items out of 2N-1 which is smaller than N! a soon as N>5: see my comment 2 days ago.

I do not see any other post from you, but I defer to your expertise.

Here is a copy of it.

JL Foulley, University of Montpellier II, France

The total number of bootstrap samples of size n out of a n data sample is simply the number of n combinations out of 2n-1 items . This is a particular case of the so called Bose Einstein statistics giving the the number of ways of distributing n particles among the g sublevels with here g=n.

Leonidas Georgopoulos likes this · Edit · 2 days ago

I understand you want to resample by BS from a MC sample. I am assumming that the MC sample may come from a MC simiulations that are relatively expensive (due to the tyoe of process that you are simulating) so you expect that BS resampling will aid in getting CL's and perharps freqeuncy distributions of the statistic which interersts you, If this is the case, then the answer to your question about the number of BS samples depends upon the what assumptions or computation you are willing to make. If you intend to get the CL's of your statistic under the normality assumption, all you need is the standard deviation and the confidence level; if you need to get nonparametric estimates of the CL's of yur statitistic, you can do so based on the freqeuncy distribution (HISTOGRAM) based on the proper quantiles for teh conficence level you wish to prescribe.

Regarding the numbes of BS samples, one plausible to presicribe is by reliying on an adaptive convergence criteria, that is, get b samples, estaimte the CL's fo the statistic of interst, generate 2b samples, and recompute the CL's, the optimal number of BS samples is that for which the CL's estiamtes converge to an accepable precision. In the case of relying on non parametric estiamtes fo the CL's based on the histogram, what you coudl graphically see is that as BS increases, the histogram changes, but it tends to converge to a given histogram. The number of BS samples that it takes to get a stable histogram is the optimal; increasing BS will not change your estiamtes.

Onc concern you should have is that depending on the statitstic which distirbution you are aiming to characterize by BS resampling, is the expected bias of the estimate--I recome that you look at the papers below.

Best,

Efron, B., Tibshirani, R. (1986). “Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy.” Statistical Science 1(1), 54-77.

Efron, B. (1987). “Better Bootstrap Confidence Intervals.” J. Amer. Statist. Assoc. 82 171-185.

DiCiccio, T., Efron, B. (1996). “Bootstrap Confidence Intervals.” Statistical Science 11(3), 189-228.

From Jean-louis Foulley:

The total number of bootstrap samples of size n out of a n data sample is simply the number of n combinations out of 2n-1 items . This is a particular case of the so called Bose Einstein statistics giving the the number of ways of distributing n particles among the g sublevels with here g=n.

So with 4 samples, the number of DIFFERENT bootstrap samples should be following Bose-Einstein formula: (n+g-1)!/n!(g-1)!

if g=n:

(2n-1)!/n!*(n-1)!

or for n = 4

=7! / 4!*3! = 7*6*5*4*3*2*1 / (4*3*2*1) * (3*2*1)

= 5040 / 24 *6 = 35

or should it be 4! = 4 * 3 * 2 * 1 = 24?

Just for a test, here are the results for n = 4:

AAAA

BBBB

CCCC

DDDD

AAAB

AAAC

AAAD

AABB

AACC

AADD

AABC

AABD

AACD

ABBB

ACCC

ADDD

ABBC

ABBD

ACCD

ABCC

ABDD

ACDD

ABCD

BBBC

BBBD

BBCC

BBDD

BBCD

BCCC

BDDD

BCCD

BCDD

CCCD

CCDD

CDDD

35 combinations. However, that does not mean that 35 bootsraps will give you these 35 different sets... Since bootstrapping is a random resampling procedure - WITH REPLACEMENT, starting each time again from the same dataset, one may have to perform many more than 35 resamplings in order to get one or more for each.

One does not need to get the different possible bootstrap samples... one needs to recreate a possible data space DISTRIBUTION.

How many samples do you need to have a representative distribution? In my experience, for phylogenetic analysis based on DNA or protein sequence data up to several hundred polymorphic sites, 500 bootstrap resamplings are sufficient. If you do more resampling, all that may change are the numbers after the decimal value...

Since there is no agreement on what a reliable bootstrap value is (50% cut off?, 75% cut-off?, 90%, or 95% cut-off) the possible small changes in bootstrap values for larger bootstrap datasets is a waste of time. If a critical grouping of taxa is very close to the cut-off value you have decided upon (just a little bit more, say 76% if you decided that 75% is your cut-off), it may be worthwhile to do a second bootstrap analysis (even with same number of bootstraps) to confirm. As each resampling is independent and unique, there is no guarantee you will see the same results and if the evolutionary signal is not consistent, and thus not reliable, you may obtain different bootstrap values.

It is for sure 35 not 16. In fact, this number M of samples increases very rapidly with the size n of the data set.

For instance for n=17 , M=1 166 803 110 so that the probability of generating by MonteCarlo the same boostrap sample becomes very small.

Sorry

It is for sure 35 not 24.

Sufficient Bootstapping by Singh and Sedory (2011) will be sufficient to answer all the questions you people have. See here:

http://www.sciencedirect.com/science/article/pii/S0167947310003981

Your first wrong question will also be answered correctly! Because your question was not wrong, it was correct!

You people can also enjoy reading my new thinking: "Saddlestrapping"

http://www.lana.lt/journal/34/Singh.pdf

I've done some bootstraps using my own programmes in Python and SAS to handle them and can say you can do as many as you want. Generally, though 2,000 is a good starting point, then go to 10,000 but there is no reason you couldn't do 100,000 if your computer can handle it. Get a feel for the numbers, and compare it to straightforward statistics of the mean, sd etc. The difference between bootstrap and standard stats will be your 'bias'.

You are right, but from a sample of n=5, we can have at most 5^5 = 3125 bootstraps or saddlestraps. We can not go as many as we like being more than 3125.

True Sarjinder, 5^5 is the # of different combinations but we are not limited by this number. We can do as many boots as we want. What we want to find is the sample statistic e.g. mean and std deviation say. Because n=5 is quite small a sample size the Central Limit Theorem does not apply here, so we bootstrap instead say 10,000 and work out the booted mean and sd (this would probably pass any peer review in journals so you would get published).

If I accept your answer, then we are not using with replacement sampling. The way you are saying I will like to select samples using without replacement sampling. These 5^5=3125 are not combinations. These are total numbers samples by the Fundamental Law of Counting! If you exceed samples more than 3125, there will be nothing new in information in any sample. It was just an example with small sample size. Total number of boots should not exceed n^n, whatever sample size could be. If you take boots more than n^n, there is no new information in your boots. Why should I do it?

ok, thanks Sarjinder. I'll read your paper.

@Sarjinder I think I see where you are coming from and agree that the total number of distinct bootstrap resamples is n^n. However, I don't think I agree that the bootstraps > n^n fail to provide any information. It is entirely possible that I sample n^n times and end up with floor(1.1*n^(n-1)) replicates of a given observation on my first pass and then do it again and get floor(0.9*n^(n-1)) replicates of the same observation on my second pass. For small n, only after taking many iterations of n^n samples will I see that the average number of instances of a given observation stay close to the expected n^(n-1) when averaged across all iterations.

Think this way: what is the expected value of a statistic over all possible bootstraps? For SRSWR sampling, the sample mean is unbiased under the assumption total number of samples are N^n, assuming N as the population size and n as the sample size. For SRSWR sampling, it is possible that sample size n could be equal to the population size N. That is the reason we use SRSWR sampling for bootstrap. If we use without replacement sampling, then only one sample of size n from a population of size lowercase n is possible. Because there is only one combination.

@Dudley: Good, at least I found one reader of my papers!

@Dudley! I have couple of e-mails like this:

Subject: About your paper" Sufficient bootstrapping"

Dear Dr. Singh,

We are currently working on your paper “sufficient bootstrapping”. The paper is very interesting. We already adapted the method with jackknife after bootstrap method to detect influential observations.

You becarefull your sample bias can monitor more before make new

@David! Yesterday I had no time. Let us take n= 20 (reasonable sample size). Assume 500 boots are enough. Total number of possible distinct boots are = 20^20 = 1.04858E+26. Now two questions:

( a ) What is the probability of including a repeated boot in the 500 boots?

( b ) If you say boots could be infinite (no limit), what will be infinity divided by infinity?

Seems we are not communicating all that well. I was talking about frequency of a given observation in an n^n size set of resamples. You are talking about a "repeated boot", presumably a repeated set of n observations. Not the same thing. Unless we sat down across from each other I suspect it would take us a long time to come to a meeting of minds.

Nevertheless, now that I know the names some folks attach to two approaches of bootstrapping (as per Wikipedia) perhaps I can clear things up a little. I agree that if using the "exact" method then additional additional boots past n^n are superfluous. However, if using the Monte Carlo method then it is possible to extract a tiny bit more information by increasing the number of resamples above n^n since with this method it is highly likely that some observations occur more frequently in the grand sample than others.

Practically speaking, this question has no practical utility anyway since, for all practical purposes, when using the Monte Carlo method you are almost surely going to run out of computer time and patience before you come anywhere close to n^n for statistically interesting datasets.

Think this way: What is the inclusion probability of a particular unit to be included in a simple random and with replacement sample? To my knowledge, there are only three methods of counting: ( i ) Fundamental law of counting, ( ii ) Permutations and, ( iii ) Combinations!! If bootstrapping method has any fourth rule of counting, then I have no idea!!

I've actually seen people try to bootstrap with as few as 3 and 4 observations....and when testifying in court as an expert witness. So, although it should be a moot question it really isn't. So I'll qualify this as an academic discussion, but fun to think about. That said I disagree with both Sarjinder and David. Here is my argument.

I looked at the simple example of a sample of size n=3 with three unique values, say 7, 12 and 15, and the statistic of interest is the sample mean. Note it actually matters if the values are relatively prime or not, but lets work with these three that are relatively prime.

I argue that there are n^n=3^3=27 possible unique bootstrap samples (with replacement), and only 10 possible unique means-with unequal probability of occurrence. Iin this situation there is one way to draw three 7s so the exact percentile for a mean of 7 is 1/27 or 3.7%. This is the most extreme one tail probability that is estimable. The second smallest mean among the 10 possible values is 8.67 (mean of (7,7,12) , which can occur in just thee ways, so the cumulative percentile associated with this value is (3+1)/27=14.8%. So it is mathematically impossible to estimate either 5 or 10 percentage points as there are only two possible values which represent the 3.7% or 14.8% points.

So no matter how many thousands of bootstrap reps are drawn, the lower tail will be many many repeated 7s and 8.67s, so the naive percentile bootstrap estimate of the 5th percentile will actually be the 3.7th percentile and the 10th percentile estimate will actually be the 14.8th percentile.

These are large biases for tail probabilities. But then who would try to bootstrap 3 numbers? Maybe we do it by accident when we have small sample sizes with several repeated values.

The real moral of the story is 1) don't use the percentile bootstrap, and 2) get to know your data before you do anything.

One other comment...for highly skewed data, none of the bootstrap methods work well with less than about 30 samples, or perhaps even more. I know this was not the question, but probably more interesting to think about the minimum sample size necessary to insure that confidence limit estimates have the proper nominal coverage and are not too variable.

@John: Great! While dealing with any percentile (including proportion), you are very much right that sample size should be greater than 30. It is written in many textbooks. Sometime I want to see all possible boots with my eyes rather than a histogram, so small sample size helps to watch all with open eyes. Also helps to teach in a class. Take take n=3, put all possible 27 boots on the black board in front of students, make frequency distribution table, and show them histogram on the blackboard. Students enjoy it. Students can see it with eyes! Later you show them on a computer with a big data set. Then students can understand: What are you trying to tell them? No doubt it is more or less academic interest!

You can use bootstraps from true condition in your research on research metherdology

Interpreting the question as:

How many [unique] bootstrap samples can be formed from a sample of N elements [where each of the N elements is distinct]?

The answer is (2N - 1) choose (N).

(This is exercise 8.4 from Larry Wasserman's *All of Statistics*.)

You can convince yourself of this by thinking about how many ways there are to put N balls into N buckets, using the 'stars and bars' counting method:

http://en.wikipedia.org/wiki/Stars_and_bars_(combinatorics).

Then it is not with replacement sampling! It could be "stars and bars" sampling!

David,

This is correct, but to calculate the percentiles directly you also need to know how many ways each of the unique estimates can be obtained. That is why we have been talking about all N^N combinations of N (unique) values taken N at a time with replacement. For example in my N=3 example there are indeed 10 (2N-1) choose N distinct sample values, but they are not equally likely. In order to attain the percentiles you need to work with all 27 (non-unique) samples.

You can get extra information out of more than n^n bootstraps. Because I once worked many years ago (mid 80's) for Dept of Water Resources in Sydney, and when I was doing a course on predicting the capacity of dam to account for a long drought, we used to use bootstrap type simulations of river flows. What we wanted to know was how many consecutive days there would be no flows. It is akin to asking how many consecutive heads comes up on a coin. However, we then abandoned that method, and used instead the Markov chain, to get the long run steady state of the dam using a transition matrix. The moral of the story is that there are better methods than bootstrapping.

Could you please show us any extra information by taking a small example with n=3?

It will help us in Academics! I will love to teach "Dudley's" methodology in my class.

Can bootstrapping handle a Small data, Big data, Bigger data, or Biggest data?

If so, how? If not, why?

Can we differentiate between Small data, Big data, Bigger data, or Biggest data?

If so, how? If not, why?

Just to clarify:

There are [(2n-1) choose n] distinct bootstrap samples. That is with replacement, the elements in each sample are not unique (actually that is the definition of a bootstrap sample). But the order of the elements does not matter.

If you use n^n samples that would mean the order of the elements matter, which doesn't necessarily make sense!

For example, when n=3 there are 10 distinct bootstrap samples (not 27, see * below for explanation):

1 x1,x1,x1

2 x1,x1,x2

3 x1,x1,x3

4 x1,x2,x2

5 x1,x2,x3

6 x1,x3,x3

7 x2,x2,x2

8 x2,x2,x3

9 x2,x3,x3

10 x3,x3,x3

* A sample {x1,x1,x2} is the same as samples {x1,x2,x1} and {x2,x1,x1}. That is, these samples are not distinct.

See Efron, B & Tibshirani, RJ. 1993. An introduction to the bootstrap, page 49

See Sufficient bootstrapping by Singh and Sedory. Effron and Tibshirani missed a point that there is a theory of distinct units in with replacement sampling scheme. It was developed in 1950s, when bootstrap was developed in 1979s. The problem is that people do not look at literature and go with their own minds!

 Dr Singh you made the point very much clear. It is really a need for a student to see how the bootstrapping method is being run inside the machine. Thus, it is always better to sample from a small sample  such as n = 3.

Thank you for your clarifications. I think that answers the question.

It depends on the distribution of the underlying populatoni and the objective of the analysis, regression 95% confidence limit on the mean etc.

In simulation studies I've conducted investigating coverage of confidence intervals, I've found that for right skewed data with less than 30 or so samples the bootstrap methods tend to undercover the true mean...i.e. the bootstrap confidence interval captures the true population mean less than the nominal confidence level.  

Again, the actual performance is a function of the underlying distribtution.  Less-skewed distributions require fewer samples to develop reliable inference. I recommend conducting simulation studies with pilot data to plan for adequate sample size when bootstrap analysis is anticipated. 

But I don't think this was what the author was asking.

One such study:

Technical Report PERFORMANCE EVALUATION OF UCL ESTIMATION METHODS: WHEN DATA ...

1000 is good :)

The number should be "large" but less than n^n.

2^n - 2 is not generally the maximum number of possible mixes because (1) the sampling comes with replacement, and (2) the same set of observations in a different order is considered a unique subsample.

@ Lisa Kirkland

The estimator of mean will still be unbiased, unfortunately it will increase the variance of the sample means. Keeping 10 samples out of 27 possible samples would not reduce the variance of the sample mean. Only SRSWR with n^n will make unbiased and efficient estimator of mean. What will happen to CLT? Keeping less boots or increasing number of boots than n^n will certainly produce biased estimates. If boots are less than n^n, no doubt there will be bias and less efficiency. If boots are more than n^n, again there will be bias but more efficiency. The unbiasedness and efficiency will match with the CLT only if the number of boots are n^n and boots are taken using SRSWR. Yes, it is proven in 1950's that keeping distinct units in SRSWR ( or PPSWR sampling) leads to efficient results, that is what we named sufficient bootstrapping!

@John W. Kern : Just take simple example, and show all steps. If someone knew distribution, then nothing left to estimate! Generally we do in Mathematical Statistics, let f(x) be known. If it is know, then what is left? Seems very broad complement by me!!! Well, I will wait your reply!

Suppose I have sample size N =10, the number of bootstrap replicates is 1000......And i want to repeat this whole process a number of times (Call it Duplication/repetitions)......Is there any number for it...like 10,20 or 100?

@Eshan Amalnerkar: Compromise Bias and Efficiency: The estimator of mean will still be unbiased, unfortunately it will increase the variance of the sample means. Keeping 10 samples out of 27 possible samples would not reduce the variance of the sample mean. Only SRSWR with n^n will make unbiased and efficient estimator of mean. What will happen to CLT? Keeping less boots or increasing number of boots than n^n will certainly produce biased estimates. If boots are less than n^n, no doubt there will be bias and less efficiency. If boots are more than n^n, again there will be bias but more efficiency. The unbiasedness and efficiency will match with the CLT only if the number of boots are n^n and boots are taken using SRSWR. Yes, it is proven in 1950's that keeping distinct units in SRSWR ( or PPSWR sampling) leads to efficient results, that is what we named sufficient bootstrapping!

Dear friends

I have to compare two independent sample with the N1=7 and N2 = 85 and if I apply bootstrap method will it cause any problem because of sample N1 have only 7 observations? .

In a case of generating a phylogenetic tree (Aaron 900 tips from 3 different species)

How accurate to perform 100 bootstraps would be? Would it make much difference to use 1.000 or 10.000?