Graph the convergence criteria and look at the incremental improvement in the model with the addition of each new simulation. Where you stop is the point where running another 1000 simulations does not lead to sufficient model improvement to be worth the time it takes to run the model. If you can provide a quantitative critical value for the incremental gain in model performance, you should be able to program the computer to stop upon reaching that value.

Alternatively, run 10,000 sets of 5 simulations. This is where the model is run with five simulations and you record the coefficient of uncertainty from the last run. Rerun this simulation 10,000 times. As a random process you will get slightly different answers from each run.

Do this again with 10,000 sets of 50 simulations.

And again with 10,000 sets of 500 simulations.

Proceed with 5000, and possibly even go to 50,000 simulations or more.

For each set of 10,000 calculate a standard deviation. The log-log plot of standard deviation and number of simulations should be a straight line (assuming that the model does in fact converge). You can now choose whatever level of accuracy is desired and defend your answer.

Thank you for the reply. An explanation with mathematical equations and a dedicated proof for the reply would be mostly appreciated. Furthermore, please provide a lucid form of explanation.    

I don't have any equations for this.

You have a complex model with some random elements. I assume the goal is to predict the most likely outcome of the model given that each run of your model has a slightly different outcome. However, collectively the average result from the model will converge to the true population mean. The more simulations that you perform the closer you would expect your model to be to the true population mean. So I assume that your question was how to tell when to stop. Can I run 5 simulations and I am done? Do I need 5,000,000 simulations? The answer assumes that the model does in fact converge. It also depends on the rate of convergence and the point at which you decide that the benefit from another simulation is not worth the effort. A model predicting the weather will not converge at the same rate as a model predicting the outcome of rolling 5 dice.

I am assuming that you are not interested in the outcome from a specific simulation. Rather, you are interested in how the model performs on average.

It is possible that I have not interpreted your question correctly. Would you ask the question differently if you were a mathematician trying for a graphical proof of a complex theorem, or a physicist simulating particle decay, a meteorologist looking at weather patterns, a zoologist looking at populations of frogs, or a social worker predicting mental health outcomes based on social norms? All of these people might use Monte-Carlo simulations of some type for the explicit reason that the problem they are working on involves equations of such complexity that it is difficult to define the problem in mathematical terms with dedicated proofs.

Are you certain that your use of "uncertainty coefficient" is the only definition from all disciplines in Science, or is it unique to your specific sub-discipline? Does a google search of "uncertainty coefficient" result in a helpful set of web pages?

Thank you. Suppose someone uses 50000 samples for the MCS. He proposes a new method for the study and compares the computational efficiency of the proposed method with that of MCS as reference. My concern is that, the number of samples  assumed for MCS (i.e. 50000) may be either high or insufficient for the accuracy of the obtained result. In this context, is it possible/ appropriate to evaluate a coefficient of uncertainty to determine the convergence of MCS. Is there any other concepts to obtain the convergence of MCS. I hope now you can understand the problem.

So you have two competing models. One is MCS (Monte-Carlo Simulation). You are interested in computational efficiency (time it takes to execute) and determining a defensible number of samples for the MCS.

So the next question is how are you evaluating the MCS? You talk about an uncertainty coefficient. How does the value of this statistic change if you take 5 rather than 50000 samples? If you take 5 samples calculate the statistic, and then take another 5 samples and recalculate the statistic do you get exactly the same answer? Obviously 5 samples is not sufficient, but it goes faster than if you start off at 5000000000 samples.

If the uncertainty statistic does not change, then something is probably wrong in the MCS.

What do you look for in the uncertainty coefficient? Is a coefficient of 2.95423532 good or bad? If this changes to 2.95421000 is that good or bad? Do you get 95% confidence intervals about the uncertainty coefficient? How narrow do these need to be before you consider your MCS good? If you cannot define a specific numerical goal how will you know if you accomplished anything?

Calculate the standard deviation of the results of the trials, (or at least the RMS) and require it (or them) to be below 20%.

Thank you. Can you please elaborate your answer?

1) Run the simulation several times (1 simulation gives 1 sample). Calculate the mean of these runs, and a standard deviation. The standard deviation should be less than 20% of the mean. Of course you can choose some value other than 20% if you want.

2) Run the simulation several times. Calculate the coefficient of uncertainty. Repeat this process several times. You now get a mean value for the coefficient of uncertainty, and a standard deviation. If the SD is < 20% of mean stop, otherwise increase the number of simulations and repeat. Do until SD < 0.20 * mean.

3) Use any statistic of your choice. Anything that your model produces. If there are multiple outputs then use all of them. Define what outputs are most relevant to your issue. Use that. The general process is the same. Conceptually #1 and #2 above are the same. The difference is that #1 uses the model output directly, while #2 uses a statistic calculated from several runs of the model. It is also the same as the other answers that I have given. However, the way that this task is performed determines what else you can do and how interesting the result is to other scientists. So with Keran's suggestion I can calculate a mean and standard deviation using 3 simulations. SD>20, so I go to 4 simulations. Then 5, 6, .... until I reach the 20% threshold. This might be a great deal of work if the threshold is reached at 3652 simulations. Alternatively, I could plan to try 10, 100, 1000, 10000, 100000, 1000000 simulations. This is now a linear regression problem where I get slope and intercept. I can then use this equation to figure out that my answer at 20% is 3652 simulations. If someone thinks this is too inexact they can use my equation to recalculate my results to whatever they think is appropriate.

Another thing that will help you is to keep track of time. 10 simulations take 2 seconds. 1000000 simulations take 4 weeks. This is part of the justification for using 20% versus 1%. Of course you have to change all of these statements as appropriate for your simulation. Do you attain 1% accuracy with a number of simulations that takes 5 minutes or 5 months? At 5 minutes I might insist on 1%, at 5 months I might decide there are better things to do (assuming you are using a fairly modern computer).

It would be difficult to improve on Dr. Ebert's explanation.

So what do you suggest.

My earlier reply, or Dr. Ebert''s first paragraph.

Now my question is why SD < 20% of mean. What is the significance of this 20%. If i have more than one output variables then shall i take maximum of (SD/mean) for all  output variable as less than 20%. Please clarify. Is there any other options where acceptable error and confidence level are accounted. Please clarify.

There is nothing magical about 20%. Choose a number that makes sense for the system you are studying. If thousands of people will die if you make a mistake then be more conservative (

If you use 20%, the central limit theorem suggests that the random distribution of results will be close enough to random that you can trust the mean.  Again, see Dr. Ebert's reply..  Remember, going to 10% will take you 4 times as long.  You can do that if you want and the code is fast enough.