By critical values, do you mean statistic values for which you would reject Normality? If so, and using simulation, conceptually you would just create lots of normal distributions (rnorm in R) for the different sample sizes and calculate your test statistic, then see what values would be appropriate for rejecting the hypothesis (e.g., one possibility would be 2.5% in each tail but other possibilities exist). You could smooth values a bit the relationship between sample size and critical values makes sense, or just increase the number of replications to get better precision. R can be slow for loops, so if your statistic is slow to compute you would probably want to avoid loops with functions like replicate() or sim() or the apply() set of functions (see Wickham's http://adv-r.had.co.nz/).
If you want to calculate these mathematically it will depend on your test statistic.
These kinds of questions are very useful, but also very difficult. You might start by thinking about whether you want to sample a distribution, or if you want to sample a population that is drawn from this distribution. The latter would be more appropriate if the primary use for the new method will be in research where populations are typically small. I am not quite sure about the quantitative definition of small in this case. Certainly anything less than 1000, probably anything less than 10,000. Anything 10 million or greater would probably be large. Some of the answer depends on the variability in the data.
There are two approaches. Daniel's suggestion was to use normal distributions. Thus the null hypothesis is true in these cases. This is where I would start, and I would plot a graph that looks at sample size and the probability of rejecting the null hypothesis.
The other approach is to use other distributions. These can be friendly like the uniform distribution runif(100,5,5) and rgamma(100,5,9), or unfriendly custom distributions. Think about things like generating a population consisting of rnorm(5000) and rnorm(5000)*6.4386 to give a bimodal distribution with two equal peaks. You then withdraw a sample from this population to test using your new method. Repeat, and you can either generate a new population, or you can resample the existing population. These custom distributions can be multimodal, skewed, and distorted to whatever degree you want. You can then run simulations to see how well the new method performs as the distributions become ever closer to normal. You can also see how well the new method performs as the two populations become more dissimilar. So how well does the new method work if you are comparing a normal population to a bimodal population where the two modes are within some distance of each other. So rnorm(5000) and rnorm(5000)*d where 0
Timothy and I, I think, are answering slightly different questions and it boils down to my first question, what you mean by critical values. I assume these were for null hypothesis significance testing of whether we can reject the hypothesis that the empirical distribution was drawn from a normal distribution. If you are using other distributions then you are not testing the null of the normal distribution being true.
However, if you goal is to get a confidence interval around the test statistic, then you would use whatever distribution you think is appropriate (perhaps via bootstrapping). I think this is what Timothy is addressing. Let's say your test statistics was kurtosis, which can be used to show non-normality. The standard errors and p-values that are given in many packages and textbooks assume a normal distribution (and you will get near these values with the approach I suggest), so have problems when the null hypothesis is false if using these to make confidence intervals. See the attached. It shows using the empirical distribution (and bootstrapping) produces better confidence intervals (but because kurtosis involves taking the error to the fourth power the estimates are still un-reliable in many situations ... there are more robust alternatives out there for tail-heaviness ... see references in the attached).
Article Problematic standard errors and confidence intervals for ske...
Mostly trying to help Usama think about the problem in a broader context, or perhaps trying to facilitate a further refinement of the question (as Daniel has asked for).
It is not clear if Usama has invented a new method, or if he is trying to better understand a method developed by someone else. Maybe critical values are going to be used to construct the equivalent of a t-table as found in introductory statistics textbooks. The methodology.pdf manuscript that was included in the question was interesting. If that is the goal, then at least part of the relevant program was published in an appendix at the end of that manuscript. Contacting those authors directly might get more help targeted to this specific problem.
thanks all .. i pretty know the steps theoretically and most of your answers but i need the codes in R with an example for any test if it is possible..
DEAR.PROF TIMOTHY.. yes iam trying to better understand a method developed by someone else but under other conditions.. this method with some steps to be implemented.. i have e-mailed them but no one have answered me yet.