I created a new test for uniformity, but so far, I've had no luck finding its critical values analytically, I could only obtain them by Monte Carlo simulation. What's worse is that histograms show that the null distribution does not approach normal distribution even at large n, so I cannot approximate it with mean and standard deviation.
Is there any sort of "standard procedure" for deriving null distribution of a test statistic? Or at least approximating it with an analytical expression?
Hi, Igor,
I can tell you how Rob and I discovered the analytic solutions which underlie the univariate optimal discriminant analysis (UniODA) algorithm, which led to the development of classification tree analysis (CTA) and ultimately fueled the discovery of novometric theory.
- https://odajournal.com/2017/04/18/what-is-optimal-data-analysis/
- https://odajournal.com/2016/09/19/novometric-analysis-with-ordered-class-variables-the-optimal-alternative-to-linear-regression-analysis/
We started with a simple idea--how to discriminate 2 classes from each other (2-tailed test) with maximum possible accuracy on the basis of a single random variable. We started with the case of the smallest possible sample size, and then systematically increased the sample size by one unit. This made the discovery of the analytic methodology required to identify the p value for a given sample geometry extremely obvious--but difficult to calculate as N became large.
- Yarnold, P.R., & Soltysik, R.C. (1991). Theoretical distributions of optima for univariate discrimination of random data. Decision Sciences, 22, 739-752.
We then extended this methodology for a 1-tailed hypothesis, and enumerated the optimal distribution. We discovered that, unlike the 2-tailed analysis, the 1-tailed analysis has a closed-form solution.
- Soltysik, R.C., & Yarnold, P.R. (1994). Univariable optimal discriminant analysis: One-tailed hypotheses. Educational and Psychological Measurement, 54, 646-653.
However, discovery of a closed-form solution is not a proof. So, we had to present a formal mathematical proof.
- Carmony, L., Yarnold, P.R., & Naeymi-Rad, F. (1998). One-tailed Type I error rates for balanced two-category UniODA with a random ordered attribute. Annals of Operations Research, 74, 223-238.
I don't have copies of these three pay-for-view papers to send. Hopefully anyone interested can obtain them for free from an academic library.
Good luck in your quest...
I'm confused. This is a test to see if a distribution appears reasonable if it had been drawn from a uniform distribution, like U(min=0,max=1). Why would you expect it to look Normal? Or do you mean for some test statistic that you have developed, and if so you should say what it is.
You might start by looking at how the distributions for existing tests of uniformity are obtained. This seems to be via theory for Weiner Processes (also called Empirical Processses). A book that contains all this is the following:
Empirical Processes with Applications to Statistics
G.R. Shorak & J.A. Wellner
Wiley , 1986 ISBN 0-471-86725-X
(there might be possibly later editions)
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Probability
- Probability Theory