Suppose we assume that the data follows Gaussian distribution. We can set a threshold on its pdf to find outliers. How would it be different from setting a threshold on a pdf, fit the exceedances with Gen Pareto distribution and setting another threshold to find outliers. Are both methods not similar?
Shehroz. I understand that determining extreme values from assuming a Normal pdf distribution does not yield a satisfactory solution. This is a 100 years old problem without acceptable solution. Method from Gen Pareto exceedences is similar because it does not fit extreme data of univariate datasets.
The method I developed starts from a Lorenz curve built from dataset. It only asumes the two extreme values and the sample media made of N positive values. It uses a Lorenz function like L= (1+a)P - aPe+1 (P=cummulated population for top to low ordered data; L=fraction of total distributed mass received by most afluent fraction P). Values of "a" and exponent "e" are easy to find from Kmax/U and Kmin/U.
Remember that the derivate dL/dP gives the cummulate distribution function in medias of distribution, K(P such that K>=Ki) for data ordered top to low. K is the variable measured.
The final shape of K is K(P) =U*[ (a+1) - (a+b) Pe]
(where a=Kmax/U-1; b=1-Kmin/U; and exponent e=b/a). K(P) has inverse function P(K), and this one may be derived in the remote case that you want to know the pdf function. You will observe that it has no bell shape at all, because it is not assumed that pdf is simetric around K=1 media.
Give me the values of Kmaximum, Kminimum and media U data, and I will give you results that match both extremes, and a proxy-uncertain value of cummulative distribution function, either as P(K) or as K(P).
OK. Hope it helps to know that there is at least one approach that matches extreme values. Emilio
Thanks for your answer @Emilio. Do you have a paper or something written on this that I can look into?
Yes Sherhroz. It is a powerpoint made of 8 pieces and written in spanish (used for a short ponence). You must get somebody that helps in translating it into english. I anex it here so you may look into it.
I will be glad in answering your questions, if needed. Ok, best wishes, Emilio
- Badges
- Science topic