Dear "I." - 

Long ago, I recall reading about order statistics in a book by Gumbel, perhaps it was the following:

 Statistics of extremes. 1958

EJ Gumbel - Columbia Univ. press, New York, 1958

You can look at the distribution of the largest, smallest, third largest, whatever, of perhaps a sample of time to failure events.   

Is this the kind of statistic in which you are interested?

For extreme values, I found these discussions on the internet:

Concentrating, I think, on talking about High extreme:

 http://www.math.uah.edu/stat/special/ExtremeValue.html

Concentrating, I think, on talking about Low extreme: 

 http://www.itl.nist.gov/div898/handbook/apr/section1/apr163.htm

The latter handbook has seemed to me to usually be fairly good, though I found problems with the way they represented another area in which I have more expertise, i believe.  But I suspect the above may be helpful. 

If you are saying that you have a large value that you expect might be an outlier, I'd usually say that you would look at a population distribution, and if it is far out in the high end tail, you have to consider whether it is a mistake - not really part of the distribution of interest - or a legitimate extreme value, and that has to depend upon scientific knowledge of the topic, the data collection methodology, and the consequences of making one error versus the other (keeping it when you shouldn't or not keeping it when you should). 

Here, however, if you are interested specifically in the extreme value(s) of some time-to-failure data, perhaps you should be looking at such a large (small) observation in light of that extreme value distribution, not the population distribution (standard deviation), or standard error of the mean. 

For cross-sectional data with which i worked a great deal, i would suspect that the central limit theorem would mean that from a sample (or random factors of estimated residuals in weighted least squares regression), if sample size is large enough, you could look at the tails of a normal or t-distribution for the sample mean to usually find suspected outliers that could be most damaging. (A true outlier could look like a legitimate value, but the extreme ones could hurt results more, and are the only ones the data alone might indicate.) 

 However, if you are specifically interested in an extreme value (high/large here?), then I suspect that you want to be more hesitant to throw a large one out.  If you look at the order statistics for the largest member of a sample, I think that will work better for your purposes, if I am anywhere close to understanding the question.  The distribution for the extreme value would be of interest, not that of the mean, or of the estimated population.   

Remember that it is generally best to keep data you have collected unless you have a good reason to believe it is in error, not just because it is prominent on a graph.  That is just a good reason to look into it. 

It has been many years since I looked into order statistics, for something at work, and Gumbel's book was old then, but I remember finding it very interesting.  -  He started each chapter with a note in Latin, as I recall.  That was not so helpful to me.  :-)     -  There may be better books, and more on the internet.   Hopefully I have not greatly misunderstood your question.  It has been a long time since I looked at this kind of thing, and may have forgotten a great deal.  New theory might also be helpful.

Cheers - Jim

Dear Jim, thank you for your answer, you are completely right while saying that I am hesitant to throw a very large value while dealing exclusively with the extreme value distribution family, specifically for our strong winds project. Especially, when given exclusively high measured value is in good relations with other measured parameters. After I state that this measured value is valid, and after I estimate the distribution parameters from my data and build a theoretical curve along with non-parametric positions for measured data and all is beautiful apart from tgiven "too" high value that is placed "kilometers" from its destination on the theoretical curve, I still do need to know whether I may call it outlier or not. Regards, Isabella

If I understand the question correctly, my response is this:  The value in question may be an extreme value, but if it is a valid datum you have collected, say without equipment error,  then it is not an outlier. 

The way I see it, an extreme value may be suspected of being an outlier, and many people may just call extreme values outliers, but I think a better definition of "outlier" is a datum that does not belong to your population (and I think I saw that definition somewhere).  It could be in a tail of your wanted population distribution, or anywhere else.  Only when an outlier is in the tail of your distribution might you suspect you found it, but remember that 0.1% of the data should be in a 0.1% tail.  If you throw them out, you are throwing out good data, not 'outliers.' 

I'd say that if you think the extreme value you are examining is legitimate, then it is just an extreme value, not an outlier. 

Thank you Jim. It is just for our working meeting tomorrow with an important client, so I need to defend my arguments for validity of a "kilometers-distant" datum that should not be called "outlier". All the problem is too short time series, only 25 annual maxima, one per year, so if I would have, say, 50-70 years of automatic measurements, this year's datum would be merely the highest one fot that 50-70-y series as well. So it is valid. Thank you once again. Isabella.

Nothing can be added to improve Dr Knaub's answer. 

I've been confronted with that question recently w.r.t. a very leptokurtic distribution applicable for reliability: in general terms, one can design a significance test based upon the hypothesized (i.e. extreme value) distribution, which gives the probability of observing the suspect value assuming the distribution for the size of the sample containing it. The challenge is to estimate the parameters of the distribution required for the significance test without being biased too much by the suspect data point itself! In practice, it's safer to use (e.g. maximum likelihood) estimators based upon the sample without the suspect value. If the latter still corresponds with a significantly low probability, it should be considered a true outlier and discarded. 

Thank you Philippe. The samples I am dealing with, are too short for the estimates obtained with MLE to be reliable, so I am using the Method Of Moments. Is your advice held for MOM as well? The point however is that my unusually high measured value is in a good agreement with other measured parameters.  Regards, Isabella

Mind that higher order moments are terribly sensitive to the presence of outliers! When using moments based upon the sample without the suspect value, I figure that should still work. 

Hi Philippe, I spoke about a valid member of the time series that passed QC. These is no reason for its not-participation in any parameter estimation.  I'll try to refine my point: maybe we need any super-extreme approach, because if we had, for example, a 100-y series of annual maxima, and not the short 25-y series, that already recorded super-extreme value would still be a highest member of that imaginable 100-y series, so it is an absolutely legitime value, in addition, recorded in a good agreement with other measured parameters. For example, in QQ-plots, maybe instead of the non-parametric extimates of CDF of the ordered data, we need to omit several dozen places in between this highest value and the rest of the series, so that the vacant places could be fulfilled from the future measurements. Hope, my point is clear: to leave several dozen of plotting positions for future. I would like suggest some "inward" censoring approach, not usual censoring dealing with not-knowing either pre-lifiteme history, or post-lifetime history, or both, but not-knowing "in-between". I would appreciate your opinion.

Hi,

actually what you propose with your statement "so that the vacant places could be fulfilled from the future measurements", can only happen if there is clear auto-correlation between those future data points, which corresponds with a systematic (as opposed to a purely extreme value distributed) trend. 

If the data are supposed to be mutually independent samples from the hypothesized extreme value distribution, what you propose would not happen unless there were some clear systematic trend in the time series (akin to the challenge of proving global warming with several different seasonal factors playing a role ... ).  

Talking of which, DO you expect any seasonal influence in your series? 

You should realize that when increasing the sample size by complementing the historical data series with more (future) measurements, those should fill in gaps at arbitrary positions in between all those already collected, and not just filling the gap between the highest value and the rest of the existing sample values, if your hypothesis is supposed to hold! THAT would be a very improbable result! 

Assessing the probability of the highest (suspect) value for being observed in a sample of a given size using the remainder of the data for estimation of the extreme value distribution parameters is the best you can do in my opinion. I commonly use an outlier screening procedure based on this concept.  

As I see it, the very definition of an outlier almost coincides with its reason for non-participation in any parameter estimation, That's exactly my point: a true outlier should never be included in a dataset used for estimation of the parameters of a hypothesized distribution, especially when using the method of moments! 

Let me try to clarify it differently: if the suspect value DOES pass the significance test, one should still perform the parameter extraction again by including it in the dataset. The distribution parameters found should still be the same within error bar then (I know, computing those error bars is mostly even a bigger challenge than the parameter estimation itself). The problem resides in the VIOLATION of the null hypothesis however (as often is the case in statistics ... ). The outlier can bias the parameters found so heavily that the significance test becomes insignificant! 

Thank you Philippe for the well-argumented explanation of your position. Here, however, I would bring an example from the meteorological reality: we have got the records from the most of the storms in the past, while a few severe storms broke the old recording equipment. In this way, our maxima series consists of the old partially recorded maxima and all new recorded maxima. And now, we have a huge gap between the today's "super-high" value and the rest of the maxima series. So, I would be tempted to keep this gap for the past extrema which were not recorded simply because the old equipment could not survive in those past storms, and to calculate the non-parametric CDF of the observed values for QQ-plot with the corresponding gap in-between their plotting position. This is what I call "inward censoring". What is your opinion? (Besides, for computing the error bars I would apply the Monte-Carlo procedure whenever possible.) Regads, Isabella

Aha, that's a different situation opening up the possibility of such a "gap". This limitation can only properly be captured quantitatively using a likelihood formulation involving a different right censoring limit (thus allowing for inclusion of the cases where the equipment was broken) for the older data and the more recent data respectively. Mind that what you propose is only an approximate trick to capture this. 

Sure approximate! To be exact, we must keep aside a known number of ranks for the missing data in the sorted series. In this way, the maximum rank (i.e. length of the series comprised from both the recorded and missing data) accounts for the missing data as well. Now, what is left (if we deal with Gumbel distribution e.g.) is, given the ranks, to compute the non-parametric CDF(X1) and CDF(X2) for the minimum X1 and the maximum X2 values of the recorded series with any Plotting Position formula (say Gringorten), and from here the Gumbel variates Y1 and Y2, for X1 and X2, (Y=-ln[-ln(CDF(X))], and from here the two parameters of the distribution, A and B (given X1=A*Y1+B and X2=A*Y2+B). I cannot see any other method like MLE or MoM accounting for the missing data in more preferrable way. For example, MLE with Multiple Imputation fills in estimates for the missing data, while we should avoid estimating the extrema. I'd like to know your opinion. Best regards, Isabella