The *ideal* way was to use a statisics software that performs a nonlinear regression and that gives you the confidence bands (for the estimate and for the prediction).

A good work-around is to transform the data so that the relationship is linear and to perform a usual linear regression on the transformed data. If there is no obvious heteroscedasticity in the residuals, the confidence bands for the regression line are trustworthy. You might then back-transform the confidence limits to the original scale (not quite correct, statistically, but acceptable). You can do this in Excel, for instance (e.g. see here: http://people.stfx.ca/bliengme/ExcelTips/RegressionAnalysisConfidence2.htm).

Klasse, Danke!

Sounds & looks perfect for probability plots, where you always try to fit a straight line. However the confidence band looks rather narrow, I guess it just covers the regression's prediction error, right? How about the sampling error? When I generate multiple exponential-distributed samples using always the same parameter, and plot those, the plots scatter more than the regression CI suggests.

Freut mich :-)

I suppose you are looking for the prediction interval / prediction band. The confidence interval is for the mean (that is, for the regression line), whereas the prediction interval is for the expected range of new values/data. In the link there are both intervals shown. The calculation differs in the "1" under the square root. The upper formula gives the confidence band, the lower one gives the prediction band.

Wer liest, ist König. Danke!

@ P.Roussel:

Please correct me when I am wrong. I possibly misunderstood you.

As far as I understood it seems to me that you did not correctly understand the problem and the answer. There is no CDF used anywhere. I said that a non-linear regression was the method of choice (and this method is based in MLE). I also clearly stated that the solution based on a transformation is a work-around, which is not mathematically/statistically correct, but often good enough to serve the purpose. I also noticed that the variance of the residuals should not depend on the predictor.

Concerning OLS or MLE: OLS is just a special case of MLE, when the error model is based on the normal distribution. Please be aware that this model is about a probability distribution, not about a frequency distribution. The normal probability model applies when the residuals are on the ratio scale, idependent, and when we have no reason to assume that their distribution is assymmetric. For the transformed data, ratio scale is given. Independence should be asserted by the way the experiment is performed. However, recognizeable dependencies between residuals and predictor values (for instance heteroskedasticity) indicate a violation of the indipendence. This means that we *could* use an improved model (error or prediction), since we *could* extract more (appropriate) information from the given data. It still has to be decided by the expert if this is required or wanted. Often, the conclusions for the relevant predictors do not change much. Even more often, it is not really clear how the model may be improved, just because we don't know. Using the normal probability is the "worst case error model", requiring the least prior knowledge about the "error generating process".

I further did not understand your argument to use MLE and then take a solution based on an approximation for "sufficiently large samples". This approximation is based on the fact the log-likelihood function approximated a parabola, which is the again just the log-likelihood for the normal error model. Hence, this essetially *is* and OLS.

Ok, I think these questions can only be answered by or in correspondence with Bernard....

Guys, big thanks for your interest and your help, this is great!

Well, I'm analyzing task completion times in usability tests. There is plenty of literature that those times are not normal distributed but I couldn't find any information about what the distributions actually are. Then I came across probability plotting, described in the NIST e-Handbook of Statistical Methods: http://www.itl.nist.gov/div898/handbook/apr/section2/apr221.htm

Right now I'm using Excel for calculations and to fit a linear regression. Typically the exponential plot fits very well, but the regression line intersects the time axis not at 0 but at some time t0 (i.e. the distribution is translated) which is roughly the time an expert user takes to click through the task. The exponential distribution parameter Lambda then describes the rate at which test participants solve the task, i.e. it appears to be a very interesting efficiency metric. Orders of magnitude range from 10 to several hundred s for the overall task completion time, typical values of t0 and Lambda are 30s and 0,003, respectively.

So far I think probability plotting is extremely helpful for usability engineers: you can easily spot outliers; t0, apart from being a metric for mere mechanical click-through time, can help to check whether or not the technical system performance is comparable in retests or comparative tests; and the solution rate Lambda would be a metric of usability that reflects directly to which extent users spend time struggling with the UI, as opposed to using it.

However, when reporting t0 and Lambda as metrics in usability testing, I'd like to be able to draw my error bars, and correctly so ;-)

That's really simple.

There still is one more complication - usually in usability tests, not all users complete the task but some stop attempting after trying for a certain time. In other cases the user thinks he's finished but in fact is mistaken. So we have the information that a particular user has not solved the task until that time. Following the description in the NIST handbook I'm treating the data as multi-censored and calculate the vertical plotting position using the "modified Kaplan-Meier" estimate described here: http://www.itl.nist.gov/div898/handbook/apr/section2/apr215.htm#Modified K - M

It's quite important to have such a correction, in particular when comparing designs, when not always the task completion rates (i.e., n) are identical.

Is there also a parametric estimate for Lambda and t0, for the multicensored case? That's probably a stupid question; I just have my psychologist's training in statistics and not much of an idea where to look. All I found so far is the plotting stuff...

Thanks a lot, this is exactly what I need!

For a first approximation, I can derive t0 from the plot, or use minimum times instead. I found some Reliability Analysis literature an how to derive failure-free time intervals. Conceptually Lambda is much more interesting as a metric, and for comparative testing you would anyway inspect the two distributions side by side.

You really helped me a big step forward!

Thanks, Bernard