The Monte Carlo simulation of synthetic data described above by Jan will not work properly if the distribution of the errors is not known.

If you don't know the underlying process that had produced your data, it will be much safer to use a Monte Carlo bootstrap re-sampling method and the related bootstrapping approaches to inference (see in particular the work of Saltelli et al. on sensitivity analysis)..

The typicall procedure to determine the precision with which the parameters can be determined is estimation of their confidence intervals. It is done by using the Monte Carlo method. You take the best fitting curve and on its basis you create syntetic data set by perturbing randomly the solution at the time points of mesurements. Then you perform the data fitting procedure once again for the new (synthetic) set of measurements. You do this n-times and then calculate confidence intervals.

I don't know if I understand, but we analysed sensivity of the model determined as elasticity of function calculated for variable. It is clearly described in my article I attached

I hope it will help

Article Model Of Multilevel Stochastic Analysis Of Road Safety On Re...

There exists a test (Jarque-Bera) for the normality of the errors for a linear model. In fact it is one of the hypothesis for validation of the linear model.

In jrb1.pdf (published in Romanian Journal of Economic Forecasting, No. 4/2009, and it can be also accessed through REPEC) it is proved first that there is an equivalence between the normality of Y and the normality of the errors.

Next there is presented (using the lemma of Kintchine) a way to transform non-normal Y to a normal resulting variable Z.

The interval of confidence of Y in the normal case, depending on the values of X1, X2,..., Xk is built taking into account the variance of Y in the normal case, depending on the values of X1, X2,..., Xk,

Sigma(1,X1,X2,...,Xk)+sigma_u*sigma_u,

where sigma_u*sigma_u is the variance of the errors, and

Sigma=sigma_u*sigma_u*(X'*X)^(-1)

is the variance-covariance matrix of the linear regression coefficients.

Sigma(1,X1,X2,...,Xk) is the squared (positive defined) form apply to the values 1, and the current values of Xj-s.

Of course, the interval is expectationplus/minus standard deviation*Student quantile.

For non-linear confidence interval, we apply the inverse transformation to the heads of the normal interval: first the normal cdf, to obtain uniform interval, and next the inverse of real cdf.

.

polynomial chaos and its link to Sobol expansion ?

http://www.gdr-mascotnum.fr/media/impec07_crestaux.pdf

http://bruno.sudret.free.fr/docs/2006%20-%20CSM5%20Sobol.pdf‎

.

see:

Metrologia 44 (2007) 246–251 doi:10.1088/0026-1394/44/3/012

Metrologia 43 (2006) 396–402 doi:10.1088/0026-1394/43/5/009

regards

You can plot R^2 versus asymptote of the nonlinear equation and visually detect the sensitivity around the optimal asymptote. See my book "nonlinearity in life and biomedical Sciences", where you can find many numerical examples. You should download some of my PDF files to examine the examples.

Thank you very much, everyone, for detailed responses! I will read the resources you provided and reply as soon as possible.

When the model is fitted, either by least squares or by maximum likelihood, the Fisher information matrix gives a first idea of the sensitivity of each parameter, being rhe second order partial derivative of the criterion function (LS or ML) respectively to each parameter or couple of parameters. The elasticity can be obtained by 1) Cholesky decomposition and 2) division of the diagonal by the central value of each parameter (one obtains the relative standard deviation). The extra-diagonal elements are less straightforward to interprete.In the neighbourhood of the LS or ML, the non linear model is very close to its linearization (principle of the Gauss method)

Compute their confidence range, by repeating fittings of data and their perturbations. Or estimate sensitivity of the optimal solution.

I just add that the procedure I propose is exactly the analytic equivalent of the Monte-Carlo method proposed by Jan Poleszczuk.

The Monte Carlo simulation of synthetic data described above by Jan will not work properly if the distribution of the errors is not known.

If you don't know the underlying process that had produced your data, it will be much safer to use a Monte Carlo bootstrap re-sampling method and the related bootstrapping approaches to inference (see in particular the work of Saltelli et al. on sensitivity analysis)..

First see if the parameters are probabilistic, in which case sensitivity can be studied accordingly. If the parameters are fuzzy, then there are standard techniques to study sensitivity.

Thank you again, everyone, for your contributions!

H.E. Lehtihet: your answer is very useful, thank you! Can you please clarify is the bootstrapping you propose done with replacement? In other words, if there are N observations in the original data set, will each random data set also contain N observations, some of which can be repeated?

Igor,

Alternatively to the least square fitting you should consider robust curve fitting in combination outlier detection and removal. This works sometimes better when outliers have a strong impact on your curve fitting algorithm.

Generally it is useful to do some data exploration work before you analyse your data. I am not sure which software you are using, but Matlab for example gives you a good statistical analysis of your curve fitting parameters.

KR

Jens

A result related to your question is the following

QUESTION: Does there exist any linear model that can EXACTLY fit the nonlinear data (this is not a least squares question), where "data" means the 2q sets of data: {i)The first q cross-correlations between input and output and ii)The first q auto-correlations of the output}

ANSWER: A necessary and sufficient condition is known for the answer to exist (relates to properties of the data), and an algorithm is available to compute ALL state space model which can exactly match the data. Such Models are called "q Markov Covers" and their derivation is listed in the publications on my ResearchGate website.

Igor, >> Can you please clarify is the bootstrapping you propose done with replacement? In other words, if there are N observations in the original data set, will each random data set also contain N observations, some of which can be repeated?

Thanks a lot again to everyone for their contributions!

So, do I understand correctly that the following generalized procedure should work for most experimentally-generated data sets?

1. Examine the data distribution (e.g. graphically and by normality testing).

2. If normal, perform parametric bootstrapping.

3. If not normal, but without a lot of extreme outliers, perform nonparametric bootstrapping.

4. If has lots of extreme outliers, perform transformation (e.g. log) to reduce their effect.

Does this make sense?

Thank you in advance!

Igor,

I think your workflow makes sense. You can have also a look at the boxcox transformation which is a slightly more sophisticated way of normal transforming your data. Which method are you using for outlier detection?

Maybe publish the data or a similar set of data and let other people have a go at it which will make it easier to give advice.

Please feel free to have a look at the paper listed below. I thought it was very useful and well written.

Edit

Fitting models to biological data using linear and nonlinear regression: a practical guide to curve (Citations: 252)

H. Motulsky, A. Christopoulos

Published in 2002.

KR

Jens

Igor,

your way of thinking may be close to the one I described in arxiv/0903.0365 (also on my RG page) article " Breakthrough in Interval Data Fitting II. From Ranges to Means and Standard Deviations". The difference is that it is not you, who says how much to increase the value of chi^2 to find the edges of the interesting region (you propose 10% - why just this number?). In my approach, this number is found as a byproduct of a procedure, you don't even have to know it. In fact, it depends not only on the precision of the data but also on numerical accuracy of your computer and on the complexity of your non-linear model as well. Here you don't have to worry about normality of your data, it is enough to know their standard deviations. The important point is to fit your data in their original form, not transformed (well, linear transformation is acceptable - this is like chnging units) as this almost certainly introduces some bias. The method is usable not only for classical curves but for implicit relations as well. Do you realize that different metrics (Least Squares, Least Absolute Deviations, Maximum Entropy, etc.) will produce different results?

Hope you'll like my approach.

Thank you, Jens, for your comments and suggestions! About outlier detection and normality testing - what would you (and anybody else interested) recommend for relatively small data sets (e.g. 10-100 observations)?

Thank you, Marek, for a useful reply! I will read the paper with interest. The 10% number I quoted was just an example, but of course it would be better to find a more "objective" cutoff value. And certainly it makes sense that different fitting methods can produce different results (sometimes a lot different).

Igor,

I don’t think that there is a formal method that defines an outlier. There is always some sort of judgement required. Useful methods for the identification of outliers are boxplots, principal component analysis, histograms, density plots and group scatter plots.

I am using Matlab and there are a few functions and plots that test for normality (normplot) and random number generators that generate data sets of a certain distribution.

Feel free to send me your data if they are not confidential and I have a quick look at them and then I can be more specific. [email protected]

KR

Jens

Igor,

It seems to me that what your may be interested in finding is a statitsctiacl description of you nonlinear regression parameters. In which case, since you only have your given data set, you have to rely on parametric estimates of their standard errors. This is the least you can do by standard least squares. Howver, there are approaches tha tallow you to estiamate your parameters and associated uncertainites. One relies on nonparametric bootstrap methods, wich estiate the distribution of your parameters based onr resampling. One othere approach is the generalized least squares whihc allows you incorporate the uncertainty in your data set (each data point and in all variables) whihc wich utilizes this information for estiamting both the mean and standard errors of your parameters. This method deffers from the standard LS in that has a different objective function to minimize teh residuals of your nonlinear fit. It is suppose to give you the best estmates of the fitting paramters as well an indication of its unceratinties as propagated from the

mean value an its respective unceraintis of each poiint in your data set.

Hope this helps

Thanks a lot, Jens! I don't have any specific data set in mind, just trying to figure out a useful strategy for dealing with relatively small data sets (usually

Thanks, Juan! Nonparametric bootstrapping does sound very useful to me!

I like to use both the bootstrap method and F-test to calculate my confidence intervals. If your function only depends on two parameters then the F-test is easy to run, but if you have more then two parameters it is hard to run over parameter space. I then use a F-test while only varying one parameter at a time and plot each parameter.

One good way is to calculate likelihood intervals. My own LS curve fitting program (CVFIT) has done this for many years. Insofar as your errors are Gaussian, the fit is max likelihood. The likelihood intervals are the tangents to the contours for Lmax - 0.5 or Lmax -2, where Lmax is the maximum log likelihood. Hold the parameter for which intervals are wanted constant a re-fit -use bisection to find the value for the fixed parameter the gives Lmax - 0.5. Such intervals will not generally be symmetrical,

A cruder way is to calculate the covariance matrix, by calculating numerically the partial 2nd derivatives for all pairs of parameters to produce an information matrix which you then invert. The diagonal elements give you variances for each parameter (and hence symmetrical limits) and the off diagonal elements give you covariances and hence correlations between pairs of parameters.

It's important that these methods assume observations are independent. Like most methods, they won't work for digitised currents (unless they are sampled below the Nyquist frequency, which is rare.

Thank you, Rolando and David!

David: is your program commercially available or open source?

CVIT.EXE It's free and open source (it's old so DOS but it can be run under Windows 7 and on Macs)). See http://www.onemol.org.uk/?page_id=8

Oops, cvfit.exe

Thank you, David! I will download your software!

As an observer, I found all this discussion also very useful for me, I'm working on pharmacokinetic where this situation is very common.

Ignacio, how do you usually approach this issue of parameter sensitivity?

I have no any particular method to deal with parameter sessitivity. I'm using Monolix for pharmacokinetic profiles evaluation, which is quite enough for me. Monolix software provide log-likelihood estimation, visual prediction cheking, shrinkage of parameter estimation and many other information useful in any model fitting situations.

Thanks, Ignacio! Is Monolix a commercial software or open source?

Thank you, Alessandro, this is interesting!