I used differential evolution to determine the interatomic potential parameters of a model by minimizing a objective function that is the sum of the squared difference of experimental and calculated values of a set of properties. Now I'm trying to calculate the covariance matrix (and correlation matrix) for the fitted parameters. Is there a method for numerically calculate this matrix by probing the objective function? I've been able to numerically calculate the Hessian at the minimum and tried to use it's inverse as the covariance matrix, but the resulting correlation matrix has values that are outside the range (-1,1) so I'm no confident it is correct.
I've found a solution and will leave it here in case somebody else comes across the same issue.
Once you have optimized the parameters of your model and found a minimum of your objective function, the correlation between the parameters can be estimated by constructing the correlation matrix. This is done by first calculating the Hessian matrix at the minimum. Since I do not have analytical derivatives for my objective function, I've done this numerically using Python and the numdifftools package. Then, I've taken the inverse of the Hessian as the covariance matrix, which have the variance of the parameters in its diagonal. Finally, I've contructed the correlation matrix element-wise by taking each covariance and dividing it by the product of the standard deviation of the parameters involved in that entry. This will result in a matrix with all its values in the range [-1,1] and ones in its diagonal.
Dear Giovani Rech,
There's another, derivative-free way for derivative-free metaheuristics such as Differential Evolution, to calculate the confidence intervals (and regions!) for the fitted parameters of a nonlinear model.
It is described in detail in the paper of Schwaab et al. [1] along with many practical examples. It works, I've successfully implemented it in FORTRAN, with no use of external library routines. The proposed method maps the confidence region for each fitted parameter, through a statistical correlation (eq. 16 in [1]) and the vast number of objective function evaluations produced during optimization with Differential Evolution or analogous (metaheuristic/swarm) methods.
Confidence intervals can thus be determined from the maximum and minimum values of the resulting confidence region around a given optimization parameter.
[1] Schwaab, M., Biscaia Jr., E.C., Monteiro, J.L., Pinto, J.C., "Nonlinear parameter estimation through particle swarm optimization", Chem. Eng. Sci. 63 (2008) 1542-1552.
Dear Giovanni,
The procedure you describe fits very well to many textbook's treatments. Thus all should work as expected, right? No, not necessarily - as you experience yourself. The first reason is that numerical values of 'best fitted' parameters are necessarily approximated, not exact. The same applies to the minimum found, its true value certainly differs from reality. All this is due to finite representation (accuracy) of real numbers in machine calculations and, perhaps even more importantly, to the finite number of steps of the minimizing procedure. So, you evaluate your Hessian (again: its numerical approximation) at the point where gradient is non-zero. Finally, the entire procedure assumes that experimental measurements are distributed normally. This assumption is never checked but very often it is simply false.
My own approach to similar problems, free from the doubtful assumptions ( http://arxiv.org/abs/1708.01452 ), always produces reliable results. The standard deviations (or variances) of fitted parameters may be (and usually are) overestimated, but their correlations are always in the correct range [-1,1].
It you need to compute the covariance, you can use Matlab or use the simple covariance model of
https://en.wikipedia.org/wiki/Covariance
- Badges
- Science topic
- Similar topics
- Engineering
- Materials Engineering
- Powders