With a non-convex non-linear optimization problem, how can you increase or amplify the remaining residual to get a better answer?
The objective function or optimization problem is a minimization.
The residual is already very low - of the order of 1e-6 or 1e-10. But the solution is still far away from the known optimal solution, given that it is a lab case.
It is a sum of differences squared. So small differences are made even smaller.
How can one amplify the remaining residuals to search them still?
I thought of multiplying the residual with a large amount, like 1e6 etc to inflate it again.
Any suggestions welcome
Hi, what do you mean by "known optimal solution", the global optimum? Which residual are you talking about (the residual of the KKT optimality conditions)?. Best
what should scaling up the residuals help. Your factor 1.0e6 is equivalent to multiply the sum of squares by 1.0e12 and nothing in the sought solution will change, neither the surface on which you move.
what you describe lets one think about bad scaling of the optimization variables
(think about a valley, flat in one direction involving one or some variables which
is however very narrow with steep walls aside.) Also a bad conditioning of the
problem setting might be the case. Sometimes I experienced the same effect but the reason was a wrong model a user presented to me : he simply forgot to add a
constant to the model where obviously it was necessary to have some. Hence you should carefully think about your model.
hope this helps
peter
As suggested, it's important to properly scale your problem. It could also be that your problem is rank deficient near the solution you're looking for, so that many solutions give approximately the same residuals. If you have an idea of what the solution x* you're looking for looks like, you could try regularizing your problem by adding a multiple of ||x - x*||^2 to your objective.
If all else fails and your model is correct, you could try restarting the solver from the current best solution using extended precision computations, provided you can do that efficiently (e.g., with GCC's quadmath library).
@ Richard , it is a simulated non convex non linear problem, so i know the global solution of the nonlinear problem
Specifying the objective/ objective function as a sum of absolutes, rather than a sum of squares should help too..?
Min X = sum(abs(x1-y1) + abs(x2-y2) + abs(x3-y3)...)
Vs
Min X = sum((x1-y1)^2 + (x2-y2)^2 + (x3-y3)^2...)
Would this make the optimization problem more complex?
- Badges
- Science topic
- Similar topics
- Mathematics
- Applied Mathematics