Hello, I currently solve the optimization problem (please see the attached figure).
Basically, this problem is equivalent to find the confidence interval for logistic regression. The objective function is linear (no second derivative), meanwhile, the constraint is non-linear. Specifically, I used n = 1, alpha = 0.05, theta = logit of p where p = [0,1] (for detail, please see binomial distribution). Thus, I have a closed-form solution for the gradient and jacobian for objective and constraints respectively.
In R, I first tried the alabama::auglag function which used augmented Lagrangian method with BFGS (as a default) and nloptr::auglag function which used augmented Lagrangian method with SLSQP (i.e. SLSQP as a local minimizer). Although they were able to find the (global) minimizer most time, sometimes they failed and produced a far-off solution. After all, I could obtain the best (most stable) result using SLSQP method (nloptr::nloptr with algorithm=NLOPT_LD_SLSQP).
Now, I have a question of why SLSQP produced a better result in this setting than the first two methods and why the first two methods (augmented Lagrangian with BFGS and SLSQP as a local optimizer) did not perform well. Another question is, considering my problem setting, what would be the best method to find the optimizer?
Any comments and suggestions would be much appreciated. Thanks.
This formulation is wrong and unclear:
1. The index k in theta_k suggests that there should be something enumerated by k - but I see no such k elsewhere.
2. It appears to be a problem in the class of nonlinear resource allocation problems. I have written a survey on a problem like that, as well as a follow-up paper with a substantial test section, to see what method for the problem would be best. It appears that your problem is exactly the same problem. Here are my two papers on this theme:
a. A survey on the continuous nonlinear resource allocation problem
M Patriksson; European Journal of Operational Research 185 (1), 1-46
b. Algorithms for the continuous nonlinear resource allocation problem
M Patriksson
European Journal of Operational Research (3), 703-722
I trust you will have very good use of it!
Let me know if I can provide answers to some questions, too.
/Michael
Michael Patriksson
1. Thanks for the response.
Since my interest was (any possible reason) why SLSQP showed better performance than the augmented Lagrangian (AL) with BFGS in linear objective function with nonlinear constraint, I did not focus on explicitly stating my problem. But, I do agree I should have clarified the problem.
So, as I mentioned above, my problem is to find the likelihood-based confidence interval for the logistic regression. In the problem, I (capital I) represents the whole index vector for the model and I differentiate k and i because, in each iteration, objective function refers to k-th element in I, meanwhile, in constraint, we add all elements in I. I rewrite the problem in the attached figure, please let me know if there is still any confusion to understand.
2. I do appreciate that you introduced your two papers. I will review them thoroughly and think about how to connect them to my question and problem.
Sincerely,
Su.
𝗔𝗵𝗺𝗲𝗱 𝗠𝗼𝗵𝗮𝗺𝗲𝗱 𝗛𝗮𝗯𝗶𝗯
Thanks for your answer.
I believe there would be some insights behind your answer, but I could not understand your point. Especially, the immune-based algorithm sounds very unfamiliar to me and how is your second answer related to the question I posed.
I would be grateful if you could add more explanations as well as some references that support your answer.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Software
- R