If the integer variable has a small range I would enumerate it. Otherwise you may try to first solve the relaxation of the problem to obtain a bound for the solution.(that is make the integer variable continuous). This only is promising if the function gets continuous when making the integer variable continuous.

For the solution of the continuous (sub-)problem I advise:

Try the Newton method, or even better the DFPS of BFGS method, from different starting points, If you always end up in different optimal solutions the problem is more difficult to solve, that is it might be non-convex or the optimum is at a constraint boundary (variables take an value at the end of their definition interval).. In the first case you might use global optimization algorithms. I have good experiences with an algorithm called CMA-ES that is part of many optimization packages and can be downloaded from the website of Prof .Nikolaus Hansen, Inria, France. There are also special algorithms for dealing with problems where the optimum is at the border of the variable space. It might be interesting to use a Generalized Pattern Search method for Box Constrained Optimization. This method is fairly simple to implement and searches also along boundaries:

http://www.ima.umn.edu/talks/workshops/1-6-7.2003/dennis/IMALec2print.pdf

As you have only 3 continuous variables you might as well compute a 3-D visualization based on a grid in, for instance and use slicing to solve the problem by inspection.

you could use ADMM, to solve this problem,however,maybe hard to proof that the convergence of the algorithm,but numerical I think it is work.Also I dont know whether your parameters are separable or not,which have some influence,

When dealing with integer ,maybe do a projection?

I am not sure whether my answer could help you

Do you have any idea of the permissible range for the integer parameter? Is it Prime?.

Not knowing the permissible space for your integer, it is difficult (at least for me to give a sensible suggestion. But you might think of using a homotropy method that allows you to explore the integer space. For example, define a homotropy function that depends on some parameter t ( such that for t=0. you have an optimum for a starting integer value , say Int_1. Then use the homotropy method to vary Int smoothy from Int_1 to the next integer Int_2 say. You can use a first order continuation method to find a good guess for your Newton iteration method as you proceed along the homotropy curve to the next integer....

In what form your 'long and complicated function is given' ? If there is anything human-readable (formula, C code, Mathematica code , ...) you should consider to post it here

or to send by e-mail to me (ulrichmutze at gmx dot de). This would inspire solutions.

I once made good experience with adding penalties to the original aim-function which represent the constraints. So, I would introduce a real parameter instead of the integer-valued one and add a penalty for the case that the value is not integer. Then I got an unconstraint problem with a few real parameters for which my Brent-like optimizer worked very fast to the optimum.

If the integer variable has a small range I would enumerate it. Otherwise you may try to first solve the relaxation of the problem to obtain a bound for the solution.(that is make the integer variable continuous). This only is promising if the function gets continuous when making the integer variable continuous.

For the solution of the continuous (sub-)problem I advise:

Try the Newton method, or even better the DFPS of BFGS method, from different starting points, If you always end up in different optimal solutions the problem is more difficult to solve, that is it might be non-convex or the optimum is at a constraint boundary (variables take an value at the end of their definition interval).. In the first case you might use global optimization algorithms. I have good experiences with an algorithm called CMA-ES that is part of many optimization packages and can be downloaded from the website of Prof .Nikolaus Hansen, Inria, France. There are also special algorithms for dealing with problems where the optimum is at the border of the variable space. It might be interesting to use a Generalized Pattern Search method for Box Constrained Optimization. This method is fairly simple to implement and searches also along boundaries:

http://www.ima.umn.edu/talks/workshops/1-6-7.2003/dennis/IMALec2print.pdf

As you have only 3 continuous variables you might as well compute a 3-D visualization based on a grid in, for instance and use slicing to solve the problem by inspection.

To simplify your problem you may try to fix 3 or 2 parameters and plot the

restricted function of one or two variables. The graphs could give you an idea for a general case.

Dear, Chuan Chen, Brian Higgins, Ulrich Mutze, Michael Emmerich, Gro Hovhannisyan

I would try your suggestion and I will inform you the results. Thank you very much for your time and effort.