You have to question enterprises partnerships... No ? It's generally an enough good way.

I'm not and expert in the area but have you looked at blackbox optimization algorithms ?

if not, you can check out the software NOMAD: https://www.gerad.ca/nomad/Project/Home.html

Take a look at bayesian optimization as well, it may be useful for you are trying to achieve.

It depends on which specific optimization method you are using and on the function you want to optimize. For example if you suspect or know that your function has many local minima, then you should vary the parameters in the way that you increase variability (for example in the case of genetic algorithms you would increase mutation or crossover probability in order to explore all the range of solutions and not to get stuck in a local minima). In other optimisation algorithms there must also be a parameter that determines the variability or spread of the solutions.

I hope it was helpful for you.

Regards.

A good way to optimize/adjust parameters for any meta-heuristic/heuristic algorithm is by the use of Design of Experiments (DOE). You must define ranges for any parameter  where you think the best values are. Among the advantages of this technique, I can point out that your experiments are run sistematically, so you avoid waist of time with meaningless experiments. You can also identify trends for the values of parameters and find out if the best value is inside your range or if you need further experiments. Another impotant feature is that you can analyse relations between parameters, so you are able to find the best combination of parameters instead of only isolated values. My personal advice is that you start with a reasonable wide range, to have an idea of the behavior of your algorithm for different settings, and based on this results perform a fine tune. It is important to mention that your are fitting your algorithm to the specific problem that you are evaluating. 

Dear Jairo Rojas Delgado

I am agree with Joaquín Gomis Fons. It really depends on the problem you want to solve. But, one way for finding the best parameters is trial and error. You may vary the parameters several time to see how it change the answers.

Time and time again I see badly phrased questions, as in the question above, where Delgado asks for proper parameter settings, without specifying the underlying real-world problem at hand. A lot of people spend their energy in reading and trying to answer questions, so, it is fair enough to ask for clearly phrased questions. And in case one is not yet able to do that, just study a bit more, read appropriate literature! There are most of the time only good settings, maybe, and not " best" settings! And , I repeat myself, it all depends on the nature of your problem at hand. I have written it before, but I repeat it here. It does certainly not hurt to study seriously a serious book about Operations Research, which includes modelbuilding as well!

Dear Prof. Nieuwenhuis,

My apologies if you see my question as "baddly phrased". Of course that when we talk about meta-heuristics there are not best settings because we can only get "approximate" good solutions "most of the time" due to involved stochasticity. However, I'm not asking for a "specific operation research problem" but for the underlying optimization issue here. I'm asking to the community for "recent advances" and experiences in an open research field that have been largely identified as "hyperparameter optimization". I hope this may clarify my question. I'll update the description of it. Thanks.

It all depends on what you want to search for, and what algorithm you use.

Consider such option.

You want to find the minimum value of a nonlinear multidimensional function, which exists in a confined space. This means that the solution lies inside the area of the permissible parameters or on the inner surface of this area. Suppose that the restrictions also non-linear and stationary. Assume that the optimal solution lies on the surface of one of the restrictions. Such tasks often occur at the optimal designing of structures: beams, plates, shells. To solve this problem, you must first create a task of mathematical programming: select the control variables, describe the objective function with the help of these variables and the restrictions in the form of inequalities. Then you need to choose the algorithm. Let this be a stepper algorithm, gradient or probabilistic.

Better use of probabilistic algorithms. They are adaptable, easily are trained, do not require calculation of derivatives, etc. How to set up such algorithm?

1. It is necessary to make sure that the point, out of  which you want to start, is inside of the area of allowable solutions. Substitute the coordinates of this point in the equations of restrictions. If the restrictions are not violated, the starting point is selected correct.

2. Select the step size. If the starting point is far from the restrictions (in the linear field), a step can be taken large. Near restrictions or in the area of aim  (central field) step should be small. Typically, before the solution of the task the space is normalized and is brought to a unitary hypercube. Then the step of length 0.1 can be considered large.

3. Select the direction of search. For this out of the starting point is necessary make some test steps to determine the direction of statistical antigradient and then  in this direction make a working step. If the search engine is trained in the process of solving the task, then further it will choose the right direction and change the step size.

I can you recommend a book Rastrigin L.A. "Statistical methods of search". Some of probabilistic algorithms are given in my article «Random Search as the Method of Nonlinear Programming. Algorithms of Random Search », published in journal IJETAE (International Journal of Emerging Technology and Advanced Engineering)? V6, Issue 9.