The Performance of these algorithms (i.e. convergence time) does matter. I tried some new evolutionary algorithms proposed by Xin-She for optimizing GP constants. I found Cuckoo Search and Firefly Algorithm relatively faster than GA. However the performance still an issue here since using another evolutionary optimization technique will add a burden to the evolutionary cycle of the GP itself.

http://www.mathworks.com/matlabcentral/fileexchange/authors/119376

As long as the number of parameters to optimize is low (e.g.,

Indeed, each technique I use increases drastically the number of required evaluation of each individual, so the ideal technique would be the one that optimize in the lowest number of iterations.

You may also check nonlinear least squares minimization for constant optimization in GP using Levenberg-Marquardt algorithm. The optimization algorithm is implemented in HeuristicLab framework and discussed over different benchmark problems in the paper attached. You can also download the experiments from the link attached. The algorithm shows improvement in the quality of the evolved GP models, However, authors in the paper made it clear that it affects the execution time and therefore the solution should be considered mainly when the problem cannot be easily solved with standard methods.

http://dev.heuristiclab.com/trac/hl/core/wiki/AdditionalMaterial#EffectsofConstantOptimizationbyNonlinearLeastSquaresMinimizationinSymbolicRegression

The best way to get rid of optimizing the values of constants is to get rid of constants as such. Each additional optimizer added to GP will significantly increase the computational cost as also mentioned in the other contributions.

But there is an approach called 'Structure-Based Constants' in which one derives the constants directly from the tree structure. This way one does not need to encode them explicitly. A constant is represented as a tree itself made up of nodes and is under the influence of the genetic operators.

http://link.springer.com/chapter/10.1007%2F978-3-642-40669-0_12

Thanks Christian, very interesting!

Hossam, you are right, my main concern is to deal between computation time and results improvement.

Dear Marc Segond,

I always use gradient descending search. As this step (obviously)  requires many and many calculations I apply this only when an individual seems promising. For example, when its fitness is near or best than the best-one. With this approach I realized that the added computational effort is absolutely reasonable. With this simple trick the GP that I use for function regression increases decidedly its speed! 

Hi Marco. Thanks for your answer. Actually, I use exactly the same trick, and it is true that it gives better results for an acceptable computational time. I have been looking also into optimizing usind DE or GA, at different steps of the evolution, but the increase in the fitness was not woth the computational effort...