Hi,

I am making an implementation of PSO for deep neural networks training. For that, I would like the PSO to find the best activation functions as well. However, unlike weights, I use a set of activation functions, which means there is only k possible values that PSO should choose from. Right now, I have tested rounding and restricting the value returned by PSO between 0 and k, and I also tried to make the value circular (i.e. using modulo k on the value returned).

These two methods don't seem right: the first one for it is extremely unfair (the first and last activation functions are almsot always selected), and the second one for the direction towards a particle makes no sense.

That made me think: how does one use PSO to find the best values within a set among others continous variables? I do have some ideas (e.g. merging 2 PSOs, one for continous data and one for enumerators, mapping the values between -infinity (or a big number) and +infinity), but I would like to hear what experts would have to say regarding that matter.

Thanks for taking the time to read through and potentially answering my question!

More Adrien Fabre's questions See All
Similar questions and discussions