Hi,

I am making an implementation of PSO for deep neural networks training. For that, I would like the PSO to find the best activation functions as well. However, unlike weights, I use a set of activation functions, which means there is only k possible values that PSO should choose from. Right now, I have tested rounding and restricting the value returned by PSO between 0 and k, and I also tried to make the value circular (i.e. using modulo k on the value returned).

These two methods don't seem right: the first one for it is extremely unfair (the first and last activation functions are almsot always selected), and the second one for the direction towards a particle makes no sense.

That made me think: how does one use PSO to find the best values within a set among others continous variables? I do have some ideas (e.g. merging 2 PSOs, one for continous data and one for enumerators, mapping the values between -infinity (or a big number) and +infinity), but I would like to hear what experts would have to say regarding that matter.

Thanks for taking the time to read through and potentially answering my question!