hi all
i am currently working with TinyGP which is a simple implementation of genetic programming for symbolic regression problems.
so I am testing it with simple problems like sin(x) symbolic regression but my problem is I get some big spikes in the numbers of Avg Fitness for the entire population however the mean best fitness is always decreasing steadily with no problems.
Hi,
My best guess is that you get these spikes because sometimes symbolic expressions are produced that have a very bad fitness (i.e., a very large number, as I guess you are minimizing), and they pass the selection process.
Try to take the median instead of the avg, and the spikes should disappear.
MOREOVER:
Sometimes, you can also get spikes at test time, and that's typically caused by some operators that are not continuous (e.g., protected division DIV(a,b) that returns 1 when b=0). They work well while you are training, but do not lead to good generalization. This was more or less known IMHO, but they recently really put this black on white in a publication (Nicolau, Miguel, and Alexandros Agapitos. "Choosing function sets with better generalisation performance for symbolic regression models." GENETIC PROGRAMMING AND EVOLVABLE MACHINES (2020)).
If you see that, I advice you either use the Analytic Quotient operator (beautifully smooth but can create expressions that are not that interpretable, see Ni, Ji, Russ H. Drieberg, and Peter I. Rockett. "The use of an analytic quotient operator in genetic programming." IEEE Transactions on Evolutionary Computation 17.1 (2012): 146-152), or a less naive version of protected division (I don't have a refernece publication for that, but I used it in my latest works). I have implemented both here: https://github.com/marcovirgolin/SimpleGP/blob/master/simplegp/Nodes/SymbolicRegressionNodes.py
Ciao,
Marco
Hi,
My best guess is that you get these spikes because sometimes symbolic expressions are produced that have a very bad fitness (i.e., a very large number, as I guess you are minimizing), and they pass the selection process.
Try to take the median instead of the avg, and the spikes should disappear.
MOREOVER:
Sometimes, you can also get spikes at test time, and that's typically caused by some operators that are not continuous (e.g., protected division DIV(a,b) that returns 1 when b=0). They work well while you are training, but do not lead to good generalization. This was more or less known IMHO, but they recently really put this black on white in a publication (Nicolau, Miguel, and Alexandros Agapitos. "Choosing function sets with better generalisation performance for symbolic regression models." GENETIC PROGRAMMING AND EVOLVABLE MACHINES (2020)).
If you see that, I advice you either use the Analytic Quotient operator (beautifully smooth but can create expressions that are not that interpretable, see Ni, Ji, Russ H. Drieberg, and Peter I. Rockett. "The use of an analytic quotient operator in genetic programming." IEEE Transactions on Evolutionary Computation 17.1 (2012): 146-152), or a less naive version of protected division (I don't have a refernece publication for that, but I used it in my latest works). I have implemented both here: https://github.com/marcovirgolin/SimpleGP/blob/master/simplegp/Nodes/SymbolicRegressionNodes.py
Ciao,
Marco
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression