this might help you

https://pdfs.semanticscholar.org/254f/3f0fce2905675445d48ca8ab61e3761b1e9b.pdf

Thank you.

Anoterh question, how to dermine number of nodes in each hidden layer ?

Dear Zouhair Chiba ,

There is no specific way to determine exact number of hidden layer , it may vary problem to problem so you can go for hit and try method for your problem.

A good measure is overfitting. The larger the number of layers and nodes, the more the overfitting. So, just increase the number of layers and nodes until overfitting appears. The point just before overfitting is the right size.

You can detect overfitting by comparing training and validation loss. If training loss goes down but validation loss goes up, then there is overfitting.

To overcome overfitting problem you can go for SVM

1- To overcome overfitting (high variance ) , we can use hyper-parameter optimization tuning algrithms

2- To overcome underfitting (high biais), we can change the architecture of ANN, or add more layers or train the ANN longer

See two links :

https://medium.com/@tm2761/regularization-hyperparameter-tuning-in-a-neural-network-f77c18c36cd3

https://medium.com/@tm2761/what-is-deep-learning-15b487adaaa3

Actually, the number of layers and nodes are hyperparameters. So, by tuning them to reduce overfitting you are using overfitting as a measure to determine the size of the network.

I highly recommend the skopt library for hyperparameter optimization: https://scikit-optimize.github.io/