Can you hepl me. How do you determine about numbers neural of a hidden layer in Neural Network.
Two general guidelines will help you. In traditional supervised learning where a training set is used with back-propagation of errors, fewer neurons don't fit as well, but produces more generalized results, while more neurons produces better training results by "memorization" of the training set, but many times fails to produce generalized results on input it hasn't seen. Also, most NNs use strictly sigmoidal activation functions with connections only to neurons in the nodes above and below their "layer".
I recommend using HyperNEAT which evolves the internal neuron structure using many activation function options. It always produces better results, but has more setup overhead, and a steeper learning curve. I recommend you read Ken Stanley's doctoral thesis on NEAT first, then read the papers on HyperNEAT next. The learning curve is worth every bit of effort you invest into it.
Good luck with your application.
http://nn.cs.utexas.edu/downloads/papers/stanley.phd04.pdf
http://eplex.cs.ucf.edu/hyperNEATpage/
That is why building a neural network is an art-form. there are many techniques but most come down to trial and error, automated by genetic algorithms or other artificial techniques.
Some have begun to wonder if there is a relationship between the branch points in the dendritic network, and the hidden layer neurons. One neuron for each branch point. In any case there is little anyone can do about the vagueness of this art-form since it is hidden in the definition of neurons.
Two general guidelines will help you. In traditional supervised learning where a training set is used with back-propagation of errors, fewer neurons don't fit as well, but produces more generalized results, while more neurons produces better training results by "memorization" of the training set, but many times fails to produce generalized results on input it hasn't seen. Also, most NNs use strictly sigmoidal activation functions with connections only to neurons in the nodes above and below their "layer".
I recommend using HyperNEAT which evolves the internal neuron structure using many activation function options. It always produces better results, but has more setup overhead, and a steeper learning curve. I recommend you read Ken Stanley's doctoral thesis on NEAT first, then read the papers on HyperNEAT next. The learning curve is worth every bit of effort you invest into it.
Good luck with your application.
http://nn.cs.utexas.edu/downloads/papers/stanley.phd04.pdf
http://eplex.cs.ucf.edu/hyperNEATpage/
Number of neurons in hidden layer in an artificial neural networks is normally determined by trail approach. However, optimum training of networks using some heuristic optimization approaches may give better results.
http://www.ijcai.org/Proceedings/89-1/Papers/122.pdf
http://homepages.inf.ed.ac.uk/pkoehn/publications/gann94.pdf
http://stats.stackexchange.com/questions/55887/backpropagation-vs-genetic-algorithm-for-neural-network-training
Mahamad, I appreciated your thoughtful links. I do believe the past twenty years have seen significant developments in both GA and ANNs in general. I believe the encoding problems have been resolved, and the complexity of the algorithm's processing has been remedied by Moore's Law in computing power.
The video linked below illustrates neural evolution in selecting the configuration of hidden layers, as well as several other topics such as modular and regular ANNs.
https://www.youtube.com/watch?v=t15wUkCXuxQ
.
I have for many years held high hopes for Fully Analog Neural Networks or FANNs. Perhaps, with our very-short-channel CMOS and its numerous metal layers, allowing very richly-crossing interconnects, we may be able to implement systems with true concurrency. Then we may be touching the eye of God, as we step into "real AI".
Barrie
Hello Everyone !
I am very glad to be received your answer. Your guidelines already help me research and applying ANN in my work better.
Thanks so much.
As all contributors to the field of neural networks already know and have explained, the state of the art approach for determining the structure of the network is still trail and error. I would suggest you to have a very careful look at the recently published book of Ian Goodfellow, Yoshua Bengio and Aaron Courville, "Deep Learning" at:
http://www.deeplearningbook.org
- Badges
- Science topic
- Similar topics
- Engineering
- Electrical Engineering