Is there any non-emperical way to select the optimal number of hidden layers and hidden nodes based on the number of samples and application, without performing a grid search and trail-and-error approaches ?
I think optimizing neural network parameters and deciding its architecture is in a sense similar to feature engineering that was a core step in the past. I mean they require time and determine how successful your model will perform.
I'd rather say at the moment such technique does not exist (if it does, I'd be very interested to know it).
In practice, people sometimes do layer-wise training and check validation loss that indicates over-fit. Then you can tune dropout and regularization parameters.
It also depends on the architecture. Max-out and convolutional neural networks have sparse connections and thus benefit from a different number of parameters. More recent highway networks can be trained really deep compared to conventional fully-connected network.
But short answer again is no. It likely does not.
Hi Ramin,
As Arseniy replied, I'm also not aware of such approach and I would like to know if there's one (or several) out there. What I usually do is to start with a simple (as possible) and then try a rough grid approach (as you mentioned) with an increasing number of hidden neurons and layers. I guess it is the price to pay in order to be able to use such a powerful tool. The ANN amazing ability to deal with really complex problems comes with that price...
Please let me know if you find another approach
BR Luis
I used a rule in this paper (https://www.researchgate.net/publication/283080717_Potential_of_artificial_neural_networks_to_predict_thermal_sensation_votes) wwhich was quite helpful. Maybe it is also helpful for your purposes!
Article Potential of artificial neural networks to predict thermal s...
Dear Researcher
since Artificial Neural Networks (ANN) is a model of Biological Neural Networks so there is no solid answer to know how many layers and neurons should be used. however many times using recurrent signals (RING) can increase the performance of the ANN. also optimization techniques such as GA, PSO , RPLNN can be adopted to increase the performance of any ANN. please check the below papers:
1. https://www.researchgate.net/publication/306048088_Optimizing_radio_frequency_identification_network_planning_through_ring_probabilistic_logic_neurons?ev=prf_pub
2. https://www.researchgate.net/publication/303402995_Modeling_Mechanical_Properties_of_FSW_Thick_Pure_Copper_Plates_and_Optimizing_It_Utilizing_Artificial_Intelligence_Techniques?ev=prf_pub
3. https://www.researchgate.net/publication/272572353_Introducing_Neural_Networks_as_a_Computational_Intelligent_Technique?ev=prf_pub
4. https://www.researchgate.net/publication/269367058_Introducing_Genetic_Algorithm_as_an_Intelligent_Optimization_Technique?ev=prf_pub
5. https://www.researchgate.net/publication/284435898_Optimal_Trajectory_Planning_for_Parallel_Robots_Considering_Time-Jerk?ev=prf_pub
6. https://www.researchgate.net/publication/224132638_Modeling_of_forced_dermal_wound_healing_using_intelligent_techniques?ev=prf_pub
https://www.researchgate.net/publication/232651566_Modeling_of_Dermal_Wound_Healing-Remodeling_Phase_by_Neural_Networks?ev=prf_pub
Regards
Aydin
Article Optimizing radio frequency identification network planning t...
Article Modeling Mechanical Properties of FSW Thick Pure Copper Plat...
Article Introducing Neural Networks as a Computational Intelligent Technique
Article Introducing Genetic Algorithm as an Intelligent Optimization Technique
Article Optimal Trajectory Planning for Parallel Robots Considering Time-Jerk
Conference Paper Modeling of forced dermal wound healing using intelligent techniques
Article Modeling of Dermal Wound Healing-Remodeling Phase by Neural Networks
Hi. Actually my recent research has suggested the number of layers to be 6, which is a biologically plausable result as the number of layers in a human brain is 6. This paper has not been published yet (in the process); however, you can find more information of the model that has achieved those result by reading the attached paper published earlier this year
Conference Paper A novel neuroscience-inspired architecture: For computer vis...
- Badges
- Science topic