Resources say that 2 hidden layers are sufficient for most applications. But I've also heard the more the number of layers in the ANN, the more complex calculation it will be able to perform. How do you exactly know how many layers to include in your application? Is there any way to theoretically determine the optimum number of layers?
Dear all,
As far as I studied literature, there are no one "proper" algorithm for the definition of the model's complexity. Nevertheless, as previously mentioned not to over-fit model is crucial.
The approach I used in my works was to plot on x-axis number of nodes (neurons) and on y-axis I plotted the performance indicators (R2 and MSE). In the final I got very nice logarithmic approximation, were R2 is growing in the beginning rapidly with each added neuron, but then R2 reaches the saturation level, when the addition of extra neurons improves model marginally (curve flattens). I stooped with the model just on the eagle of that flat curve.
I suggest to run the model for couple times with the same structure (rhe number of neurons and layers) if you choose to generate random initial data population.
Regarding the hidden layer - I used one in my research (see link Solar and pellet...). I tried with 2 as well but did not gain much in model accuracy. My studied system was solar and pellet combined system for heat distribution and accumulation. I can regard it as relatively complex and dynamic and one hidden layer did great job for me.
Another thing that I what to point out - I found some differences with training algorithms used as well. Some works better and you should use trial-and-error method here as well (see link Application of ..... Fig.4).
Good luck with research, Lelde
Conference Paper Application of Artificial Neural Networks for Detection of D...
Article Solar and pellet combisystem for apartment buildings: Heat l...
there is a good paper explained : Yann N. Dauphin & Yoshua Bengio
Big Neural NetworksWaste Capacity. This article exposes the failure of some big neural networks to leverage added capacity to reduce underfitting.
Optimal number of the neurons in the hidden layer and number of hidden layers determine by trying different networks. It is found that, best structure of the ANN model has minimum value of performance function (MSE, MRE and MAE and etc).
In addition, you can combine ANN and optimization methods ( GA ,PSO and etc) to choose best number of hidden layers and neurons.
Good luck
I followed optimization of the hidden layers and node volume by running several experiments with range of hidden and node volumes and measure the performance indicators (Choose a couple; RMSE or MRE). This could be done easily for selected outputs. You will find the optimal levels. This protects protects your model from over-fitting. Hope this helps
Article A gronomy J our n al @BULLET Volume 10 5 , I ssue 3 @BULLET ...
Dear all,
As far as I studied literature, there are no one "proper" algorithm for the definition of the model's complexity. Nevertheless, as previously mentioned not to over-fit model is crucial.
The approach I used in my works was to plot on x-axis number of nodes (neurons) and on y-axis I plotted the performance indicators (R2 and MSE). In the final I got very nice logarithmic approximation, were R2 is growing in the beginning rapidly with each added neuron, but then R2 reaches the saturation level, when the addition of extra neurons improves model marginally (curve flattens). I stooped with the model just on the eagle of that flat curve.
I suggest to run the model for couple times with the same structure (rhe number of neurons and layers) if you choose to generate random initial data population.
Regarding the hidden layer - I used one in my research (see link Solar and pellet...). I tried with 2 as well but did not gain much in model accuracy. My studied system was solar and pellet combined system for heat distribution and accumulation. I can regard it as relatively complex and dynamic and one hidden layer did great job for me.
Another thing that I what to point out - I found some differences with training algorithms used as well. Some works better and you should use trial-and-error method here as well (see link Application of ..... Fig.4).
Good luck with research, Lelde
Conference Paper Application of Artificial Neural Networks for Detection of D...
Article Solar and pellet combisystem for apartment buildings: Heat l...
theoretically determine the optimal number of layers, yet I find that does is try and error to get the smallest MSE value that can be tried 2 layers or 3 layers when you get the smallest MSE that is the best.
Three layers which include one input layer, one output layer and one hidden layer are enough. The cells of hidden layer can be changged according to the model's complex to be calculated.
According to some authors, three layers of active units can represent any pattern
classification (Beale and Jackson 1990). So, I usually have 1 input layer, 1 output layer and 3 hidden layers.
Regards. Luiza
I usaully have 1 input layer, 1 output layer and 1 hidden layer. The cells of hidden layer can be changged according to the model's complex to be calculated. The kind of ANN structure had been used to estimate the masses and positions of loose part in NPP.
in my opinion, most typically between 2 and 3 layer, try to get the minimum value of MSE
- Badges
- Science topic
- Similar topics
- Applied Mathematics
- Calculations