How many are hidden layers and hidden nodes required in the Deep Learning?

Hi Ashri Shabrina Afrah

Based on references [1][2], you can choose the Number of Hidden Layers as follows:

If the data is linearly separable, then you may not need any hidden layers

If the data is less complex + low dimensions/features -> 1 to 2 hidden layers

If data has large dimensions/features, -> 3 to 5 hidden layers

Meanwhile, there are many rule-of-thumb methods for determining the Number of Nodes in Hidden Layers. According to Jeff Heaton[3], the rule of thumbs are:

The number of hidden neurons should be between the size of the input layer and the size of the output layer

The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer

The number of hidden neurons should be less than twice the size of the input layer

Having said so, different tasks/datasets required different design. You can follow the above guidelines, and then fine tune based on your experiments. Meanwhile, you can analyze the complexity of your multi-layer perceptron as shown in [4] (pg 5).

Good luck trying :)

References:

[1] https://www.linkedin.com/pulse/choosing-number-hidden-layers-neurons-neural-networks-sachdev/

[2] https://stats.stackexchange.com/a/1097

[3] Introduction to neural networks with Java

[4] Conference Paper Deep Reinforcement Learning for Secrecy Energy- Efficient UA...

Muhammad Azam

Determining the optimal number of hidden layers and hidden nodes in a deep learning network is a challenging task and often requires a combination of empirical experimentation and domain knowledge. There is no one-size-fits-all answer as the optimal architecture can vary depending on the specific problem, dataset, and resources available.

However, there are some general guidelines and practices that can help you in designing your neural network architecture:

Start with a simple architecture: Begin with a small number of hidden layers and nodes. This allows you to establish a baseline performance and avoid overfitting the data.

Increase complexity gradually: If the initial network is not performing well, you can gradually increase the number of hidden layers or nodes to capture more complex patterns. Monitor the performance on a validation set to avoid overfitting.

Consider the complexity of the problem: Complex problems may require deeper architectures with more hidden layers to learn intricate representations. Simple problems, on the other hand, may not benefit from deep architectures and may be effectively solved with shallow networks.

Use architectures that have shown success: There are some popular and widely used architectures in deep learning, such as Convolutional Neural Networks (CNNs) for image processing and Recurrent Neural Networks (RNNs) for sequence data. These architectures have been proven effective in specific domains and can serve as good starting points.

Use model selection techniques: Techniques like cross-validation or grid search can help you systematically explore different network architectures by evaluating their performance on validation sets. This allows you to compare and select the best-performing architecture.

Leverage existing literature and research: Stay up-to-date with the latest research in your specific domain or problem area. Research papers, articles, and online forums can provide valuable insights and recommendations for network architectures that have achieved good performance on similar tasks.

Remember that designing an effective deep learning architecture is an iterative process. It requires experimentation, analysis, and fine-tuning based on the specific problem you are trying to solve. The choice of architecture is a balance between model complexity, computational resources, and the amount of available data.

For further reading and more in-depth information, I recommend the following resources:

Deep Learning book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville: http://www.deeplearningbook.org/

Neural Networks and Deep Learning course by deeplearning.ai on Coursera: https://www.coursera.org/learn/neural-networks-deep-learning

Practical Deep Learning for Coders by Jeremy Howard and Sylvain Gugger: https://course.fast.ai/

These resources cover various aspects of deep learning, including network architecture design, and can provide you with a solid foundation to explore and experiment with different architectures.

What is the latest trends in SDN researches? what are the most important fields to research in?

What is the original color of bromothymol blue (BTB)?

Verifying pure bacterial isolate?

What is the most Operating System (OS) which can be INSTALL on Tiny hard devices like Raspberry Pi (RPi) hardware ?

Why should we choose deep learning to machine learning? Why is CNN better for image classification? Can I get Typical Answer?

What are some of the research gaps in machine learning and artificial intelligence in africa?

Anyone plz suggest relevant topic for research in image processing+Deep learning.?

How to increase a model accuracy ?

Is there a powerful system for the security of the systems distributed on IoT systems?

Can you share me how teacher education is managed in your respective national educational system?

Can NFL theorem be valid in infinite search space in ML?

How to choose ANN number of hidden layers and nodes?

What is the difference between L1 and L2 regularization in machine learning?