In general, increasing the number of hidden layers can enable the network to learn more complex representations of the input data, which can lead to better performance. However, adding too many layers can also lead to overfitting, where the model performs well on the training data but poorly on the test data.

You can check out our work Conference Paper Impact of PCA-based preprocessing and different CNN structur...

where we discuss the impact of changing the amount of layers and filters.

There are associations that just can't be represented by a network with a given number of hidden layers, period. Every neural network with a fixed number of hidden layers has a critical capacity. In order to represent an input/output association it's necessary to vary the number of layers appropriately. An example is the tiling algorithm of Mézard and Nadal, Article Learning in feedforward layered networks: The tiling algorithm

It suffices that the activation be a monotonic function. The case of non-monotonic functions was studied by Adler, https://arxiv.org/pdf/adap-org/9406001.pdf

The selection of the activation function and the number of hidden layers have a significant impact on how well deep neural networks execute classification tasks. The hidden layers apply weights to the inputs and direct them through an activation function as the output. Apart from this, activation functions are used to perform nonlinear transformations of the inputs entered into the output of a neuron.

As a summary, the number of hidden layers and the selection of activation function are important factors that can significantly impact the performance of deep learning classification tasks.

The impact of varying the number of hidden layers in a deep neural network on its performance for a specific classification task can be different for each task and dataset. In general, adding more hidden layers can increase the model's ability to capture complex patterns in the data and can lead to better performance. However, adding too many hidden layers can lead to overfitting, where the model memorizes the training data but does not generalize well to new data.

The choice of activation function can also have a significant impact on the performance of a deep neural network. Different activation functions have different properties that can affect the model's ability to learn and generalize. For example, the sigmoid function is often used in the output layer for binary classification tasks because it produces a probability between 0 and 1, while the ReLU function is commonly used in hidden layers because it is computationally efficient and has been shown to work well in many cases. The conclusion is that we required to tune these two factors based on the various problems such as type of dataset, model used, other hyper parameters and so on.

Dear Sami Naili

Your question is interesting. You may go through the following articles for your solution -

Article Prediction of compaction parameters of compacted soil using ...

https://www.ijraset.com/best-journal/hyperparameters-of-artificial-neural-network-for-the-best-prediction-of-geotechnical-properties-of-soil