I would suggest you check this discussion and the answer of "doug" in the link below : https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw

From what I know, when it comes to data, If you have a lot of data with not many variations, You should think of adding more layers. Still you'd need to experiment as it can be that you are just adding layers without any performance gains.

Hi Kishore Balasubramanian : There are many rule-of-thumb methods for determining an acceptable number of neurons to use in the hidden layers. The number of hidden neurons should be between the size of the input layer and the size of the output layer.

These numbers are usually empirical and they are learned based on trial and error on validation sets. A common practice is to pick an existing architecture that is known to work well on a similar task and fine tune it based on your task and data.

In theory, a 2 layer network with non linearities is capable of representing any function, however, in practice deep networks have proven to be more useful for several reasons.

thanks all for the replies

Particularly if a large amount of data and computing resources are available, automatic optimization of the hyper parameters is recommended. Often it is not only the number of neurons but also other influencing variables (such as learning rates, optimizers, structure of connections, ...) that determine learning success. For example Baysian optimization is suitable for hyperparameter optimization.

Actually there is no particular answer for your question layers may vary based on the parameters you considered. No of layers are based on your input size, stride size, max-pooling and padding type. For the last layer you should get atleast 3*3 sized dimension for better result.

This is hyper-parameter, but there is one direction to use optimization algorithm like PSO to do this job (increase time complexity) or by trial and error. Also you can use AutoML

thank you