Batch Normalization, Weights and layers normalization, scale normalization are some factors you may consider for improving the performance of A deep learning network.

Have a look at https://medium.com/techspace-usict/normalization-techniques-in-deep-neural-networks-9121bf100d8

https://mlexplained.com/2018/11/30/an-overview-of-normalization-methods-in-deep-learning/

@Imran Jamal

@Muhammad Ali

Thank you very much