Can L2 norm of the gradient tensor explain the training performance?

Dear RGs, During model training, we can dump both loss, the range of the weights, etc. One indicator of interest is the L2 norm of the gradient. like its maximal/minimal value, its distribution...

10 October 2017 7,022 2 View