I'm aware of the gradient descent and the back-propagation algorithm. What I don't get is: when is using a bias important and how do you use it?
Hey there, not sure what you mean. One way to see bias mathematically is considering it a weight with parameter always 1. Therefore, bias is just a constant learning parameter. Always keep in mind that machine learning will handle when to use the bias, you just feed the model: it is not your decision, like it is meta-parameters such as learning weights. One possible scenario is where the model may want to use the bias is when the neuron is fed with constant values. The neural networks has to be generic enough to handle both variations and constant values.
If you make it zero all the weight, the bias will create a constant-firing neuron.
- Badges
- Science topic