I was wondering if it is possible to replace the loss function of a model with a layer having as activation function the loss function. If yes, how will this impact the network? How will the gradient descent work? How can i obtain the prediction if i do this?
without any loss function, what are you going to optimize in your ANN? ANN act as a mathematical function, then we calculate prediction from this model and at last, we calculate the difference between predictions and the ground truth. Are you discussing a completely different story compared to this?
Hi Denisa
A neural network without an activation function is essentially just a linear regression model. The activation function does the non-linear transformation to the input making it capable to learn and perform more complex tasks.
Another goal of the activation function is to keep your output values in the range [-1,1] with a zero mean (for instance).
So you need to normalize your output value so that the loss function will be able to measure the inconsistency between predicted value (^y ) and actual label (y ).
If your loss function gets the "raw" output of your last layer, you do not know what will be the variation of the values y and thus will not be able to fit the appropriate ^y. Your network will not be able to learn.
If I correctly understood what you want to do.
- Badges
- Science topic