We know that sigmoidal activation function is well studied in neural network approximation, to generalize it we take the softmax activation function.
Dear Hawraa Abbas Almurieb,
In mathematics, the softmax function takes an un-normalized vector, and normalizes it into a probability distribution. That is, prior to applying softmax, some vector elements could be negative, or greater than one; and might not sum to 1
https://en.wikipedia.org/wiki/Softmax_function
https://papers.nips.cc/paper/7130-svd-softmax-fast-softmax-approximation-on-large-vocabulary-neural-networks.pdf
Regards, Shafagat
Dear Hawraa Abbas Almurieb,
In mathematics, the softmax function takes an un-normalized vector, and normalizes it into a probability distribution. That is, prior to applying softmax, some vector elements could be negative, or greater than one; and might not sum to 1
https://en.wikipedia.org/wiki/Softmax_function
https://papers.nips.cc/paper/7130-svd-softmax-fast-softmax-approximation-on-large-vocabulary-neural-networks.pdf
Regards, Shafagat
Dear Hawraa Abbas Almurieb,
It may be helpful to take a look at the short review on the difference between Softmax and Sigmoidal activation functions here: https://medium.com/aidevnepal/for-sigmoid-funcion-f7a5da78fec2
Hope it could help,
Mahdieh Zabihimayvan
- Badges
- Science topic