sir entirely agree with you but mostly we used this activation function in the particular mentioned neural network. I have read in so many research paper.
May be this is also one of the reasons: The sigmoidal functions are most widely used in back propagation network because of the relationship between the value of the functions at a point and the value of the derivative at that point which reduces the computational burden during training. [as per the author S N Sivanandam & S N Deepa, "Principles of Soft Computing"]
A common choice of activation function in MLP NNs is the Sigmoid function, since it takes a real-valued input and squashes it to range between 0 and 1, i.e., it produces a curve with an “S” shape similar to TLU. Sigmoid function is differentiable often used to introduce nonlinearity in the model. A reason for its popularity is because the Sigmoid function [f=1/(1+e-x)] satisfies a property between the derivative and itself [f'=f(1-f)] such that it is computationally easy to perform.
However for RBF NNs, a RBF is a real-valued function whose value depends only on the distance from the origin, i.e., it is a function w.r.t. the origin or a certain point. Accordingly, a RBF NN often takes Gaussian function as its basis function which is symmetrical about a center maximum. Gaussians are probably the most used basis functions in RBF.
Agreed with Mahmoud Omid. Beautifully elaborated. Yes, this is to confine the possible outcome of the network between 0 and 1 (real value), while sgn (signum) is hard valued mean either 0 or 1.
I think you are talking about activation function right? If we consider BP-MLP, Back propagation (BP) is used for training multi-layers network and any activation function function you can used in hidden layers but in output layer it is suggested to keep purlin(.) only.
thank you manish sir for keeping your opinion, but to know the exact reason, read the expert opinion above.
thank you Christian Bauckhage sir for such a nice elaboration !!
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Algorithms