Randomized learning of feedforward NNs was proposed as an alternative to gradient-based learning which is known to be time-consuming, sensitive to the initial parameter values and unable to cope with local minima of the loss function. In randomized learning, the parameters of the hidden nodes are selected randomly and stay fixed. Only the output weights are learned. This makes an optimization problem convex and allows us to solve it without tedious gradient descent backpropagation, using a standard linear least-squares method. This leads to very fast training. The main problem in randomized learning is how to select the random parameters to ensure the high performance of NN.
Your opinions, experiences, and solutions?
Dear Grzegorz Dudek I agree with you that it is a new trend for NNs. But in my opinion, the better solution will be to use the combination of Randomized learning of feedforward NNs and gradient-based learning. I don't meet such combination now.
Thank you Vadym Slyusar for your opinion. I think that we can initialize weights and biases better for gradient-based learning using achievements from the randomization-based learning field.
- Badges
- Science topic
- Similar topics
- Social Psychology
- Attitude