I have a question about learning filters in Convolution Neural Networks, CNNs.
Apparently, using fixed filters, like Gabor, is not common anymore and filters in CNN can be learned in each depth. it has been mentioned in some papers that Backpropagation is used for this purpose (as in the attached image).
I have two questions:
1- Is Backpropagation the only algorithm used for this learning task?
2- for learning using Backpropagation, should we consider all next layers (including pooling and FC layer)? Actually Forward and Backward phases and calculating classification error in backpropagation for learning filters is not clear for me. it would be very kind if someone describe it to me or introduce me some material to read.
Thanks
Backpropagation is one of the most used training algorithm because it produces accuracy and fast results. To learn about other alternatives a suggest you to read:
David Kriesel, "A Brief Introduction to Neural Networks", Springer, 2005.
Good Luck!
The intuition of back propagation is calculating the deviation from the truth, packpropahate that deviaton to the learning netwok to help it learn better parameters. It is used in supervised learning where the network is fed by labeled images which represent the truth. Backpropagation is analogous to adaptive filters; same concept as adaptive filters.
@Jose and @Marwa
please note that my question is not about back propagation. It is about filters in CNN
Yes, the filter weights are learned by backpropagation, in summary, you need to think about your entire architecture as a big computational graph where you are taking gradients w.r.t. to every parameter of your network (including filter weights). Take a look at the cs231n course, it will surely help you.
http://cs231n.stanford.edu/syllabus.html
Backpropagation basically adjust the Neural Networks weights by calculating error from last layer of network in back word direction.
Like when we pass data to network, in last layer we calculate error by comparing prediction with ground truth value. the error is backpropagate to second last layer to adjust weights and so on..
in CNN weights are convolution kernels, and values of kernels are adjusted in backpropagation on CNN
That happens modifiyng the weights of the links according to the answer in the CNN pattterns
Nobody has given an exhaustive answer yet. I'm pretty much sure OP knows how backpropagation works for vanilla feed-forward neural networks (the FC layer in CNNs), but rather, he fails to understand how, for each convolution depth, the values in each kernel are learned.
- Badges
- Science topic