In each stage of a Constitutional Neural Network, filters are used. Now which filter? So many are there in lit. If a new kernel window acting as filter can be created, how to do it. What are the general characteristics followed for kernel design of a filter. Thank You all.
Dear Kishore,
Convolution filters can be initialized in one of the following ways.
1. Randomly assigning weights for the different filters. See some refs:
(a) Jarrett, K., Kavukcuoglu, K. and Lecun, Y., 2009, September. What is the best multi-stage architecture for object recognition?. In 2009 IEEE 12th International Conference on Computer Vision (pp. 2146-2153). IEEE.
(b) Saxe, A., Koh, P.W., Chen, Z., Bhand, M., Suresh, B. and Ng, A.Y., 2011. On random weights and unsupervised feature learning. In Proceedings of the 28th international conference on machine learning (ICML-11) (pp. 1089-1096).
2. Handcrafting the weights of the different filters to detect specific features during convolution. This approach is not all that interesting in the context of deep learning since it defeats the purpose of learning features without really engineering them!
3. Learning filter weights using unsupervised training schemes. For example, there are literatures describing the use of auto encoders, deep belief nets, K-means clustering etc. for unsupervised learning of convolution filters. See some refs:
(a) Masci, J., Meier, U., Cireşan, D. and Schmidhuber, J., 2011, June. Stacked convolutional auto-encoders for hierarchical feature extraction. In International Conference on Artificial Neural Networks (pp. 52-59). Springer Berlin Heidelberg.
(b) Lee, H., Grosse, R., Ranganath, R. and Ng, A.Y., 2009, June. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th annual international conference on machine learning (pp. 609-616). ACM.
(c) Coates, A., Lee, H. and Ng, A.Y., 2010. An analysis of single-layer networks in unsupervised feature learning. Ann Arbor, 1001(48109), p.2.
Hope this helps.
hi
this link may help you
http://www.robots.ox.ac.uk/~vgg/practicals/cnn/
http://deeplearning.net/tutorial/lenet.html
Dear Kishore,
Convolution filters can be initialized in one of the following ways.
1. Randomly assigning weights for the different filters. See some refs:
(a) Jarrett, K., Kavukcuoglu, K. and Lecun, Y., 2009, September. What is the best multi-stage architecture for object recognition?. In 2009 IEEE 12th International Conference on Computer Vision (pp. 2146-2153). IEEE.
(b) Saxe, A., Koh, P.W., Chen, Z., Bhand, M., Suresh, B. and Ng, A.Y., 2011. On random weights and unsupervised feature learning. In Proceedings of the 28th international conference on machine learning (ICML-11) (pp. 1089-1096).
2. Handcrafting the weights of the different filters to detect specific features during convolution. This approach is not all that interesting in the context of deep learning since it defeats the purpose of learning features without really engineering them!
3. Learning filter weights using unsupervised training schemes. For example, there are literatures describing the use of auto encoders, deep belief nets, K-means clustering etc. for unsupervised learning of convolution filters. See some refs:
(a) Masci, J., Meier, U., Cireşan, D. and Schmidhuber, J., 2011, June. Stacked convolutional auto-encoders for hierarchical feature extraction. In International Conference on Artificial Neural Networks (pp. 52-59). Springer Berlin Heidelberg.
(b) Lee, H., Grosse, R., Ranganath, R. and Ng, A.Y., 2009, June. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th annual international conference on machine learning (pp. 609-616). ACM.
(c) Coates, A., Lee, H. and Ng, A.Y., 2010. An analysis of single-layer networks in unsupervised feature learning. Ann Arbor, 1001(48109), p.2.
Hope this helps.
Hi P.V.V. Kishore,
Oyebade has given a very good detailed answer. Just want to add a clarification: With CNN/ConvNet the goal is to learn the filters; you don't actually design these filters (or kernels). They will be learned during training as long as the training converges.
And yes, initializing the these filter parameters with good defaults (e.g. references in Oyebade's answer) before starting the training is key to convergence especially in very deep networks.
Regards.
- Badges
- Science topic