Several Bag-of-Words techniques appear in the literature for representing either actions or objects, extracted from video sequences and images respectively, into sparse histograms of primitives occurrences. Which technique (clustering-encoding) under your point of view is the most appropriate one? What is the current State-of-the-Art trend?
The question: "which is the most appropriate one" is ill posed. As such, there is no real answer to that.
I'd start from the fact that the Bag of Words (BoW) approach is not encoding action or objects. What happens is that you have an unordered set of local features and want to compare them to another one possibly with a different cardinality.
So you could go for a fast Earth Mover Distance (Pele, O.; Werman, M., "Fast and robust Earth Mover's Distances," Computer Vision, 2009 IEEE 12th International Conference on , vol., no., pp.460,467, Sept. 29 2009-Oct. 2 2009) and compare them.
But probably you wan to get a fixed length representation which you can feed to a classifier. So you can go for the now ubiquitous strategy of quantizing + encoding of your local features. The so called BoW.
Is there something new? Well yes, and I'd say that there is a lot more to the traditional histogram count of the features assigned to every centroid. I can suggest the wonderful paper R. Arandjelović, A. Zisserman, "All about VLAD", IEEE Conference on Computer Vision and Pattern Recognition, 2013 (http://www.robots.ox.ac.uk/~vgg/publications/2013/arandjelovic13/arandjelovic13.pdf) which summarizes the Vector of Locally Aggregated Descriptors technique which is a huge improvement on classic histogram, plus a real breakthrough approach to the often neglected problem of overfitted codeword generation (my terminology).
Finally, don't miss to include spatial piramids in your work (S. Lazebnik, C. Schmid, and J. Ponce. "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, June 2006, vol. II, pp. 2169-2178. - and all the contents of http://www.cs.illinois.edu/homes/slazebni/publications.html should be treasured).
Bag of words is a more compact representation of the features in the image or video which you want to recognize or localize as per your application. Popular feature extraction methods like SIFT give out a lot of features , a good fraction of which are correlated or redundant , so using a clustering method on these features , where the centroids represent the dimensions , and counting the number of features at each centroid gives a value at that dimension. I feel kmeans is a good clustering method for to generate bag of words. Here is a good reference
http://csce.uark.edu/~jgauch/library/Video-Retrieval/Sivic.2003.pdf
This approach does slow down the algorithm, because of an added step in the feature extraction process and it also depends on the number of centroids(dimensions) you consider.
The question: "which is the most appropriate one" is ill posed. As such, there is no real answer to that.
I'd start from the fact that the Bag of Words (BoW) approach is not encoding action or objects. What happens is that you have an unordered set of local features and want to compare them to another one possibly with a different cardinality.
So you could go for a fast Earth Mover Distance (Pele, O.; Werman, M., "Fast and robust Earth Mover's Distances," Computer Vision, 2009 IEEE 12th International Conference on , vol., no., pp.460,467, Sept. 29 2009-Oct. 2 2009) and compare them.
But probably you wan to get a fixed length representation which you can feed to a classifier. So you can go for the now ubiquitous strategy of quantizing + encoding of your local features. The so called BoW.
Is there something new? Well yes, and I'd say that there is a lot more to the traditional histogram count of the features assigned to every centroid. I can suggest the wonderful paper R. Arandjelović, A. Zisserman, "All about VLAD", IEEE Conference on Computer Vision and Pattern Recognition, 2013 (http://www.robots.ox.ac.uk/~vgg/publications/2013/arandjelovic13/arandjelovic13.pdf) which summarizes the Vector of Locally Aggregated Descriptors technique which is a huge improvement on classic histogram, plus a real breakthrough approach to the often neglected problem of overfitted codeword generation (my terminology).
Finally, don't miss to include spatial piramids in your work (S. Lazebnik, C. Schmid, and J. Ponce. "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, June 2006, vol. II, pp. 2169-2178. - and all the contents of http://www.cs.illinois.edu/homes/slazebni/publications.html should be treasured).
- Badges
- Science topic
- Similar topics
- Engineering
- Electrical Engineering
- Signal Processing
- Image Processing