What is the difference between them, and where we can use each type of these features?
Main difference: you won't know what learnt features will look like before you learn them.
In any predictive pipeline, you might imagine listing a number of statistical functions that you think might be informative for your (classification) problem. Maybe you think that the mean, variance, median, min, max, kurtosis of your data are useful measures for discrimination, and in truth, for simple datasets this is probably a sensible set of features to look at. Now, because you have specified these features explicitly yourself, we call these hand-crafted features. For more complicated tasks (eg image recognition) you may specify different statistical functions, but the feature representation will still be hand crafted.
Another class of features are those that can be learnt from data itself. In a sense you still have to specify something (eg the model that will discover features, and its parameters) but the actual feature learning is achieved by an iterative optimisation procedure, and so these are learnt features. There are a number of methods that can learn features from data including restricted Boltzman machines or autoencoders, and you could classify these all as falling under the umbrella of representation learning that has recently received significant interest. Other options include sparse coding and dictionary learning, which decompose the data into a linear combination of a set of bases. The most important parameter that you must set for all methods is the capacity of the representation, ie the number of bases used in the decomposition. You can cross validate over this parameter, though, which means that you can select it objectively in a data driven procedure.
I find learnt features very appealing. In particular, they offer you the opportunity to cross-validate your feature representation, and it is not immediately obvious you might achieve this with the hand-crafted feature methodology.
https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine
https://en.wikipedia.org/wiki/Autoencoder
https://en.wikipedia.org/wiki/Sparse_dictionary_learning
Main difference: you won't know what learnt features will look like before you learn them.
In any predictive pipeline, you might imagine listing a number of statistical functions that you think might be informative for your (classification) problem. Maybe you think that the mean, variance, median, min, max, kurtosis of your data are useful measures for discrimination, and in truth, for simple datasets this is probably a sensible set of features to look at. Now, because you have specified these features explicitly yourself, we call these hand-crafted features. For more complicated tasks (eg image recognition) you may specify different statistical functions, but the feature representation will still be hand crafted.
Another class of features are those that can be learnt from data itself. In a sense you still have to specify something (eg the model that will discover features, and its parameters) but the actual feature learning is achieved by an iterative optimisation procedure, and so these are learnt features. There are a number of methods that can learn features from data including restricted Boltzman machines or autoencoders, and you could classify these all as falling under the umbrella of representation learning that has recently received significant interest. Other options include sparse coding and dictionary learning, which decompose the data into a linear combination of a set of bases. The most important parameter that you must set for all methods is the capacity of the representation, ie the number of bases used in the decomposition. You can cross validate over this parameter, though, which means that you can select it objectively in a data driven procedure.
I find learnt features very appealing. In particular, they offer you the opportunity to cross-validate your feature representation, and it is not immediately obvious you might achieve this with the hand-crafted feature methodology.
https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine
https://en.wikipedia.org/wiki/Autoencoder
https://en.wikipedia.org/wiki/Sparse_dictionary_learning
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning