Once a professor mentioned us that certain algorithms for dimensionality reduction and visualization work for certain types of data. Many of us know that, for example, PCA, ICA, t-SNE, autoencoders etc work well in continuous data type. But what if you have a combination of categorical, categorical-ordinal data, binary and continuos data... what are the correct steps and algorithm to manage such combination of information or by separate (categorical data, binary, categorical-ordinal data)?
Do the previous algorithms work on well on these type of mix data types? The one-hot encoding is enough to threat the categorical data and then use the respective transformations? How we treat categorical-ordinal data... do we apply one-hot encoding and then transformation techniques or make the assumption as it is a continuous data?
Dear Manuel Martinez,
Read through the following to get the best reduction techniques that could work with your datasets.
www.toknowpress.net/ISBN/978-961-6914-09-3/papers/ML14-521.pdf
https://www.researchgate.net/post/Is_there_any_mixed_qualitativ_and_quantitativ_dimension_reduction_method
https://www.quora.com/Can-one-use-dimension-reduction-algorithms-like-PCA-for-categorical-variables
https://www.researchgate.net/post/Is_there_any_mixed_qualitativ_and_quantitativ_dimension_reduction_method
Best!
Maybe google just for simple 'feature selection'.
It is implemented in Matlab: https://www.mathworks.com/help/stats/feature-selection.html
AFAIK it deals with any data for which You can provide an evaluation function.
Dear Manuel Martinez, from my perspective, most useful methods such as PCA, ICA, autoencoders etc relies upon a fundamental notion of distance between patterns. Note that these methods may even yield discrepant results depending on the chosen distance (Euclidean, Mahalanobis, Manhattan, etc.), which corroborates the idea that distance is a more fundamental concept than the pattern representation itself.
If you agree with this point, then you will also see that the most important aspect of your question would not be if patterns are represented as numerical or categorical, but if you can define meaningful distances between them (not necessarily the distances imposed by a given numerical representation, for it can be meaningless, sometimes).
Accordingly, what I suggest is two-fold:
a) Define a clear and meaningful measures of distance between patterns, no matter if it is categorical, numerical or even heterogeneous. Clearly, the better your distance measure, the more it should reflect the phenomenon behind collected data.
b) Use Multidimensional Scaling to find a suitable metric space for all your data (i.e. a single and meaningful metric space were all kinds of patterns can be translated into a real-valued vector).
After these 'pattern translation' steps, all of-the-shelf methods can be easily used.
Clearly, step (a) is the most important and difficult one. But it may be surprisingly simple! For a practical and successful example of step (a), please have a look on the word2vec strategy, where the concept of 'context' is the key to the conception of a kind of meaningful distance between words (instead of comparing words themselves).
Thanks all for your answers!
Jugurta Montalvao, your answer outstand among other. It really helped me to picture it in a different way.
- Badges
- Science topic