I've some question about clustering data set using SOM. What the pre requisite of the data?
I'v mixed data types (categorical, binary, continues).
I'transformed categorical to dummies then merged my data set into one. I did not apply min-max scaler as I've many dummies variables.
I preferred using pca. I applied pca to reduce features by feature importance. then I use these axes as input in my SOM (SOMPY).
Is this a good approach?
If my binary data has for example 95% of 1 and 5% of 0 or vice versa, is this a problem for clustering?
Your approach is good. The prequisite for the standard SOM using Euclidean distance is to have continuous, real-valued features. PCA is a good idea as it also removes correlations.
Now, other versions of SOM exist, that can natively handle categorical/binary features. See for instance:
- Binary SOM : http://cedric.cnam.fr/fichiers/RC235.pdf
- Bernoulli SOM : https://www.academia.edu/download/39469000/BeSOM__Bernoulli_on_self-organizing_map20151027-3791-1lxjog3.pdf
- Mixed SOM : https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2005-17.pdf
- Relational SOM to work directly on a distance matrix, so you can use any distance : Article On-Line Relational SOM for Dissimilarity Data
This Link of Bernoulli SOM is broken.
https://www.academia.edu/download/39469000/BeSOM__Bernoulli_on_self-organizing_map20151027-3791-1lxjog3.pdf
the link is working from scholar: https://scholar.google.fr/citations?user=d9zWc3AAAAAJ&hl=fr
- Badges
- Science topic
- Similar topics
- Instrumentation Engineering
- Instruments