Think of a scenario where there's a dataset with 15 attributes(i.e. columns heading). I want to apply the clustering algorithm on that dataset but not taking all 15 attributes but taking any of the combinations of those 15 attributes(that's what I mean by dynamic attributes clustering). How can I build any model which could be able to determine the optimal number of clusters and do clustering for any combination of the attributes provided?
Please look into non-supervised approaches such as PCA-DA (Discriminant analysis using Principal Components), SVM, or PLS-DA which will select the most relevant variables (among your 15 attributes) to perform clustering. The model complexity (number of components/or Latent Structure, in the case of PLS) can be determined, then clusters will appear or not depending on the data structure.
If you want to determine a specific number of cluster then, try to apply K-means clustering. There are many approaches for clustering but usually these are enough.
Arno Germond thanks for the answer........actually I don't want dimensionality reduction here....the problem case is that I want to build a model that can be able to apply clustering for any subsets of the given attributes.....actually the user will pick his preferred attributes and I want to apply clustering for all the possible preferences a user can have..... I hope I'm clear now
Parva Jain
Please find the answers below.
1- Your question:
How can I build any model which could be able to determine the optimal number of clusters and do clustering for any combination of the attributes provided?
Answer
Technically, you need to build a model that works on the selected feature subset. To do so, a search method like "BestFirst" must be applied to find the feature subset. Then, you need to apply an evaluator to assess the merit of the selected subset. Finally, invoke cluster algorithm like EM (expectation maximisation) to find the number fo cluster (Note that finding the number of clusters is not a trivial task by default, but EM does this job using internal cross-validation.)
2- Your question:
actually the user will pick his preferred attributes and I want to apply clustering for all the possible preferences a user can have.
Answer
If you have a user who selects specific features, then no need to have a search method, just apply EM directly on the selected features.
HTH.
Dr. Samer Sarsam
Samer Sarsam your answer was really very helpful
can you throw some more light on your second point plz?
You're welcome, Parva Jain
For the second point, the user (human-manual selection of the features) needs to list the required features of the data. Then, a clustering algorithm will cluster the instances/records. In this sense, a typical clustering process must be conducted using the EM algorithm.
Cheers,
Dr. Samer Sarsam
Samer Sarsam sorry for disturbing you again
I'm not able to apply the EM algorithm on my dataset because of a lack of knowledge about it. So are comfortable with me reaching you to explain the problem I'm facing in my project and I will be more than thankful for any guidance from you.
Once again thanks to you for helping this much
Parva Jain
Sure, you can cantact me at any time.
Cheers,
Dr. Samer Sarsam
How about applying a naive bayesian clustering algorithm which can find all the possibilities.
For dynamic clustering I suggest you to follow https://hal.archives-ouvertes.fr/hal-02004417/file/DSAA16.pdf
Article A dynamic K-means clustering for data mining
https://tel.archives-ouvertes.fr/tel-01402052/document
- Badges
- Science topic
- Similar topics
- Engineering
- Computer Engineering
- Computer