High dimensionality is one of the challenges for text data clustering.
Techniques are listed in http://en.wikipedia.org/wiki/Dimensionality_reduction
In the text processing, two main streams are: indexing/querying and summarization.
For text data clustering, the equivalent of dimensionality reduction is "summarization". My favourite method for this is semantic compression. Try looking into wordnet and similar onotology management systems - you should get pretty good results.
For those who would like a very gentle intro to that semantic compression, here is something I put on my scratchpad may years back: http://gopalakrishna.palem.in/compression.html Not particularly well-written, being a scratch pad, but gives a good start.
I think Singular Value Decomposition is also one of the dimensional reduction technique for text data clustering. There are another advantages with singular value decomposition for text clustering is to predict number of clusters for through SVD.
http://www.indianjournals.com/ijor.aspx?target=ijor:ijaritac&volume=2&issue=2&article=004
I think rough computing can be used for reduction in text data while clustering.
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Materials Science
- Materials
- Inorganic Materials
- Metals
- Alloys
- Steel