I reviewed several Machine Learning material. None of the books introduces LSI as a classification method.
It is primarily used for retrieval, but you can turn it to be a classification model. After obtaining the low-rank approximation of the training term-document matrix with SVD you can use the computed matrices to project a new (test) document in the reduced space. Then using cosine similarity you would compute the similarity between the test document and all the training documents in the reduced space. The documents that are more similar to the new document can provide you with a guess about the category (similar to a KNN classification method). Another alternative is to use the projected training documents to train a classification model (SVM, Decision trees, etc.). Another alternative is that since SVD is similar to PCA you can compute projection matrices per category and use them to classify new documents based on a reconstruction error (see paper attached).
Article Minimizer of the Reconstruction Error for multi-class docume...
Strictly speaking LSI/LSA does not carry out a learning procedure in which the algorithm adjusts parameters to "learn". Some of the books includes topics such as PCA for pre processing but is not considered a learning algorithm. That being said, there is room for LSI/LSA in books on pattern classification which goes along the lines of what Carlo's mentioned though they are covered mainly in information retrieval and NLP because of their main use is in these areas.
I think it is not considered a classification technique, even though it is an important step for information classification. It is considered a feature extraction/reduction technique which map your observations to a set of orthogonal dimensions that makes the retrieval process much more fruitful. This mapping process depends on rotating the dimensions and scaling the data in a way that magnifies the diffrence between the data which make the classification much easier.
Simply it makes you see the same data from another perspective or another angle that allows you to separate the classes.
I hope this helps
Regards,
Ahmad
- Badges
- Science topic