I have some files containing a Persian sentence, a tab and then an English word in each line. The English words show the sentence class. some files have 2 classes, some 3 and some more. I extracted 1000 words from the file and made a term document matrix. The columns of the matrix are the classes and the rows are the words. Now I want to change this matrix to SVD which returns u, sigma and V (Vt) and then do dimension reduction. 1) How can I do that? (I've enclosed the code (python3) but I'm not sure if it's right or no. I copied from the net)
2) when I print the term document matrix, it only returns the start and last lines of the matrix (because it's too large). How can I print all of the matrix?
Then I have to find each word's vector according to u*sigma. 3) How should I make such vector (actually a matrix which is the indexes of each row of u*sigma matrix)?
hint: this a part of LSA project.
Hello Vahideh,
If your original matrix is M*N and you want to transform it to M*n, then you can use the following piece of code.
from sklearn.decomposition import TruncatedSVD
svd = TruncatedSVD(n_components=n, n_iter=7, random_state=42)
svd.fit(term_document_matrix)
svd.transform(term_document_matrix)
- Badges
- Science topic