I want to classify PDF documents belong to various topics and field of study. As SVM (support vector machine) is a suitable approach to classify such kind of textual documents, can anybody provide technical details and the logic behind SVM working. How we can use this approach and what is its working procedure to use it for multiple classes. Suppose we have documents related to various fields such as zoology, chemistry, mathematics etc. how this approach can classify these documents for us?
Dictionary of terms can be extracted from pdf. That's just fine. If possible, use a test like Chi-sq or similar to find terms that are unique to each class . Simple ratios are also ok to find ones that differentiate between classes.
More importantly, the terms should occur with greater frequency within documents in a class and should occur in most documents with a class, rather than being skewed toward a few document in that class.
1. Find a dictionary of terms that belong to each subject.
2. Convert pdf documents so as to extract text using OCR. This takes some doing using https://sourceforge.net/.../tesseract-ocr/ , allandin ghoscript and image convertors
3. Use R /python or other textmining libraries to construct vectors/columns that denote presence/absence or frequencies of each word from all word in all categories.
Textmining will include method to mine topics/terms that can be used in pt 3 after removing punctuation, frequently ocurring stopwords and recognizing nouns/vrebs etc.
4. Use Non-supervised/supervised methods (svm) to build model from a table containing above term/frequency information for each document.
Dictionary of terms can be extracted from pdf. That's just fine. If possible, use a test like Chi-sq or similar to find terms that are unique to each class . Simple ratios are also ok to find ones that differentiate between classes.
More importantly, the terms should occur with greater frequency within documents in a class and should occur in most documents with a class, rather than being skewed toward a few document in that class.
Dear Zafar Ali, I have an article "Tutorial on SVM" that will be published very soon. I will share it with you later.
For something as complex as text document classification, I doubt a default SVM kernel would provide you the efficiency you need. You would have to customize your kernel for better results.
However, as an ensemble method, (bagging to be precise), random forests ought to give you a more robust classification efficiency compared to a default set of SVM kernels and would be much better than a single decision tree.
Coming back to the features encoding, You may look at TF.IDF product. TF is term frequency, and IDF is inverse document frequency. The words in each document that has the highest TF.IDF product are the most distinguishing words in that document. You may also use any meta data that the PDF might provide in addition to raw text such as creation date and number of pages.
There are many papers that are published regarding text categorization with SVM. But the point that I was particular about is its tendency of high bias given an unsuitable kernel. This phenomenon of bias-variance of SVM is explained in the Journal of Machine Learning [Link provided].
In short, if you don't get the kernel right, you are going to end up with a wrong model.
There is a paper published in the Journal of Computers [Link provided] that compares Random Forest for text categorization with SVM along with other methods. It mentions how RF is superior to the other methods with pleasing results.
http://web.engr.oregonstate.edu/~tgd/publications/jmlr-valentini-bv.pdf
https://pdfs.semanticscholar.org/9b2f/84d85e5b6979bf375a2d4b15f7526597fc70.pdf
- Badges
- Science topic