Dictionary of terms can be extracted from pdf. That's just fine.  If possible, use a test like Chi-sq or similar to find terms that are unique to each class . Simple ratios are also ok to find ones that differentiate between classes.

More importantly, the terms should occur with greater frequency within documents in a class and should occur in most documents with a class, rather than being skewed toward a few document in that class.

1. Find a dictionary of terms that belong to each subject.

2. Convert pdf documents so as to extract  text using OCR. This takes some doing using https://sourceforge.net/.../tesseract-ocr/ , allandin ghoscript and image convertors

3. Use R /python or other textmining libraries to construct vectors/columns that denote presence/absence or frequencies of each word from all word in all categories.

 Textmining will include method to mine topics/terms that can be used in pt 3 after removing punctuation, frequently ocurring stopwords and recognizing nouns/vrebs etc.

4. Use Non-supervised/supervised methods (svm) to build model from a table containing above term/frequency information for each document.

Dear Rajeev Gangal,

Dictionary of terms can be generated from terms we extracted of PDF documents or we should find some other source for that? Also, can we use SVM for more than one classes?

Thanks

Dictionary of terms can be extracted from pdf. That's just fine.  If possible, use a test like Chi-sq or similar to find terms that are unique to each class . Simple ratios are also ok to find ones that differentiate between classes.

More importantly, the terms should occur with greater frequency within documents in a class and should occur in most documents with a class, rather than being skewed toward a few document in that class.

Rajeev Gangal, I appreciate your answer, first i will design a dictionary of words using PDF files. After having that dictionary i will apply appropriate machine learning approach to classify and train our system.

Dear Zafar Ali, I have an article "Tutorial on SVM" that will be published very soon. I will share it with you later. 

Thank Khan Muhammad for upvoting my answer.

Dear Loc Nguyen,

Which one among SVM, Decision tree and random forest algorithms is suitable for text classification? Which one out of these would outperform in classifying large size PDF documents?

Thanks

For something as complex as text document classification, I doubt a default SVM kernel would provide you the efficiency you need. You would have to customize your kernel for better results.

However, as an ensemble method, (bagging to be precise), random forests ought to give you a more robust classification efficiency compared to a default set of SVM kernels and would be much better than a single decision tree.

Coming back to the features encoding, You may look at TF.IDF product. TF is term frequency, and IDF is inverse document frequency. The words in each document that has the highest TF.IDF product are the most distinguishing words in that document. You may also use any meta data that the PDF might provide in addition to raw text such as creation date and number of pages.

Dear Ebenezer R.H.P. Isaac ,

according to your opinion, Random forest outperform SVM as far as classification of large size PDF documents are concerned. If yes is the case, please recommend some articles, where random forest is explained and used for such problems.

Thanks

There are many papers that are published regarding text categorization with SVM. But the point that I was particular about is its tendency of high bias given an unsuitable kernel. This phenomenon of bias-variance of SVM is explained in the Journal of Machine Learning [Link provided].

In short, if you don't get the kernel right, you are going to end up with a wrong model.

There is a paper published in the Journal of Computers [Link provided] that compares Random Forest for text categorization with SVM along with other methods. It mentions how RF is superior to the other methods with pleasing results.

http://web.engr.oregonstate.edu/~tgd/publications/jmlr-valentini-bv.pdf

https://pdfs.semanticscholar.org/9b2f/84d85e5b6979bf375a2d4b15f7526597fc70.pdf

Hi,

There is a lot of work about SVM in text classification. There are many variants of SVM for this task depending on the distribution and properties of your data and categories; but if your problem is only multi-class classification, a good starting point is with a linear SVM, which is usually good enough (Decision Trees and Random Forests are a bit outdated in text classification). However, it is a good practice to optimize the parameter C with a validation set. A good library to train fast linear SVM is LibLinear [link attached].

I have done some research in multi-class document classification, with a method based on the reconstruction error of PCA [publications attached].

Best

https://www.csie.ntu.edu.tw/~cjlin/liblinear/

Article Minimizer of the Reconstruction Error for multi-class docume...

Conference Paper Document Categorization Based on Minimum Loss of Reconstruct...