Dear Nathalie,

Is your text handwriting or machine typing?

Hello Natalie,

Normally you do not use one corpus per class but the whole corpus. The class labels should be evenly distributed to get good results. I used stratified cross validation and logistic regression for text classification. It worked better for me from other learners such as naive Bayes or multilayer perceptron. I used tokenization as input. Depending on what you are trying to achieve, opinion mining, topic classification ... you might need pre-processing steps such as data deduplication, outlier removal etc.

Hope this helps

Peter

Term document matrix is based on frequent terms and you can build it for a given corpus using R

There are some things you should do before getting to the term document matrix.

With 9 classes it is important to know:

1) how frequent each class is in the training set. If any of the classes are extremely rare, it will be difficult to get good results for those classes unless you are fortunate.

2) how much accuracy you need for each class. 

3) the cost of mistakes. It could be that some mistakes are really bad, and others don't matter so much. This is true in the real world. The cost of a false negative in a medical diagnosis is often very different from the cost of a false positive, for example.

If this is just an academic exercise, it may be OK to simply say that all mistakes have the same cost, and in that just assume that the cost is 1.

First of all, put aside a proportion of your dataset for future use in testing your final classifier. I suggest 20%, which will be close to 20k examples. You want to do this carefully, making sure that the proportions of the classes in the 20% test set are as close as possible to the proportions in the training set. If you use Python and sklearn, there is an API call which makes this easy. In the R machine learning libraries you want to use, there is probably something similar. The process of separating out the training set from the test set in this way is called stratified sampling. Look that up in the documentation.

Now, working with the training set, move on to your machine learning task.

You can build TDF(Term document frequency) model for given corpus by using Python.

Hello everyones,

Many thanks for all your advised answers;

Dear Hossein Izadi : all the texts are machine typing.

Dear Peter Wlodarczak : I'm trying to associate a sentence to a group, based on its content (the sentences look like 'technical sentences' and are not structured as noun + verb + complement). This is actually not based on any opinion but on some words association instead. But the more I think, about it, the more I think that I'll gain in efficiency if I was able to perform words association (for the moment, I built the DTM tokenizing by word, and weighting thanks to term frequency-inverse document frequency algorithm, but I guess there will be more sense to associate the less common words in each sentence).

Dear Prafulla Bafna : Yes, this is finally what I did.

Dear Chris Brew  : Yes, you are indeed totally right. I've already encountered some issues in some other machine learning applications, due to unbalanced classes. Do you think it could be a solution to artificially balance the classes multiplying randomly some observationsof, let's say, the weakest classes, to get the same numbers of observations than inside the other ones ? What's your opinion regarding unbalanced classes issues ?

Also, as this is not an academic exercise, I will have to take into account the errors cost. Is there a correct strategy to put this kind of weightning in place ?

Dear Uma Dulhare : I finally chose R, but I guess that the libraries available in R and Python are quite similar.

Thanks again to all for your advices !

Best regards,

Nathalie

Dear Nathalie,

about the classification error costs, you may use the Bayesian Decision Theory approach which gives you a clean decision rule that optimises the classification in terms of the costs that you have.

The second chapter of this classic book "Duda RO, Hart PE, Stork DG (2000) Pattern Classification, 2nd edn. Wiley-Interscience" is a good starting point.

If you want to go a bit deeper into the problem of cost sensitive learning, I'd suggest:

C. Elkan, The foundations of cost-sensitive learning. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence - Volume 2, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, IJCAI’01, pp 973–978,

If you're even more interested into this type of problems and you want to do some visual data mining, I can suggest a paper I wrote recently:

Di Nunzio, G.M.

Interactive text categorisation: The geometry of likelihood spaces (2017) Studies in Computational Intelligence, 668, pp. 13-34.

http://dl.acm.org/citation.cfm?id=1642194.1642224

https://link.springer.com/chapter/10.1007%2F978-3-319-46135-9_2

No. There isn't a generally applicable good answer to unbalanced costs. The Python scikit-learn library lets you play with a parameter called class_weight for several of its classifiers.  This is similar but not identical to duplicating some of the examples in the way that you suggest.

However, the effect on the classification is hard to predict, because all classifiers, by their nature, embody assumptions about the input data (in machine learning terms, they are biased) and the results depend both on the nature of the bias, on the quality of the input data, on the details of the implementation and (it feels like) perhaps even the weather and the phase of the moon. The best thing to do is to try stuff, look carefully at the results, think, and crucially, be very systematic and careful in using cross-validation correctly.

I recommend this now rather old paper from Google as an example of clear thinking and intelligent use of simple ideas in settings where machine learning meets reality.

https://research.google.com/pubs/pub37195.html

Please let me know if this reference/site is helpful to you:

Short Text Classification Using Very Few Words - Semantic

Scholarhttps://pdfs.semanticscholar.org/1715/cdee73241094c09b6ca0b37dd1c8cd4ddd7d.pdfCached

by A Sun - ‎Cited by 75 - ‎Related articles

ABSTRACT. We propose a simple, scalable, and non-parametric approach for short text classification. Leveraging the well studied and scalable. Information ..

Please see attached file for the freely downloadable reference:

Dennis

Dennis Mazur

Nathalie Jeanray Did you succeeded on finished the categorization? if so can explain how you did it?