I have data which contains a column wit text, For example
message
1. today,i,go,there
2. i,go,there, tomorow
suppose i have my python conditions
that, if message contains "i,go,there" then its class A, if message contains"today" its Class B
etc.
How do i make such conditions to be known by the classifier. a machine learning model.
Thank you Your help will be appreciated.
Briefly, what do you need, it is to use the bag of words approach. The essence of it consits in encoding all the words by unique numbers and counting their frequencies. These frequences later could be used by a machine learning algorithm.
If you use scikit-learn, a golden standard for building machine learning models with Python, look through the link (Extracting features from text files): http://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html
To increase quality of your model, you will probably need to exclude some non-informative words, such as 'the', 'a', 'but', 'and' etc.
If you are building an unsupervised learning model, your pipeline could be the following: 1) Extracting features via the bag of words approach; 2) applying some unsupervised learning algorithm (k-means, hierarchical clustering, etc) 3) applying expert-like classification rules (Expert-like classification rules could be implemented in Python, e.g. if some word occurred in a sentence, force the sentence to put into class A; if another (specific) word occurred in a sentence, assign it class B etc.)
Hope that helps...
Thanks Dimitry,
But actually am doing supervised learning.
i know which pattern of words exist in a specific class. Now all i need if to make the classifier whether SVM or Random forest know my conditions i mentioned above.
that is ; if column message in my data contain string pattern "i go there" . the classifier should know its class A. etc
Thank you
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining