I do several projects involving supervised learning on text and am always interested in finding useful features. For example, some of the ones that are helpful:
- the words themselves: unigrams, bigrams, trigrams
- % of unique words
- readability indices
- vocabulary richness indices
Would anyone be willing to share any text features that they have found useful for supervised learning on text?
@Graham: Thank you, Graham. I can certainly see the value in each of those.
@Manpreet: Are you referring to sentiment analysis?
What kind of problem have you tried to solve? For example if you are working with thematic classification, bag of words seem to be very helpful, but if you are dealing with sentimental classification, for instance, looking at positive/negative/neutral words can help.
Some features that I have used are n-grams of words and characters, part of speech (n-grams too). Also I used parsers to get the parse trees of each text and then find the rules of these trees to generate a bag of rules (analogous to bag of words).
@Gabriela: I deal with several types of classification and regression problems on text. I really like this idea of the "bag of rules". I have tried using parsers in the past, but abandoned them due to the time it takes to parse longer sentences. I may need to revisit this though.
- Badges
- Science topic
- Similar topics
- Computer Science
- Computer Communications (Networks)
- Internet