I have a binary feature that i want to use it with textual features i.e. unigrams. I use logistic regression and TF/IDF for representing text. So i simply add a unique feature, say ss or oo, to text of each instances. But in practice, i see adding more number of these features to instances, say two oo or ss or more get me a better results. What is the reason? How these weights improve the classification results? Should not logistic regression can get more weights for this features instead of weighting them by hand?
Hi,
Seeing the data would be much helpful for us to provide an accurate answer. Anyway, I believe the reason behind your result is that the attribute is related to the existing attributes (including the target class). In addition, adding such attribute may also affect on your data size and classification algorithm that you used.
Note that, we can't apply your method to other datasets. More important is using relevant attributes in order to form a meaningful conclusion.
HTH.
Samer
Dear Samer,
We know that for example using weight 6 for this feature can improve the results. My exact question is when we train with the normal method i.e. using one feature, or weight one, why logistic regression can't make appropriate weight to get this result?
Dear Abdalsamad,
On general, your result makes sense because the data will contain two perfectly discriminating attributes. Thus, you could decrease the Logistic's ridge value to allow it giving a higher weight to these perfect discriminators.
Cheers,
Samer
Logistic regression measures goodness of fit (the training data) using the log-likelihood of the model. The ridge estimator is derived as a restricted maximum likelihood estimator.
Cheers,
Samer
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining