Some feature of my classification task can get only limited amount of values, e.g. only values 0 and 1. What is your opinion and experience: is it useful to fix values of this feature and to solve classification task without this feature a several times for different values of this feature (e.g., one time for value = 0 and second time for value = 1). Thanks a lot for your answer beforehand. Regards, Sergey.
Actually I never see anyone do this before, it really depends on your motivation.
Mostly, people might transform the categorical features into a bunch of binary features. Say, there's a feature, Color, of three values: white, yellow, black, and then you can transform the feature into three binary features: ifwhite(1/0), ifyellow(1/0), ifblack(1/0). For any categorical feature that owns merely 2 values, don't bother to use them with other numerical features.
What's more, some machine learning methods work really well on categorical features like random forest.
I would agree with Tingyan Wei....it seems to me this feature would probably not bring much to the classification (especially, if you can run your algo without it or fixing it to a value). In my experience, I have not (I don't think) ever had a binary feature in-and-of-itself....
Dear Tingyang Wei and Jorge E. Rivé , thanks for your answers. In my opinion, conclusion isn't such evident. First, on my task there are simultaneously both numerical features and binary. Consider, for example, following simplest classification problem. x is binary feature with values 0 and 1. y and z are numerical features with values 0...1. F =x^2+y^2+z^2+rand. Label = -1 if F< S else Label =+1. For this task it is usefull to separate it for two tasks - for x=0 and for x=1.
If I understand them correctly, then I disagree with both Tingyang Wei and Jorge E. Rivé . I believe that throwing away a feature just b/c it is boolean is equivalent to throwing away potentially useful information. For example, Bernoulli process characterizes many many phenomena in the world, you can model them using statistics and make accurate predictions based on those models. Another example, making predictions from survey data. Application example, when diagnosing patients with diseases, some diseases don't afflict males, and others don't afflict females.
To answer your question, Sergey Porotsky. I think what you're describing is sort of a test of how sensitive is your model to changes in that particular variable. I think it could be useful in some cases, for example, if you want to add some weights to your features indicating how important they are to the final classification decision. I guess a follow-up question would be, why did you want to do it in the first place, what is your motivation?
About my motivation for this question:
1. To improve Sensitivity, Specifity and other criteria
2. To use this approach to support solving of classification tasks with missing data
To be specific, I think this approach might help in two scenarios.
One is like what Tom Arjannikov said about measuring the sensitivity of some feature to conduct some statistics analysis, but I think some really simple measure can be used to achieve this including correlation coefficient or mutual information in information theory. The other situation is that you can really understand the domain knowledge of the problem you are trying to solve, which resembles the scenario of F=x^2+y^2+z^2+rand Sergey Porotsky describe.
However, if you are totally blind to your data background, or the data is just too huge in dimension or sample-size, this trick would make no difference and you are merely manually selecing features, which is not recommended.
By the way, feel free to have a try if the data isn't that huge, and that might enrich your intuitive of the data maybe.
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining