Currently, I am training a MaxEnt model (LibLinear Logistic Regression Trainer) to identify named entities. I figured out that when I change the order of the features inside the code the results changed. I do not know whether reordering the features could affect the results while training and testing a MaxEnt classifier or not. Maybe there is a problem in the code itself. I would like to know if that's possible?
For example, if I have those set of features in this order:
1-features.append(CombinedFeature(CapWord, PrevWord))
2-fearures.append(AllCaps)
3-features.append(POS)
Then I re-order them as follows:
1-fearures.append(AllCaps)
2-features.append(POS)
3-features.append(CombinedFeature(CapWord, PrevWord))
The f-score decreased! Any suggestions or advice?
This is something that we observed heavily with the MALLET toolkit.
There is no theoretical reason why this should happen - the order of features should not influence classification, i.e. f-score. Since it apparently does, it is either a bug in the implementation of the classifier, or a machine precision problem (float vs. double) that surfaces when multiplying many small values.
The former can be addressed by using a different implementation that does not suffer from this problem, the latter could be addressed by increasing machine precision, but never solved since machines have limited precision.
- Badges
- Science topic