On Supervised Learning Algorithm tuning usually the aim is to optimize some one-point criteria (recall, precision, F1, F2, etc.). But sometimes, when we build a classifier, exists a "physical" requirement for the label to change monotonically depending of a particular parameter changes. Example - if the classifier of the presence-absence of diabetes, depending on the values of some input parameters, gave the output value "yes" at a sugar level of 120, then it is clear that it should give the same output value at a sugar level of more than 120. And vice versa, if it gave the output value "no", it should give the same output value with a sugar level less than 120 (at constant values of the other input parameters). What is your opinion, should we support this requirement on the classifier or not? Thanks you in advance.
Any reliable domain knowledge is useful in machine learning.
If you use a neural network for classification, a sufficient condition for the output to be a monotonously increasing function of a given input is that all weights between the input of interest and the output be positive. Therefore, the optimization of the cost function with respect to the weights during training should be performed under the constraint that these weights be positive.
Dear Maysam, thanks for your answer. But how can we explain for patient, that, for example, for sugar = 120 there is diagnostic "diabet - NO" and for for sugar = 110 there is diagnostic "diabet - YES" ?
Dear Gerard, thanks for your answer. My remark - in my opinion, it is correct only for Linear Kernels (I use SVM)
My answer was unambiguously about neural nets. I am not surprised that it does not apply to SVMs with non linear kernels. Actually, in my experience, I always found implementing domain knowledge into SVMs to be more difficult than in neural nets; feasible, but usually cumbersome and computationally demanding.
- Badges
- Science topic
- Similar topics
- Psychology