Support Vector Machine (SVM) uses a kernel trick to classify data. As a supervised classifier, it uses label corresponding to the testing data which is similar to a neural network. The kernel function arranges data in a higher dimension to make the data linearly separable and does not add any non-linearity to the given data. However, most kernel apply non-linear function to separate data in a higher dimension. The activation of weights in SVM is not similar to a neural network. Can we call the SVM as a linear classifier based on the non-linear kernel application?
By default SVM works as a linear classifier when it maps a linear function of the n-dimensional input data onto a feature space where class separation can occur using a (n-1) dimensional hyperplane. But when it uses a kernel, then it maps a non-linear function of the input data onto the feature space, and then it is said to work in a non-linear mode.
I agree with you Ranjan. The SVM is mapping features in a higher dimension using kernel tricks, particularly for all kernels except linear kernel. But, it is not adding any non-linearity with the data like convolution in a neural network. After mapping data non-linearly, the SVM classifies data classes linearly. In this sense, isn't it a linear classifier?
An SVM with a non-linear kernel is a non-linear classifier in the original data space. Consider the decision hyperplane in feature space; by definition, it is linear. To obtain the equivalent decision boundary in the original data space, the hyperplane must be transformed by applying the inverse of the non-linear kernel. As the resultant boundary will be a non-linear function of the original input variables, the SVM can be said to be a non-linear classifier when the kernel used in non-linear.
If this is difficult to visualise, the diagrams at https://statinfer.com/204-6-5-the-non-linear-decision-boundary/ may help.
- Badges
- Science topic