How can one identify whether the samples are linearly separable or not before applying a binary classifier? (or) when should I opt for linear SVM and non linear SVM?
In one of my works, I applied linear SVM and kernel SVM for the same dataset. Kernel SVM performs better in terms of accuracy. Does it mean that Dataset is not linearly separable?
Hi,
In my experience it is rather exceptional to have linearly separable data. Using the kernel trick is very fast (once you know the best set of parameters) so it's usually better to directly apply a non linear SVM.
If the non linear is more accurate, I would say that your data are indeed better separated non linearly.
Hi,
The most intuitive method to identify whether the samples are linearly separable is to plot the sample. Of course, it is impossible to visualize the sample if the dimension of the feature space is more than 3. Generally, the non-linear classifiers with optimized parameters can get better results than linear classifiers, but their model are also more complex. So, there is a trade-off between these two classifier types.
If the accuracy of non-linear classifiers is significantly better than the linear classifiers, then we can infer that the data set is not linearly separable. Otherwise, the data set is linearly separable.
Best regards.
Plot a VAT image of the data set in 2D. It will give the answer for your choice. Please check it immediately.
First you have to find out the data is an elongated or circular. For this you can take the help of the paper entitled "A novel multi-seed non-hierarchical data clustering technique", IEEE Trans. on Systems, Man and Cybernetics, Vol. 27, No. 5, pp. 871-877, 1997. On that basis if you see the data is circular then go for linear kernel. Otherwise non-linear kernel.
You can try with with both linear and not-linear classifiers then decides which one yields better reutls. For example you can use SVM.
Though SVM is a linear classifier which learns an (n – 1)-dimensional classifier for classification of data into two classes. But SVM it can be used for classifying a non-linear dataset. This can be done by projecting the dataset into a higher dimension in which it is linearly separable.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training