When trying to fine tune the SVM classification model by controlling the slack/cost parameter "C" or "nu", there is a corresponding effect on the number of support vectors (SVs) available for modelling. How do you decide the best number of support vectors to use and so accept a particular "C" or "nu" when you have many "Cs"/"nu" with good cross-validation accuracy?
Hi,
since you have the same accuracy for multiple values of C, I'd recommend to use the lowest possible C in order to minimize the risk of over-fitting. I guess the standard classifier tuning scripts (like the one included in LIBSVM) does that automatically for you.
As Christian said, in general there is no need to worry about the number of support vectors, you should just be careful not to over-fit. Therefore it is highly recommended that you use cross-validation to select the right parameters.
Cheers,
Zoltan
Yes, i indeed i noticed that the lower the "nu" value (in "nu" SVM) i choose, the lesser the number of support vectors which definitely will improve computation speed. However, my challenge is in deciding the best threshold so as not to risk under fitting too. In fact, if i just go by the lowest possible "nu" in my situation, my data will be underfitted even though i still have 100% cross validation accuracy during tuning. Thank you for all your contributions
- Badges
- Science topic
- Similar topics
- Engineering
- Electrical Engineering
- Signal Processing
- Image Processing