I am trying to implement a 'spatial probit' model using K-nearest neighbours matrix. I come across that the rule-of-thumb is sqrt of sampled size (N) (my sample size is 383).
However, I appreciate, if someone can provide the proper method to get the optimum value of K.
Hello Dayakar,
if I understand correctly you need to find the K which gives you the best K-Nearest model.
If so you I would do the following:
1) Reserve part of your data (say for instance 25%) for testing the accuracy of your model
2) Choose a random value for K (say 1)
3) Use the remaining part of the data (75%) to develop your model using the current value of K
4) Calculate the accuracy of your model against the testing dataset
6) Increase the value of K and repeat points 3), 4) and 6) for N values of K
At this point you just choose the K which gives you the best result in terms of accuracy
Hope this will help you
Regards
K value indicates the count of the nearest neighbors. We have to compute distances between test points and trained labels points. Updating distance metrics with every iteration is computationally expensive, and that’s why KNN is a lazy learning algorithm.
There are no pre-defined statistical methods to find the most favorable value of K. Initially, a random K value is initialized and the computation process is started. Generally, choosing a small value of K leads to unstable decision boundaries. The substantial K value is better for classification as it leads to smoothening the decision boundaries. Further, derive a plot between error rate and K denoting values in a defined range. Then choose the K value as having a minimum error rate.
- Badges
- Science topic
- Similar topics
- Mathematics
- Algebra
- Matrix