.

Have a look at the "condensed nearest neighbour" approaches :

N. Bhatia et al,

Survey of Nearest Neighbor Techniques.

International Journal of Computer Science and Information

Security, Vol. 8, No. 2, 2010.

http://arxiv.org/pdf/1007.0085

F. Anguilli,

Fast Condensed Nearest Neighbor Rule.

Proceedings of the 22nd International Conference on Machine

Learning, Bonn, Germany, 2005

http://wwwinfo.deis.unical.it/%7Eangiulli/papers/icml2005.pdf

.