kriging is an exact (perfect) interpolator (radial basis function interpolation looks different but in fact is equivalent to kriging), See the gstat package in R.
Polynomials will only fit on "average", i.e. the interpolated values at data locations will not by the same as the data. Least squares is based on assuming a model for the interpolating function and again it will not fit the data at the data locations
Interesting. Not exactly an area I concentrated on, but I wondered if LOESS might be easier to make less vulnerable to data quality problems. So I checked and it seems so. I think you can get into this using SAS, and attached is a link i found comparing methods and using R.