A surface wave (whether mechanical or electromagnetic) is one that travels along a surface or an interface between two mediums that have different densities (for mechanical wave) or refraction indexes (for electromagnetic wave) and keeps being trapped between the two mediums. In other words, keeps propagating along the surface unless a discontinuity appears.
Examples:
1) In the case of mechanical waves, they are they waves you see generating on the surface of the water when some energy source (animal) swims above the water surface.
2) In the case of electromagnetic waves, they travel along a real (materials with relative permittivities > 1) or artificial substrates (using a series or metal disks or rods as in Yagi-Uda antennas) without getting radiated, unless a discontinuity appears where reflection and diffraction happens at the edges, giving rise to radiation. Sometimes, the discontinuity is engineered such that the patterns are shaped accordingly.
Since you are inquiring about microstrip antennas, then it is the wave that generates and travels inside the substrates of microstrip antennas and stays confined within the substrate (gets reflected at the air-substrate interface as well as the metal layer). It is a source of problems in array systems where the elements are closely spaced, such that the surface waves generated in one of the elements can continue propagating to the other element, producing undesired coupling, affecting the performance of the array. For a single antenna element, the surface-wave effect is insignificant. Surface waves will generate as a result of the substrate's excited TE and TM modes that resonate at relatively very high frequencies such that the effect on the antenna performance is insignificant, especially for those designed to operate at significantly lower frequencies, which is most probably the case.
The surface-wave phenomenon is extremely useful in designing high performance antennas with end-fire radiation patterns and wide bandwidths.
You may refer to a good reference that explains this with equations to calculate the resonances that give rise to surface waves in relation to the substrate parameters used and so on. Check section 1.51 in "Microstrip
Antenna Design Handbook" by Ramesh Garg, Artech House.
In fact, we can deal with the surface wave problem from energy distribution perspective. As we all know, both the energy and displacement of surface waves are concentrated in a very thin layer near the free surface, and decay rapidly from a distance far away the free surface. In the attached paper, a novel method based on energy distribution was proposed to identify surface waves. The energy distribution method has a clear physical meaning, and can capture the surface modes accurately.