Can you clarify what you're trying to do? Are you trying to design a transition from coplanar stripline to an SIW for example? If so, you might find the attached paper helpful. There are also many papers in the literature that discuss many of these transitions (e.g. microstrip to SIW, CPW to SIW, etc.)

Conference Paper Substrate-integrated waveguide transitions to planar transmi...

Dear Sir

The generation of TE and TM mode from the TEM mode can easily be obtained by changing the angle of the feeding probe. In other words, make an inclination angle in the feeding probe in such a way that the TEM signal propagates toward one of the ground surfaces.

Good Luck

Basically, the nature of TE & TM modes comes from the resonances in the structure.

Roughly, as the structure can afford half wavelength of the EM waves, you can expect the outcome of the generations of TE & TM modes.

Noted that, generally, people don't want the generations of higher modes due to the dispersion issues which makes your signal distorted.