In CST if it is metamaterial like absorber, use unit cell B.C  and also open B.C along the direction of incidence. If it is terminated by mirror, use PEC B.C for underneath plane and if the substrate is semi infinite use open B.C with appropriate substrate length.

Specify the periodicities and scan angles and phase shifts from Boundaries.

You can specify number of floquet modes received  at Zmin and Zmax by right clicking on them and also change other options like polarization and etc. 

from post processing ---- optical choose Absorption - reflectance - transmission.

In the setup solver choose Zmax for excitation in source type with the appropriate mode. for example both TE00 and TM00 are plane waves but with different polarization, that is Ey and Hx in the former and Ex and Hy in the later one.

I hope it help you.