Which MTM would be best for the gain enhancement of Slotted waveguide Antenna? I know we need negative refractive index MTM but which shape or type i.e. single layer double layer would be the best?
Dear Muhammad Umer Shahzad as a start point I suggest to your attention this conference paper: Conference Paper Metamaterials on antenna solutions
In any case, the multilayer concept will be better but such design will have more losses. The one-layer solution can work only as Frequency Selective Surface. You need to test your design in the simulator HFSS or similar.
It depends on the frequency range. In the literature same authors suggest two layers of MTM to increase the gain of a UWB antenna; one layer for lower frequencies and the second one for higher frequencies. Others considered one layer to increase the gain over a wide bandwidth as in the reference below. You need to compare the performance of MTM structures using floquet ports over the frequency of interest. I suggest the following youtube video as well which can help you to simulate your MTM structure:
https://www.youtube.com/watch?v=L9ErVgTmIwI
Conference Paper Gain enhancement and miniaturization of UWB antenna using me...