For example: i want to design a filter in a specific frequency band with a desired insertion loss/return loss characteristics. Does a guideline exist to choose the suitable technology?
In my opinion, everything depends on your operation frequency. If it does not exceed a few GHZ you can use micro-strip lines. On the other hand, over 10 GHz is better to go with other technologies SIW, RWG.
If you are designing the filter you first need to determine the required Q you need. Waveguide is the lowest Q followed by coaxial cavities, dielectric resonator, then ceramic coaxial, ceramic printed, ... TFR, SAW,... printed circuit on FR4. To determine the Q you need to know the loss tangent and dielectric dissipation factors and the mode.
Simple question with a very long answer. First, what type of response, low pass, high pass, bandpass. Relative bandwidth? Frequency? Vibration requirements? EMC requirements? Temperature range?Reflectionless?
Visit filter specialist company websites like K&L or Lorch, enter specs. see what they suggest to get some feeling for application of different types.
Rect WGs have a high Q factor, and low losses, they are advantages of Rect WGs. But RWGs are very large as they can't integrated with printed circuit.
Microstrip and strip lines have a very bad Q factor, high losses,.... But they can integrated to printed circuit.
SIWs or substrate integrated waveguides are a trade of between rect WGs and mictrostrip and strip lines. SIW has an acceptable Q factor and it can be integrated with printed circuit. In fact SIW can be approximated as a RWG.