Any magnetic materials anywhere in the waveguide or in adaptors etc?

I have checked the ends of the V cable connecting the VNA to the adapter V-SMA to the waveguide, and they are magnetic (they attract a small magnet). So yes, the end of the cable is magnetic. I would say they are far away enough from the magnet, but certainly they may be affecting... if there isn't other explanation, what are their behavior in that case?

That sounds a bit unlikely. I see now that you write electromagnet. Either the field from the magnet does something to change the properties of the waveguide, but then there must be some magnetic material in the the guide structure, unless the magnetic field is AC, in wich case you introduce some currents – or the magnet itself interacts with the waveguide fields. Varying the current in the electromagnet would certainly change the magnetic properties of the electromagnet also at high frequency. Difficult to say want is going on. Perhaps someone has seen similar phenomena.

The static magnetic field probably has a small AC component, maybe of much less amplitude than the DC main field. Some remedies are to filter out the AC component or to short circuit the AC component with a capacitor across the DC power supply, maybe with an adjustable resister in series with the capacitor..

You need to use only non-magnetic components - cables, connectors, waveguides, etc. You need to place at a some more distance your VNA from the source of static magnetic field.