You have not indicated what you want to measure. Is it rms value? That is measured by squaring the wave, averaging it and then finding the square root. Modulation index and Vdc are very different. Vdc might mean effective Vdc and is likely to be the average of the waveform. Ideally it should be zero for an ac. Vdc is sometimes taken as average of ac after full wave rectification, though it should be specified clearly as such. Check what you need. Modulation index is a measure of sinusoid content in the final waveform and is pi/4 even with square wave.. More on this can be found in the second reference given below.
The rms value of the output voltage Vxo,rms of each phase (x=a,b,c) of a two-level, three-phase inverter measured with respect to the middle point "o" of the dc bus is the same for any PWM waveform (it is independent of the PWM technique) and it is equal to Vxo,rms=(Vdc/2)^2.
If the inverter phase voltage Vxo does not contain a dc component (it has a mean value equal to zero) then the rms value of the output voltage Vx-,rms of each phase (x=a,b,c) measured with respect to the negative "-" bar of the dc bus is the same for any PWM waveform (it is independent of the PWM technique) and it is equal to Vx-,rms=(Vdc^2)/2.
On the contrary, if the inverter phase voltage Vxo contains a dc component Vxo,ave then Vx-,rms=Vdc(Vxo,ave+Vdc/2) independently of the PWM technique.
I've understood "...RMS value of output voltage..." as the total RMS including harmonics. If you refer to the fundamental component only, that I've written is not correct.