I haven't performed this experiment exactly, but I think you would be much better with an input polarizer. I don't know the nature of your sample but it has a structure it  may be that having the input light polarization would could be helpful if you used it control the polarization.  Also you have a polarizer before the camera You may want to consider seeing how the scattering is received by the camera using a quarter waveplate or otherwise controlling the received polarization.

However what I don't see in you diagram are any apertures. You may want to have a variable aperture in front of the sample, and another one in front of the lens. The reason is that the backscatter should be very highly peaked and very directional, and the apertures would allow you to limit the light to small angles. This would also cut down stray light. 

Probably keeping optics very clean would help.  A lot may depend on your sample.

Thank you Prof.John for your valuable reply. My samples are mostly in colloidal form.

I looked around a little and I think the attached paper has nice diagram that shows a set up that you could use. Looking at milk also seems like a good way to do some cheap testing to see if the set up is working well since it looks like the colloid could change over time.