Best way of separation is using 200 micron net. Pass the sample through it and the phytoplankton will go through the mesh. The organisms retained on net will be mostly meso-zooplankton. However, microzooplankton will also pass through the net. Also the net needs careful thorough rinsing to avoid blockage as it will retain phytoplankton also if blockage happens. But the best way avoid such issue is collecting phytoplankton and zooplankton separately in 55 and 200 micron nets
I think for separation of them,besides using mesh and other mechanical methods, could be used chemical methods i.e . Applying different reagent,for the zooplanktons more sensetive than phytoplankton.
You can start with any of the net suggestions above, like the 200 micron net, then move to smaller screens (sieves). The next finer step would be picking cells with a micropipette to get the alga you want.
It might look easy to just sieve your samples as many suggested, but in the end, it is advantageous to identify the species. E.g. so many of the protists are mixotrophs, i.e. fall in both categories (phyto- and zooplankton). Why do you need this distinction anyway? Maybe for writing it up, it might be easier to stick to the size fractions, instead of using the terms phyto- and zooplankton.
I am currently using a sieving procedure through sieves with different mesh sizes. It works quite well. The reason for my classification is given in the report above