Pure unsaturated polyester resin having about 50% styrene content shrinks more than epoxy resin. The volumetric shrinkage of unsaturated polyester resin is somewhere between 5 and 13%, depending on the polymerization temperature. This is the reason why an additive called low profile agent, typically a thermoplastic polymer, is added to reduce the apparent shrinkage by inducing the porous morphology within the phase separated thermoplastic phase, by taking advantage of the different vitrification temperature of the polyester and thermoplastic. The thermoplastic phase acts as a reservoir of styrene monomer. On the other hand, epoxy typically shows the volumetric shrinkage between 2 and 7%. If the volumetric shrinkage is a problem for your particular application, a new thermoset resin called benzoxazine resin shows near-zero shrinkage upon polymerization and shows much higher thermal and mechanical properties than the epoxies and unsaturated polyester resins.
In general, epoxy shrinks less than polyester and vinyl ester resins. Not talking about mechanical property, as epoxies cure with low shrinkage, cosmetically, polyester and vinyl ester resins shrink up to 7% volumetrically and because the resin continues to cure over long periods of time this effect may not be immediately obvious. In comparison, epoxies shrink less than 2% and an epoxy laminate will be a lot more stable and have better cosmetics over a long period of time than a polyester one.
Polyester resins have short curing time and high shrinkage that causes stress, micro cracking of resin, and imprecise adherence of resin to fibers (micro pores).