In EELS, you only collect forward scattered electrons with a very small scattering angle (due to the loss of some 100 eV energy). The collection angle for EELS is typically below 100 mrad (in most cases 10-50mrad). So, electrons scattered at higher angles will not contribute to your measurement. Also, EELS samples should be very thin to avoid multiple scattering (this leads also to beam spreading). Multiple scattering gives bad signal/background ratio because the sharp features in your spectrum wash out.

In EDS, you collect x-rays from a much larger angle, in fact the larger your detector the better your signal. Anyway, most of the x-rays will not hit the detector. The perfect EDS detector with 100% yield would completely surround your sample and convert every single x-ray, this is of course impossible to construct. But you collect all x-rays generated in the sample and leaving it in the direction of the detector. Electrons travelling through the sample and being scattered multiple times at any angles will generate more x-rays, and all of them contribute and increase your signal. In contrast to that, in EELS you collect basically all of the inelastically scattered electrons from a very small angle (when you choose an appropriate collection angle by camera length and apertures). Multiple scatterd beam or spreaded beam will not be collected.