We can assume that faster growing fish have a faster metabolism and consume more food resources. Parasites that take resources directly from the intestine of their fish host (acanthocephalan parasites for instance) might thus benefit with more resources available and grow larger.
This will depend largely on the balance between nutrient availability and the efficiency of fish immune system. In the first case, as Sophie answered, intestinal parasites are surrounded by digested food but they are far from the reach of fish immunity components. In contrast, blood parasites will be more vulnerable to immune defenses although it is surrounded also by available nutrients which blood carry to other body parts. Therefore, parasites in the second example will show very limited growth in comparison with first example even in large fish.
I agree with Sophie. However in the case where fish hosts have similar resources available, one can imagine that there is variation between fish in resource allocation. For example fish that allocate more energy to growth, and thus grow faster, can spend less energy on their immune defence than conspecifics (or species) that invest less in growth and more in immune response. This could result in parasites being able to establish more easily within fast growing hosts, or make it easier for them to suppress the host immune response against them and thus allow for faster growth.
Parasites depend on host-derived energy for growth and development, and so are potentially affected by the host's ability to acquire nutrients under competitive foraging scenarios. Although parasites might be expected to grow faster in hosts that are better at acquiring nutrients from natural ecosystems, it is also possible that the most competitive hosts are better at countering infections, if they have an improved immune response or are able to limit the availability of nutrients to parasites. I first quantified the ability of uninfected three-spined sticklebacks Gasterosteus aculeatus to compete in groups for sequentially-presented food items, and then exposed either the best or worst competitors to infective stages of the cestode Schistocephalus solidus. Fish were subsequently raised in their original groups, under competitive feeding regimes, for 96 days, after which fish and parasite growth was determined. Unexpectedly, pre-exposure host competitive ability had no effect on susceptibility to infection, or on post-infection growth rate. Furthermore, despite a 120-fold variation in parasite mass at the end of the study, pre-infection competitive ability was not related to parasite growth. The closest predictor of parasite mass was body size-corrected host growth rate, indicating that the fastest growing fish developed the largest parasites. Faster growing hosts therefore apparently provide ideal environments for growing parasites. This finding has important implications for ecology and aquaculture.