The colossal and exhaustive text on aquatic insects, “An Introduction to the Aquatic Insects of North America” by Merritt et al. includes a brief discussion, and literature review, on rearing immature aquatic insects. However, in my, admittedly limited, experience, most species are easy to keep in the laboratory.
The key concern is to reproduce their outdoor living conditions as closely as possible. That generally means paying attention to current, dissolved oxygen and temperature. Use water from the source of the insects. Stream insects naturally require moving (recirculating) water and high oxygen tensions. We have had good success keeping caddisflies in a cold room, or even a refrigerator, with a simple aquarium filter and an aeration stone.
Depending on your resources and objectives, you may want to invest in a more formal recirculating tank, and worry about environmental variables like cover and light:dark cycles. But we have had reasonably good success keeping insects for a short while in ordinary aquaria, as long as they were kept cool and the water was moving. The only nuisance is that, like keeping fish, the water should be replaced every week or so, which entails a lot of hauling water from a local stream or lake.
A few soft leaves, or lettuce, will support leaf-shredders for a good while. If the litter is conditioned by exposure in a stream for a week or two, it makes more nutritious food. Mayflies seem to be pretty undemanding, apparently getting by on particles in the water or on the substratum. I have raised mayflies to emergence in a fish tank. I have no experience with filter-feeders, which are probably tricky, and predators need prey.
I hope that helps. It would be useful to hear from others who have deeper experience than mine.