This is a complex question, but at least one reason which comes to mind, is that a harsh environment for young may not be such as harsh environment for adults. If, once adults reach sexual maturity, they are relatively unlikely to die due to environmental pressures or predators etc., the iteroparity can be favored because adults are relatively invulnerable to outside pressures. If these relatively invulnerable adults experience an increasing mortality rate, then there is pressure for semelparity or at least a decreased age at first reproduction. The vulnerable juvenile, relatively impervious adult is a pattern which exists in some way or another for many organisms from plants to amphibians to fish to birds to mammals.

I agree with Benjamin, many different situations may be envisaged. In addition, small and short-lived organisms may breed continuously until they die (is this semelparity, or is the timescale relative?), resistant eggs or propagules may be produced to survive unfavourable situations, and parthenogenesis may be advantageous, as in crustaceans of ephemeral pools.

High adult mortality due to external causes is often thought to favor shorter life span, intense and precocious reproduction, with annuality as the extreme possibility (Gadgil and Bossert, 1970; Charnov and Schaffer, 1973; Michod, 1979; Charlesworth, 1980; Young, 1981; Reznick et al., 1990, 1996; Hautekeete et al. 2001 2008).

However the opposite is also sometimes thought to be possible (Reznick et al., 2004) when extrinsic mortality decreases population growth rate or can be limited by greater body maintenance (e.g. prey/predator system).

Besides recent contributions, basic theories are in 'Evolution of life history traits' (Stearns)