sometimes the males even "willingly" offer themselves to the female. One reason that has been proposed is that male sacrifice may ensure no other male will copulate with that female, possibly displacing the first mate's sperm. This idea is supported by experimental evidence, showing that, on average, a female spider has a lower probability of mating a second time if it devours its first mate. A recent paper points to a measurable fitness advantage accruing to self-sacrificing males: their offspring number is increased. When the female is fed another prey than its mate (eg a cricket), offspring number is not increased, suggesting there is a specific "nutrient" in the male that pulls off that trick... see Curr Biol. 2016 Oct 24;26(20):2794-2799. doi: 10.1016/j.cub.2016.08.010.
In part of your question, you asked "which type of species display this habit more frequently and why". In wolf spiders and other spiders, sexual size dimorphism (in this case, how much larger the female is relative to the male) plays a big role. The bigger size difference, the higher rate of cannibalism. This trend has been found in Hogna helluo (now Tigrosa helluo) (Wilder & Rypstra 2008), Schizocosa ocreata (Persons and Uetz 2005), and is also a trend across a variety of spiders (Wilder & Rypstra 2008 and citations within).
Check out the Wilder & Rypstra paper by searching for the title "Sexual size dimorphism predicts the frequency of sexual cannibalism within and among species of spiders". It should be available here on Research Gate.
There's a lot more to be said, but hopefully that helps!
A general answer is that the female is increasing her fecundity via the nutrition obtained from eating her mate. From the male perspective, surviving to mate again may be preferable, but toward the end of his life the prospects of remating diminish. Thus, one may expect the male to offer less resistance to being eaten. These predictions need to be checked against data, and there are now many studies in the animal behavior literature addressing the question. Michael Greenfield CNRS Tours, France