If it is actually milk, it may be full of biotin, that gives the unwanted background by binding streptavidin-peroxidase-konjugat. Look for biotin-block.
If the previous suggestions are not efficient enough, I would strongly suggest to use an alternate reporting system such as alkaline phosphatase-based systems, for instance.
You use a primary-antibody raised in mouse on mouse-tissue? You use a secondary-antibody raised in rabbit that binds to mouse-immunoglobins? I think that could be the cause of your problem. I hope you can find a primary from a different species.
i completely agree with Gudrun. Use a rabbit polyclonal primary Ab, which has been tested for IHC and works in Hu Ms and Rt, such as the one from United States Biological (see the link) an then the labelled polymer-HRP anti-rabbit. This is the most straightforward.
It exists tricks when one need to deal with Ab raised from a species on the same species, but we do not need to worry here in the first instance ! ;-)