You will need to do some washing after the antigen retrieval anyway, so just use cold buffer/MilliQ water for it. I usually move the slides from the metallic rack I used for antigen retrieval into glass jars, then just do 3x5 mins to wash. By the time washing is done, the slides will be cool enough.
We put our plastic MW rack with the slides and hot buffer under a very thin stream of tap water carefully not to place it directly over the slides (the glass may crack), but on the surrounding plastic. This way it slowly cools down the buffer. Excess liquid is discharged slowly (Citrate buffer mixed with tap water). After 5 minutes, the slides are cool enough. This currently works for our method, but I'm not sure that this is applicable in all cases.
We have let the slices to cool down in their own rack (with their hot antigen retrieval solution) plunged in another container in which I have put cool water with ice, and this reduce time in 10-15 minutes, and do not cause background. After that I wash the slice and follow the protocol as usual. You have to value if this cause background in your case.