http://www.nature.com/srep/2013/131218/srep03533/full/srep03533.html tells us that
"There are three noise sources: the measurement circuit noise, the thermal fluctuations and the Johnson noise".
===
I would say that improving the circuitry, lowering working temperature, and adding a thermal stabilization should do the job. I guess, you can't modify the bolometer itself, can you?