ShinEtsu Chemicals of Japan seems to produce transparent silicone material which can withstand temperatures from -30 degrees centigrade to +200 degrees centigrade.
Almost any material, transparent or otherwise, will survive cooling to the temperature of liquid He (4.2 K) and lower if it is cooled slowly and uniformly enough. Materials with low thermal conductivity and large coefficient of thermal expansion will crack if cooled quickly or non-uniformly because the coldest area will contract while the rest remains warm, thereby producing tremendous stress in the material.
Another problem is differential thermal contraction. For example, if a Perspex window, which contracts ~2% to 4 K, is glued to a heavy stainless steel flange, which contracts ~0.3% to the same temperature, something will break, whereas the same window, not attached to anything, will remain intact if uniformly and slowly cooled.