Many of the kits say that you can elute with water (for example the Zymo kits is what we have in the lab now). So it should be fine. But as you say the DNA is likely to survive storage better in TE.
Actually, the buffer is for the elution rather than storage (or at least, in addition to the storage). That is because you need basic solution for elution. Hence, you can use water, but it shall be fresh or you should check the pH, because it may acidify when standing for long time.
Tris-HCl helps to maintain a stable pH environment for the DNA to be in so that it minimizes degradation. Tris-HCl buffer by itself at the concentration that we typically use (10 mM) doesn't really impact PCR, unless it contains EDTA (i.e., Tris-EDTA buffer or TE buffer). EDTA is a chelating agent that can binds to the Mg2+ that Taq polymerase need to function (Mg2+ serves as a cofactor for the Taq polymerase enzyme). So, when you have high concentrations of EDTA, they can bind to the Taq polymerase and reduce the efficiency of your PCR amplification. Typically, TE buffer contains 10 mM of Tris and 1 mM of EDTA brought to a pH of ~8 using HCl, but because of the ability of EDTA to chelate Mg2+, many kits now use only 0.1 mM of EDTA which is very little compared to the usual amount of MgCl2 that we use in PCR, so it doesn't usually inhibit PCR.
I work with low concentrations of DNA all the time since they're mostly fecal or other environmental DNA, and I would recommend eluting your DNA in either TE buffer (with only 0.1 mM of EDTA) or in Tris buffer (without EDTA) at around pH 8. Eluting in nuclease-free water is definitely fine and won't matter much if you are using the DNA immediately and is not planning to store it longer term. However, if you want to store it for extended periods, I wouldn't advise to elute in nuclease-free water.
Just a tip, if you want to increase DNA concentration, you can preheat whatever elution buffer you decide to use to ~60C and after adding the preheated elution buffer to your column, let it sit for ~5 min before centrifuging. You can also reduce the volume of elution buffer used to 50 uL instead of 100 uL, but you need to make sure that you'll have enough for your downstream applications.