If your system is reactive in water, then you have to change water medium by other like Silicon oil, Ethylene glycol, etc. I hope it will work very nicely in the measurement of density using Archimedes' Principle.
These RG discussions may possibly help: https://www.researchgate.net/post/Can_I_use_water_displacement_test_in_getting_the_density_of_lime https://www.researchgate.net/post/Use_of_Benzene_in_determining_the_density_of_glasses_by_archimedes_principle https://www.researchgate.net/post/How_do_i_calculate_for_percentage_porosity_in_a_ceramic_material/1 https://www.researchgate.net/post/How_do_i_determine_the_relative_density_of_my_ceramic_sample
You can use an inert intrusion fluid like silicon oil as previously recommended. Nevertheless, is important for you to know if there is open porosity in your ceramics. In this case an impregnation fluid with a low viscosity is required.
You can have a look to EN 623-2, ISO3923-1, ....
In addition to pycnometry, mercury intrusion porosimetry will give you good results.
The best method would be to use another fluid than water, which does not react with the sample, as has been mentioned. If you use He-pycnometry beware that samples with an open porosity will give you wrong results. He-pycnometry is useful for measuring on powders and samples with closed porosity. Samples with open porosity will get too high values of density using He-pycnometry.
See ASTM C188 Standard Test Method for density of hydraulic cement. This spec shows how to determine the powder apparent specific gravity using kerosene. If you want to determine bulk specific gravity, apparent specific gravity, and absorption (apparent porosity) of bulk samples, you can find the equations in ASTM C373 Standard Test Method for Water Absorption, Bulk Density, Apparent Porosity, and Apparent Specific Gravity of Fire Whiteware Product. Note that the determination is for the "apparent porosity," which is the accessible porosity (interconnected). Higher fired ceramics likely have closed porosity, which can be determined from comparing the apparent specific gravities of the powdered and bulk specimens (the difference is due to the closed or inaccessible porosity). Other anhydrous liquids could be used, such as propylene glycol, isopropanol, or ethanol, but kerosene has typically been used.