There is no absolute definition of 'ultrastructure'. When the term was first introduced (in 1939), it was used to refer to structures that could not be resolved by conventional light microscopy, which meant structure smaller than about 0.2 micrometre. Since that time, microscopical methods have extended the capability of light microscopes, and many new imaging methods have been developed that can resolve structural details that could not be resolved by the electron microscopes of the 1950s and 1960s. The Journal of Ultrastructure Research began publication in 1957. A paper published in the second issue (December 1957) was titled "High resolution microradiography with ultrasoft X-rays", and a paper in 1958 has the title "X-ray diffraction of stopper cork". Clearly, the editor of this influential journal considered that fine structure revealed by techniques other than electron microscopy was also "ultrastructure". I think what's important here is not the method, but the outcome.

From the practical point of view, I think the term 'ultrastructure' should be used for all cellular and extracellular structures revealed by transmission electron microscope because in the published research it is rarely used for structures revealed by other techniques.