This depends on what you mean by hologram and fringe separation? If it is an off-axis hologram, then one takes the Fourier transform of the image and locates the carrier frequency.
If you are talking about Fresnel fringes for an in-line hologram, there isn't a well defined answer to the best of my knowledge. The Fresnel fringes come from the free-space propagation of the wavefront, which is often modeled by the Frensel propagator. Goodman's book on Fourier optics has a good discussion on the topic. In Fourier space Fresnel fringes make a streak, not a point, so there isn't really a defined spacing. One can measure the maximum-to-maximum fringe peak spacing, but it usually won't change in a deterministic manner unless you have a very specific geometry.
Since it is digital (as you say), I also recommend the Fourier analysis for a given frequency. You may find Fourier transform packages for most of scientific programs in the web. However, for very simple and ideal cases, you may also use the algebra from "Physics of optical recording" by Schwartz, Kurt