I am working on numerical wavelet analysis, recently for approximating some highly oscillating function in a very small support in finite domain some difficulty arise.
It depends entirely on your context, like whether you have a continuous or a discrete inner product. The Dirac function (or rather its action as a distribution in an integral) can be approximated in may ways. For example, as a limit of normal (Gaussian) distributions, or in a completely different setting, in terms of cardinal sine functions, i.e., the sinc functions, sinc(x) = sin(pi*x)/(pi*x). Check out the Wikipedia article on sinc functions to see whether it might be of use to you.
But without further knowledge of your context, it's impossible to answer the question.
Dirac delta function can be approximated using a sequence of Gaussian functions. But there are other methods also but you have to select it depending on the nature of the approximating functions you are seeking.