As per my knowledge there is no such modelling software available for wave over topping. but i can suggest you one software which is used to design a breakwater. you can allow over topping and you can see the response. click the following link for more information about the software.
Both webpages have a dedicated video (youtube) section. Have a look at it, you''l be amazed by how much you can do with an open source (i.e., free of charge!!) and high quality software.
If your question is about the numerical methods available (and not about the available software) the answer will be finite volumes, finite elements and finite differences are the numerical methods mainly applied to this kind of problems. About the models themselves, depending on the size of the problem, Navier-Stokes 3D or shallow water models with or without dispersion and available models that can do that many besides the ones cited above (HySEA https://edanya.uma.es/numericalintromenu; GeoCLAW, any of the many tsunami models existing, NEOWAVE, FUNWAVE, and many others).
Actually I am looking for numerical modelling techniques for prediction of wave overtopping of some existing data. Will the above answers help me with that?
A useful starting point, if the main aim is determining wave overtopping discharges foar various wave conditions against a threshold criteria, is the EurOtop wave overtopping manual. This contains various empirical formulae derived from many physical laboratory studies of rock/rubble revetments, dykes and vertical walls.
http://www.overtopping-manual.com/manual.html
More definitive assessments on the actual breakwater configuration and surrounding physical context e.g. bathymetry, coastal planform, wave refraction, can then follow through physical laboratory or numerical techniques as described by other respondents.
I think, Karthika, you must be more precise in expressing what do you really want or need. First you ask about numerical modelling techniques, and then, in you last message, you say "I am looking for numerical modelling techniques for prediction of wave overtopping of some existing data". Again you talk about numerical modelling techniques, but you say "of some existing data". And I wonder if you are really looking for numerical techniques or indeed what you want and what you need is a numerical model, code, software for taking that "existing data" you mention, and then to perform a numerical simulation. Concluding, first you must clarify what you are really looking for, making more precise your question.
I think she is working on breakwater and looking for a software which predicts wave over topping for different water levels and at different crest levels of breakwater.
In simple methods, if you use long wave equations for wave transition there is some simple schemes for modeling of flooding and drying over bed. If the bed is rigid or porous makes the problem some easier or more complicate. For this simple approach I propose papers of Van der Meer and Van Gent 1992 and before. But if you have to use more complicated equations, as Dr. Ketabdari said VOF is a practical method, however if you have to simulate over topping over a porous breakwater you have to solve the NS equations in porous and hydraulic media.