Also, are you asking about just a single interface? If so, you are looking for what is called the Fresnel equations. They calculate the reflection and transmission coefficients for angle angle of incidence. If your problem contains multiple interfaces, then see prior comments. If your surface is rough, very often the roughness is treated as a separate layer with some effective material properties.
La UIT tiene dos recomendaciones que se entrelazan en ese calculo. Yo lo he usado para la teoria basica de propagacion en sistemas celulares y en modelos de propagacion de linea de vista, estas son : R-REC-P.526-11-200910 y R.REC-P.527-3-199203.