There is quite a lot of literature on this subject. For example try searching “angular dependence of thermal emission”

Neither thermal emission nor thermal reflection are generally independent of angle. However, for many surfaces, it is fairly independent until the angle gets to be quite large (> 45 degrees). In the visible we would recognize these surfaces as being “flat” (not flat, as in a plane, but ”flat” as opposed to “glossy” like paint). This is, as you suggest, a function of surface roughness. Unfortunately, the roughness scales with wavelength, so something that seems rough and so flat in the visible might be fairly smooth to 10 um light. Anyhow, with sufficiently rough surfaces, the emission per unit area tends to follow cos(theta) vs angle at low angles (Lambertian). Meanwhile the observed projected area increases with cos(theta). Taken together these mean that the radiance is constant. This is almost always the case for particulate matter where reflection often involves multiple bounces. However, even for particulate matter the emissivity falls off above 45 degrees and sometimes dramatically all the way to zero at 90 degrees. Nevertheless they can be quite flat from 0 to 45 degrees.

With less rough surfaces both emission and reflection can have large dependence’s on angle. The scale of roughness is the wavelength, and that depends on temperature.

You are right, there is indeed a large body of literature addressing the angular dependence of both thermal emission and reflection. Most discussions, as you mention, appeal to Lambertian assumptions and surface roughness arguments, which work well up to moderate angles but break down at grazing incidence.

In our recent work we tried to make this more precise by deriving a quantitative thermography equation that explicitly incorporates the angular dependence through emissivity, reflected apparent temperature, and the opto-geometric factor of each pixel. This formulation allows the angular effect to be written in a closed equation valid for non-perpendicular infrared measurements: Thermography Equation for Non-Perpendicular Infrared Measurements: Derivation, Analysis, and Experimental Validation

It might be of interest because it links the qualitative statements about angular dependence to an equation that can be used directly in measurement and calibration.