a) experimental: prepare homogenous standards having the same density and geometry as your samples, #1 material with low K,Th,U content, #2 same as #1 with known amount of K added, #3 same as #1 with known amount of Th added ,and #4 same as #1 with known amount of U added.
From the differences between measurements #N-#1 you may deduce the contribution of each of the three elements to the spectrum and, more specifically, the contribution of each of them to three regions of interest corresponding to strong peaks of each element, usually 1.46 MeV for 40K, 2.61 MeV for Tl208 (progeny of Th) and 609 keV for Bi214 (progeny of U), assuming secular equilibrium in Th and U families, or at least the same disequilibrium conditions in sample and standards.
It is then possible to calculate a matrix relating the measurements in the 3 ROIs to the three activities.
b) Monte Carlo simulation. The NaI detector is usually quite simple and can be simulated with a good accuracy. Experimental measurements are still necessary to evaluate the gaussian broadening of the peaks at the different energies, and an experimental validation of the MC model is always desirable.