.

gammafn is the name for the gamma function ; even "small numbers" (say more than 170) can produce "infinity" results therefore the "out of range" warning

https://www.biostars.org/p/14299/

the way around is to use the Log of the gamma function (lgamma) but if you're just a user of the package, this might be hard to fix

.

Yeah, sounds hard to fix to me. But I will try to get into it, thank you for your answer!

The error that you are facing is related to the values of your variable not fittin g the requirements of the gamma function. may be you need to change the method and function you are using for the analysis. or may be make transformation that fits the data to the gamma function.  

The warnings, especially when they appear often, should not be disregarded.  It may happen, that the results produced by the software, are meaningless.  Brian, are you sure, that your computed results make sense? 

First question: What is the role of the gamma function in the applied statistical model? Judging from the information  that a treatment-subgroup interaction is tested . I guess that a F-test is calculated, and the probability distribution function of F (also of chi-square) somehow use gamma functions. Getting some extremal arguments for these functions indicates that something is wrong with the data. May be you have some empty classes in the assumed linear model, may be too few degrees of freedom, may be the (the non-zero) data  have variance equal zero? May be too many parameters to estimate and too few data? 

 In any case, to consider the results as trustworthy, the reason of the warnings should be clarified.

Sorry, this does not sounds optimistic.  

Anna

Thank you for your recommendations, Mahmoud and Anna! Off course, I am going to explore the reason for the warnings.