Lucy Whitehall Yes, you should remove such interaction terms. The regression coefficients require the variable matrix to be inverted which is not possible if you have singularity. What it shows that some features are highly similar to some other (or combination of) variables. You can also try stepwise regression if you do want to include interaction terms and find out what features to retain.

DO NOT TRY STEPWISE REGRESSION. It is unstable i.e. successive runs of the SAME data give different results. See the ref's and the attached for details and what to do about that. I prefer to use adaptive lasso rather than stepwise. If you Google Boos Statistics North Carolina State University adaptive lasso you will find a nice website with references and examples and finally a very easy to use R program for your data. Hope this helps. Best wishes, David Booth

PS I forgot to say that if you give us some info on your stat package it might help. My suspicion is that some variables are entered incorrectly or there is not enough data since your basic model is now a representation of some sort of factorial experimental design. BTW using lasso techniques assumes the goal of your regression is prediction. Again see the attached paper. Best, DB