Can I run a regression analysis only with interaction terms after I investigated in a first model the main effects? The problem is, if I put everything together in one model the results do not make any sense.
Make sure that your independent variable are uncorrelated.
Sample size could be an issue.
Often when I hear phrases like "if I do this the results do not make any sense" I also find that there is an assumption that a failure to detect a significant difference is proof that there is no difference. This is really bad statistics. A lack of significance is not proof of anything. Many would say that the presence of significance is not proof of anything.
What testing in statistics gives you is the probability of finding an observation as extreme as that observed given that the null hypothesis is true. The 0.05 value is entirely artificial. There is no proof either way no matter how you use that artificial value.
In general, your model should be hierarchical. If you have a dependent variable y and three independent variables A, B, and C, then you start with y = A + B + A*B + C + A*C + B*C + A*B*C where B*C is the interaction between B and C. If A*B*C is not significant, then it can be removed from the model. If A*B is significant then the model must have both A and B regardless of significance.
If you are using statistics in an exploratory capacity, then I don't see a reason for changing this suggestion. If you are fitting a model based on known relationships then there are cases where the interaction term could be significant but not the main effect. However, you cannot use your data to prove such a model.
If you happen to be using polynomial regression, then the advice about keeping the model hierarchical does not apply.
It might help if you provide more explanation on how the results do not make sense.
Of course, I checked for multicollinearity and I suppose my sample size (1800) is also big enough to analyze the following question in general.
I am interested in demographic differences (age; sex) in my dichotomous outcome between three countries (A: n= 380, B: n=600, C: n=700).
In a first analysis I conducted a main model with age, sex and country to predict my outcome and after that I used three separate regression analyses for each country one regression model to investigate country differences in demographics concerning my outcome. I compared then if sex or age was a sig. predictor in each country model. However, the Reviewer said that this is unacceptable and asked me to use one model with interaction terms (age x country and age x sex). So far so good. After I added the interaction term to my main model the results of my main regression changed a lot. For example before I added the interaction term to the main model participants from country A showed higher odds for my outcome compared with participants from country B or C, which I could also see by looking on the frequency rates 10% vs. 5% and 6%. However, after I added the interaction terms participants from country C showed higher odds. Moreover, sex and age were not sig. predictors anymore after adding the interactions to the main model. Therefore, I though it might be possible to test in one model the main effects and in another the interactions.
So what do you think? Can I just conduct two models?
Thanks Mehmet. But my question was, can I calculate two regression models. First one with main effects and then another one with interaction terms only.
What I mean, use just one regression model with interactions. Why two models? Do you need to select among different models? You say that the results differ, so this may indicate a significant interaction. From your explanations I don't understand why the results of model with interactions are not acceptable. The model with only interaction terms does not make sense as Timothy pointed.
The meaning of the interaction term depends on what main factors are in the model. Almost surely, the meaning of the interaction in a model without main effects has not the meaning you think it has, or a meaning that would be practically relevant.
Thus, unless you are very sure about the interpretation of the interaction in an "interaction-only-model" and you have a clear explanation why and how this is relevant for the research problem, then ok. Otherwise I would listen to the reviewer.