No, there is no requirement that all possible product terms be included in the model.  Many authors (e.g., Aiken & West, 1991) do maintain that if you include a product term, all lower order interactions involving variables in that product must be included in the model.  In your case, suppose you did want to include the 3-way product term, X*Z*K.  In that case, you would have to include X, Z, K, X*Z, X*K, and Z*K too.  But it sounds like your model of interest is:

  Y = Constant + B1*X + B2*Z + B3*K + B4*X*Z + error

That's fine--especially if there is some rationale (e.g., prior literature) for including that one product term, but none of the others. 

HTH.

http://books.google.ca/books/about/Multiple_Regression.html?id=LcWLUyXcmnkC

No, there is no requirement that all possible product terms be included in the model.  Many authors (e.g., Aiken & West, 1991) do maintain that if you include a product term, all lower order interactions involving variables in that product must be included in the model.  In your case, suppose you did want to include the 3-way product term, X*Z*K.  In that case, you would have to include X, Z, K, X*Z, X*K, and Z*K too.  But it sounds like your model of interest is:

  Y = Constant + B1*X + B2*Z + B3*K + B4*X*Z + error

That's fine--especially if there is some rationale (e.g., prior literature) for including that one product term, but none of the others. 

HTH.

http://books.google.ca/books/about/Multiple_Regression.html?id=LcWLUyXcmnkC

Thank you very much Bruce!

I am interested only in the two-way interaction but I wanted to add a third variable for control purposes.  So in that case, I wanted to make sure if I could include the third variable as a sole variable or if it was necessary that I interact that as well.  Thanks again! 

@BRUCE is correct. To extend it for the second part of your question from @Bruce's equation:

Y = Constant + B1*X + B2*Z + B3*K + B4*X*Z + error

Here, you are asking whether you should test for the interaction for the term K with Z and X. This K is your control factor. The answer is YES, if found "significant" then it cannot be a control factor. if found significant, you must report thus extend @Bruce's statement to B5 and B6:

Y = Constant + B1*X + B2*Z + B3*K + B4*X*Z + B5*XK + B6*KZ + error

In this case (1), you do not have a control factor but 3 interaction effects. You would need to look for a new control factor. If the test for the interaction of between K and X & Z is not significant, then K may be called control factor. The test for significance would follow:

T = (B1 - B2) / SQR(C + D) / E

A = n1SE12

D = n2SE22

E = n1 + n2 - 2

Test for the following pairs: (X,Z), (X,K) and (Z,K).

Eltion: Yes, you can add and further variables for control purpose to the model. This is independent of the interaction you like to study.

You want to investigate Y ~ X*Z (this is a "model formula" not a mathematical formula)

But you know that Y is influenced by K, and K varies in your experiment, so you want to "adjust Y for the impact of K to more clearly see the effect of X*Z":

Y ~ K + X*Z

Bruce (correctly) noted that Y ~ X*Z means Y ~ X + Z + X:Z (main effect X and Z should be included when the interaction X:Z is analyzed).

So, when X:Z is in the model, then X and Z have to be there as well. This is the only restriction. You can add other predictors to the model as you wish, linke in Y ~ K + X+Z+X:Z, or Y ~ K² + K + X+Z+X:Z, or Y ~ A + B + K + X+Z+X:Z, or Y ~ A+B+A:B + K + X+Z+X:Z, or...

If the researcher is aware about the possible interaction, then he/she should test this through the appropriate statistical tests. If the result shows the presence of interaction, then yes to be included.

Dear Professor,

using (either null hypothesis- or significance-) tests is usually not senesible for model selection. There are hundreds of more imoprtant things to consider, like the substantial, scientific ideas behind a particular model, the purpose of the model, the costs of obtaining the values of the variables, etc etc.

If you really quantify the benefits and losses of your possible alternative models and if you really specify the maximally tolerated expected loss ... then and only then is it sensible to base a decision about the choice of a model on the result of a hypothesis test (because all the important questions had to be addressed to come up with the alternatives, the costs, the loss function, the study design, the determination of the corresponding error rates, etc).

Ritualied testing based on dogmatic conventions is like soliloquizing when you actually want a discussion. You may learn more about your particular opinion, but do not improve your point of view.

Eltion,

First, endorsing Jochen's point. Do not use significance testing for model building, especially removing ns terms in a post hoc fashion.

For the original question, Bruce describes the classic thinking -- for any included interaction, all subordinate terms to that interaction should be included.

However, there is a back-and-forth in the methodological literature related to your question: whether all possible interactions of the same order of the one of interest (e.g., for your example, all possible third-order interactions, and, by extension, all second-order interactions) need to be included to avoid model misspecification, regardless of your hypotheses about those interactions. This is not my expertise, so I don't have citations handy, but this has been an active topic for the last 5-10 years or so.

Pat

No, there is no need to include all the interactions in the model if your requirement is only two way interaction. Write your model accordingly.

 Hi Eltion Meka,

The following is based on Greene Econometric 6th edition, I thought considering them might be useful to your decision making.

  Two common errors in regression model specification:

1. Omission of relevant variables (leads to biased estimator, but possible lower variance).

[In this case exclusion of those interaction which belong to the true model will cause the biased estimation of coefficient. ]

2. Inclusion of superfluous (unnecessary) variables ( leads to unbiased estimator, but higher variance)

[In this case if the interactions do not belong to the true model, then the estimated coefficient will have higher variance.]

---------------

The following is my own idea, which may not be correct

It seems model building is a process , you need to go back and forth, test the residuals, restriction , functional forms and some fit criteria to reach a reliable model.

The entire statistical analysis - any statistical analysis - is an art.

"One-click-solutions" or automated recipies are never a good thing to apply. Unfortunately, many stats books give the impression as if this was the case - they are like cookbooks (some even have it in the name or the advertising texts!).

There may be (more or less) "common", well-known and well-understood problems that do have known and widely accepted solutions, but major task is still to recognize if the problem at hand really belongs to this category.

What is often totally ignored is that many stats books provide "guides" what claculations can be done while completely lacking the scientific part. So peopple tend to do the calculations but forget to think about the science. A prominent example is that hypothesis tests are performed and p-values are calculated, but the interesting questions are not touched, like: (i) about the reasonability of the data and the model, (ii) about the value and performance of thinkable competing, alternative models, (iii) about the estimated effect size and their substantial relevance.

So, yes, the analysis is a process. It starts with thinking. Model-building is an intellectual process, that is guided (but not dictated) by data. Reaseach is exploration. Also exploration of data, leading to new ideas and the adoption of models, trough thinking, and also guided by the purpose of the model.