So panel data itself has a multilevel structure - it is wrong to think of panel data as 'single level'. In your case, that panel structure is occasions (level 1) nested within firms (level 2), and it is this structure that xtreg will model (assuming that is how you have specified your panel structure). So if you use the fe option with xtreg, this will, as you say, control out all variation at the firm level and above, and so solve the problem of higher level omitted variables (but not occasion level omitted variables).

If you are interested in higher levels, you would want a three-level RE model - occasions (level 1) nested in firms (level 2) nested in states (level 3), which could be fitted using the mixed command. Those higher level variables cannot be identified in the FE model (it's already controlled out by the firm fixed effects).

Note that you can account for firm-level omitted variables with RE by using the mean of the occasion level variables, so you don't have to use FE - see https://www.researchgate.net/publication/233756428_Explaining_Fixed_Effects_Random_Effects_Modeling_of_Time-Series_Cross-Sectional_and_Panel_Data for how).

Article Explaining Fixed Effects: Random Effects Modeling of Time-Se...

i always like to split the file into cases and compare levels in that way, this runs panel regression for each level instead of making interpretation of 'Mixed'-models or other combined methods difficult. 

So panel data itself has a multilevel structure - it is wrong to think of panel data as 'single level'. In your case, that panel structure is occasions (level 1) nested within firms (level 2), and it is this structure that xtreg will model (assuming that is how you have specified your panel structure). So if you use the fe option with xtreg, this will, as you say, control out all variation at the firm level and above, and so solve the problem of higher level omitted variables (but not occasion level omitted variables).

If you are interested in higher levels, you would want a three-level RE model - occasions (level 1) nested in firms (level 2) nested in states (level 3), which could be fitted using the mixed command. Those higher level variables cannot be identified in the FE model (it's already controlled out by the firm fixed effects).

Note that you can account for firm-level omitted variables with RE by using the mean of the occasion level variables, so you don't have to use FE - see https://www.researchgate.net/publication/233756428_Explaining_Fixed_Effects_Random_Effects_Modeling_of_Time-Series_Cross-Sectional_and_Panel_Data for how).

Article Explaining Fixed Effects: Random Effects Modeling of Time-Se...

Andrew Bell is right. Fe option does not allow us the freedom to see the impact higher level variables i.e. state level on firm level level outcomes. Knowing the impact of higher level variables is much more interesting for econometricians as this may have important things to reveal about of policy. While using Fe option may give better control over unobserved but it does little in giving us information that may help us in policy making.

Thanks Andrew Bell for such a lucid response differentiating a panel regression from a multi-level model. However, people opine that a panel regression with FE option with clustered robust errors (cluster variable, state) should solve the issue of heterogeneity of the different states. So, if I run such a panel can I still differentiate state-level variances accounting for firm performance in the different states over time?

Also, should the ICC of the null model be THE deciding factor for doing an MLM? What about theory being evidence of support for doing an MLM?

Thanks,

Sumita 

So I think robust standard errors might help 'fix' (although see below) issues like heteroskedasticity across states, but it won't bring that state-level variance back into play - that has already been removed from the model by the FE's, assuming that firms are hierarchically nested within states. (Although it may be that firms move between states, or belong to more than one state? In which case robust SE's would be a way of dealing with any remaining clustering)

You might want to have a read of this paper by Gary King: http://gking.harvard.edu/files/gking/files/robust_0.pdf It argues that robust standard errors are a correction that doesn't fix the underlying problem that is causing the differences between the models and it is better to model those problems explicitly. A sensible course of action might be to run the model with and without robust SEs; if there is a difference in the SEs, think about why, and if not don't worry about it!

For me, I would fit a multilevel model: occasions nested in firms nested in states, and explcitly model any heteroskedasticity (if present) using random slopes. I would say that heteroskedasticity is substantively interesting and not something that should be 'corrected'.

Re deciding on MLM - I would say testing the significance of RE variance (using likelihood ratio test, AIC, BIC etc) is how you would decide if those REs are needed. I guess there is no harm in including additional levels theory says it should be important, but if your model finds no variance between units, then that is what if finds, and as a general rule parsimony is a good thing. Eg if there are no differences between states, that (a) is interesting in itself, and (b) makes your life significantly easier from a modelling point of view.

Attn Andrew Bell, what if I controlled for the year effects by using an ARMA (Auto Regressive Moving Average) model and subsequently add the predictors (both state- and firm- level) and finally the cross-level interactions?

Thanks in advance for your response!

So I'm not very expert in ARMA models, but that wouldn't solve the issue of dependence within firms and within states - you would need random effects for these levels to deal with that.

A

Got it thanks. Your article is extremely informative as I am currently faced with a situation you described. The Hausman Test supports an FE estimation, when my research question itself is to investigate the impact of macro variables on a micro variable...Before reading your article, all along I had the intuition that using that test is just wrong for my current research....as resorting to a panel regression totally defeats the basic premise of my paper. However, am not sure if I can proceed with a panel regression with RE effect (ignoring the Hausman Test)? Thanks again! You have been extremely helpful and informative as you are an expert of both panel regression and multi-level modeling.

You should be fine, so long as you include the firm mean of occasion-level variables in the model, as suggested in that paper. (note there might still be issues of omitted variables for those higher level variables, but at least you can get estimates for them which you cannot with FE)

For further development of the debate you might to look at this recent paper

https://www.researchgate.net/publication/299604336_Fixed_and_Random_effects_making_an_informed_choice

and volume 2 gives an extend account of the multilevel approach to panel studies including 3 level models and those with 'extra' serial autocorrelation

https://www.researchgate.net/publication/260772180_Developing_multilevel_models_for_analysing_contextuality_heterogeneity_and_change_using_MLwiN_Volume_2

Article Fixed and Random effects models: making an informed choice

Book Developing multilevel models for analysing contextuality, he...