Dear Olatunji, I am not sure about the truth of your statement: "the calibration exercise are useful supplements to traditional econometrics analyses because they widen the range of empirical information used to study a problem". I can only agree with its first part when we consider counterfactual scenarios (which by definition cannot be addressed by a purely econometric model). Its second part is not correct. To which kind of empirical information are you referring? "A priori information"? Experimental information requires more control about the real environment of the experiment, not a "postulated environment". The experiments you can make with a DSGE model (as well as the counterfactual scenarios) have the validity of the set of assumptions made to built such a model, from which the credibility issue clearly arises. IF AND ONLY IF all these limitations are explicit inside an academic or professional discussion, then it is scientific (opposite to being "just politics").

Dear Olatunji, Dear Carlos greetings from Paris. I do agree with Carlos on his remark, so i will just add a few ideas from my own experience.

First, I will recall that econometricians are not in a better position given the bold assumptions that are never discussed behind most econometric models. I refer here to the classic famous 1938 critic by Keynes of Tinbergen work for the SON in the mid 30's. When we consider the exploding number of meta analysis on econometric results in different fields with even more ludicrous hypotheses: random independent sampling in a population of studies and models... Can we seriously support such claims ? Of course not, but we need some form of synthesis of a large body of litterature. (One of my papers was processed in a few recent meta analyses about car fuel consumption elasticities !) So my opinion is that the credibility of econometrics is not much better, however econometrics is by construction apt to deliver uncertainty estimations by confidence intervals. But econometrics can deliver good and useful results on a rather narrow spectrum of contexts. To be a bit too fast maybe, it can only do that with statisticals tools resting on means and well behaved distributions. When your data drift away from those contexts its usefulness decreases rapidely I fear.

Second, there are other ways to compute uncertainty without econometrics. Here I must mention the case of climate simulation models encountered in my field of climate change modelling. Climatologists use huge climate simulation models fed with a very detailed set of documented hypotheses. The models are runned on super computers (because the computations cannot be distributed but the results analysis can hopefully !) and even in one run the results are processed by statistical analysis on variables of interest: temperature and rain fields, surface ocean temperature, ressure and winds fields, etc. Climatologists find no problem to do that, and in fact it the only way to understand what happens ! For the IPCC reports, hundred of scenarios on a few dozen of models are averaged. Those are linked to economic models, one of which my own lab contributes (IMACLIM model). In economic modelling our macroeconomic model IMACLIM is runned many hundred times and results are summarized to provide uncertainty: there are no econometrics, because the model is a special kind of CGE but by controlling the framework of scenarios we can discuss the credit of our results in the Energy Modelling Forum, all is done the way Carlos refer to with the additional difficulty that models discussed come from different theoretical frameworks, some being engineering while other are more economic and some are hybrid (our own IMACLIM model is hybrid). So, on my point of view, there is nothing wrong to perform just that with a DSGE model: run it on the space of your scenarios, sum up and discuss within the scenarios you designed but never forget their limits. What you are doing here is prospective modelling as we call it in climate change science a field where it is very useful and extensive.

Well, I think i am getting too long. I must admit the litterature on econometrics and calibration is scant but I found a few interesting papers I can show you want. I also know of a few interesting papers from French CGE teams that are interesting too.

I wanted to speak about other problems like the one model one dataset syndrome, and the tractability syndrome and the algorithmic syndrome... But as the Che said, that is another story...

All the best. Sincerely.

Franck.

@ Franck and Carlos, you might be wondering I never responded to your arguements. I will do. At appropriate time. I am just waiting for other submission from others scholars before I let the cat of the bag. Carlos statement is not the true position. Not even close.

Thank you Franck! I supposed we both were waiting to some response by our friend Olatunji. In general, I am in agreement with many of your points, Franck. However, please allow me to add some remarks to them. First, the maintained assumptions that were never discussed on those 1938 days are kind of everyday tasks for today’s econometricians, in particular those iid from a known distribution (usually Gaussian), without mentioning the development of robust econometrics. Of course, those meta analyses are always very welcome today (just as in the past), but both uncertainty of the estimates and their confidence intervals are still the key by-products of econometrics. My point is that econometrics’ credibility has been improving quite a lot during the last 80 years.

Second, there are alternatives to compute the uncertainty behind our models’ estimates and forecasts. You provide a nice examples considering the usefulness of well-documented scenario simulations with Climatologists’ models as well as your own hybrid IMACLIM model. I can only add a small comment to the extensive practice you describe: by assigning probabilities to the more comprenhensive scenarios across models it may be posible to gauge the balance of risks as these can be related to some specific assumptions (like global, regional or national ‘clean economic growth’, for instance).

Finally, it will be wonderful to have some of your views about whether there must be a balance between the number of parameters to be calibrated and the number of parameters to be estimated (conditional to the calibrated ones), with some references to the papers you consider ‘the key ones’ and to the French teams working on CGE models. Please do say something about those three syndromes as they should be related to your experience with the IMACLIM model’s simulations. Best regards, Carlos.

@ Carlos and Franck, still waiting for more contributions to the debate. Your answers are available.