I have read it in several articles that a true validation of a hydrologic model is not possible because a lot of parameters need adjustments and we never know the perfect combination for sure. If this is the case, then how can we be sure that a hydrologic simulation is correct? How much I can rely on my model?
And how to measure the uncertainty of a hydrologic model?
This is a very complex subject. Validation of hydrological models is often incomplete due to data limitations. Most model validations in the literature are done against a single variable (i.e. stream discharge), but other important components of the hydrologic cycle (e.g. soil moisture status, water content in the snow pack, groundwater baseflow) are usually not considered due to lack of data. Some of these variables are inherently difficult of validate due to challenges in scale (e.g. point measurements of soil moisture). These limitations, however, should not hinder the application of hydrological models, which can be used for several purposes (e.g. informative models, predictive models, diagnostic models) and aid the advance of hydrology. One branch of hydrology actually deals with predictions in ungauged basins, where no or minimum monitoring is available. At least, model exercises should inform about important variables and direct experimental/monitoring efforts. I would suggest you the book Rainfall-Runoff Modelling: The Primer (Beven, 2011). It covers many important aspects of hydrological modelling and discusses model selection, parameterization, and model uncertainty. Beven, K.J., 2011. Rainfall-Runoff Modelling: The Primer. Wiley, West Sussex, UK.
as I know hydrological parameters itself unpredictable,
but with experience of inter-relationship of different parameters we can predict with the help of hydrological models
they will certainly not replace actual things but definitely give correct idea about occurrence
secondly, there are uncertainty measuring coefficients ,with which we can rely on models
This is a very complex subject. Validation of hydrological models is often incomplete due to data limitations. Most model validations in the literature are done against a single variable (i.e. stream discharge), but other important components of the hydrologic cycle (e.g. soil moisture status, water content in the snow pack, groundwater baseflow) are usually not considered due to lack of data. Some of these variables are inherently difficult of validate due to challenges in scale (e.g. point measurements of soil moisture). These limitations, however, should not hinder the application of hydrological models, which can be used for several purposes (e.g. informative models, predictive models, diagnostic models) and aid the advance of hydrology. One branch of hydrology actually deals with predictions in ungauged basins, where no or minimum monitoring is available. At least, model exercises should inform about important variables and direct experimental/monitoring efforts. I would suggest you the book Rainfall-Runoff Modelling: The Primer (Beven, 2011). It covers many important aspects of hydrological modelling and discusses model selection, parameterization, and model uncertainty. Beven, K.J., 2011. Rainfall-Runoff Modelling: The Primer. Wiley, West Sussex, UK.
Well, there are number of statistical techniques which can be used to analyze the performance of a model. Example of such are the Nash Sutcliffe, R2, PBIAS etc
In addition to the valuable explanation of Marcos, I would like to add that also validation of hydrological models using (satellite-)observed actual evapotranspiration (AET) can be a useful way to build confidence in hydrological models. For instance, a hydrological model is calibrated using streamflow data and validated using AET data (or vice versa). Also, the model can be calibrated using both streamflow and AET data simultaneously and validated using the same data for another time period.
When validating (hydrological) models, it can be very useful to consider the framework proposed by Vit Klemes in his paper in Hydrological Sciences Journal in 1986: 'Operational testing of hydrological simulation models'. In this framework there are two dimensions (time and space) for model validation. Validation in time can be stationary (split-sample test) or non-stationary (differential split-sample test), validation in space can be done using one basin or multiple basins (proxy-basin test). The most extensive way of hydrological model validation is the non-stationary test for multiple basins: the proxy-basin differential split-sample test. In that case the model is for instance calibrated for catchment A for a wet period and validated for catchment B for a dry period.
Model evaluation should not be limited to measuring how well the model predictions match historically observed data. As explained by others this is inherently limited due to lack of suitable measurements on all model outputs (e.g. soil moisture). There are several other important considerations - 1) One must carefully and comprehensively evaluate the fidelity of the model is representing the hydrologic system (e.g., a watershed) of interest; 2) The space/time-scales of model runs and the space/time-scales at which hydrologic processes manifest in the system is of particular concern. 3) How many parameters were tweeked during calibration and over what ranges is another important consideration. Model Identifiability analysis is used to evaluate how many model parameters can be estimated (down to a measurement error) from available data. Usually this number is fairly low even when there are a lot of data due to persistence of hydrologic phenomena. As they say, one can fit an elephant if there are 5 or more calibration parameters. 4) The purpose of the model must guide how the calibration is to be carried out and must be defined prior to model calibration and evaluation. 5) Sensitive model parameters must be identified and how well these model parameters are represented either using measured data or are properly constrained to acceptable ranges must be looked into. 6) Comparison must be limited to aggregated values (e.g., yearly totals) in coarsely discretized models. 7) Even when calibration is properly done, models tend to have greater predictive strengths over shorter time-frames than longer time-frames as the system over short time scales is more similar to the one which it was calibrated for. Therefore, greater trust can be placed on short-term/scale predictions than long-term dynamics. 8) Model results should never be taken on face value. Proper uncertainty bounds must always be placed on the predictions. 9) Models whose formulations and limitations are properly documented are more trustworthy than those that are essentially black-boxes regardless of their predictive abilities. Transparency is the key 10) Post-audits or revisiting the model after some time (when new data is available for evaluation) is another important factor to improve our trust and refine the model as necessary.
There is a vast literature on model calibration, verification and uncertainty. The following papers provide good starting points in addition to the primer by Beven that was provided by Marcos:
Oreskes, N., Shrader-Frechette, K. and Belitz, K., 1994. Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263(5147), pp.641-646.
Jakeman, A.J. and Hornberger, G.M., 1993. How much complexity is warranted in a rainfall‐runoff model?. Water resources research, 29(8), pp.2637-2649.
Jakeman, A.J., Letcher, R.A. and Norton, J.P., 2006. Ten iterative steps in development and evaluation of environmental models. Environmental Modelling & Software, 21(5), pp.602-614.
As several colleagues have said earlier, the accuracy of model-based forecasting depends on three quite different things: (a) the relevance of the model (the algorithms used must adequately reflect the relationships between the phenomena), (b) the volume of information that has been taken into account (which helped to calibrate the model) and (c) the completeness of this information (for example, if we want to simulate exceptional events, such as the centennial flood, we must have measurements during a few high floods).
Mr. Nurruzaman, it seems to me that you're asking the question of whether there exist relevant models (and the answer is yes: there are software tools whose predictive quality has been well confirmed in many river basins). Unfortunately, it is not enough to have a relevant model. There is still a need for enough data (in numbers and accuracy) to calibrate this model.
Thank you Dr. Collignon and everyone else in this discussion. Actually, I have worked only on one urban drainage project where I was an assistant hydrologic modeler. I used XPSWMM with an extreme shortage of data for calibration and validation in that project. I know that there are other software tools too.
My question was that even if the calibration and validation results show agreement (using NSE, R2, RMSE, etc.) between observed and predicted data, is that enough to accept the result? As there are so many parameters to adjust that there are chances of failing to simulate future conditions accurately.
So, I understand from the valuable inputs of the experts that it will depend basically on the quality of the observed data and the purpose of modeling.
As many other colleagues have said, there are many parameters that cannot be well defined for a whole basin and there are many kind of data that can be observed. So my suggestion is that you may test several hydrologic and pick the one which better fits the observed data you have, and/or want to predict; and the one which the required data are the closest as possible of the data you have available with a good degree of certainty. And as for how much you can trust it and evaluate the its uncertainties the better way is to use statistics....
Most of the things are already said. I just want to add that it might be good if you consider not only stream discharge for your calibration. Models such as the GeoTop also calculate turbulent fluxes. If you have such measurements available, you can make use of them and achieve a more robust calibration (equifinality concept of Beven!).
We are currently trying to publish such a study.
Hope this helps.
I found this interesting paper that may help you to some extent.
1. Article Validation of a watershed model without calibration
- Badges
- Science topic
- Similar topics
- Geoscience
- Hydrology