I attended a training to learn Hydrological Modelling Using SWAT, which is a widely used model, back in 2018.
We discussed about the details regarding calibration and validation of a model and were told that 7:3 is like a golden ratio for calibration and validation. Basically, if you have 10 years of observed data, you calibrate the model for 7 years and validate the same for 3 years. As a beginner in hydrological modelling back then, I engrained this piece of information in my brain.
But as I keep reading through various research materials, I have realised that there is no obvious pattern or thumb rule for this. I recently read a paper which calibrated a model for 1 year worth of observed data while validating the same for 5 years.
So I'd like to know your approach when you work with a hydrological model and the factors which influence your decisions on how much to calibrate or validate.
Hello Harsh Upadhyay
Mostly we utilize the longer period of observed data for calibration purposes as I have observed in many of the literature. During calibration, we are about to set the best-fitted parameters and these calibrated parameters are kept the same during validation, so the calibration is complex and required more understanding of the time series of data, once the Calibration part is over one can easily perform validation. You can also divide your calibration and validation period based on the mean values of data, For example, if 2005-2013 data sets we have, we can select 2005-2009 for calibration and 2010 to 2013 for validation ., Also based on their mean values u can choose the time periods, ex- mean of 2005-2009 data sets can be nearer to mean of 2010-2013 data sets.
I observed that if we select the longer period for calibration, the model performance was better with NSE, R2, and other objective functions in SWAT-CUP..
A recent paper suggests that "calibrating to older data and then validating models on newer data produces inferior model testing period performance in every single analysis conducted and should be avoided. Calibrating to the full available data and skipping model validation entirely is the most robust split-sample decision."
The authors here conducted multiple calibration-validation experiments using calibration sub-periods of different lengths to train hydrologic models.
It can be found here: doi.org/10.1029/2021WR031523
Calibration and validation is the same process for different strings of values , there is no proper ratios for it .In calibration ,the variables and parameters can be adjusted to obtain a good correlation between the observed data from model and actual data. In validation the same set of parameters will be used for obtaining a correlation between these values as well as simulation process. Mostly calibration can be done over more period of years ,it makes the model more related with the actual scenario and gives the good simulation results,which tend to replicate the natural scenario.
Dear Harsh Upadhyay,
Interesting question. In addition to the very useful comments and replies from colleagues, I would like to add some observations and comments. The calibration and validation strategy depends on different factors, such as the purpose of the modelling and possible final application, the availability of data and the hydroclimatological conditions in the studied catchment(s).
The modelling purpose can be very general (e.g. simulation of hydrological regime and behaviour of a catchment), specific (e.g. low flows) or supporting water systems and infrastructure design (extreme, non-observed conditions) and future impacts (e.g. due to climate change). Calibration and validation strategies for general purposes can be more straightforward by using the split-sample test and dividing the time series in two parts, indeed generally somewhat more data for calibration (e.g. 2/3) than for validation. Specific purposes and support for design and impact assessment require more tailor-made approaches showing the robustness of the model. For instance, for flood design purposes one would like to show that the calibrated model can also be used for more extreme conditions than the calibration conditions. Hence, one might select the calibration and validation periods in such a way that the validation period includes more extreme events than the calibration period. In this and similar cases a so-called differential split-sample test is therefore more appropriate. Similar reasoning applies for climate change impacts studies, where you might want to show that your model can also be used for changed climatic conditions. This can be (partly) achieved by selecting a validation period with (hydro)climatological conditions more similar to the expected changed climatic conditions than the calibration period.
Data avaliability obviously determines what is possible in the calibration and validation. When for instance streamflow data are not sufficiently available, you might look for alternative data regarding other output variables such as actual evaporation (e.g. based on satellite information) or state variables (e.g. soil moisture, snow and groundwater). Different calibration strategies for multiple variables (and objectives) are available in the literature, see for instance Article Multi-variable calibration of a semi-distributed hydrologica...
.Recommended splits for calibration and validation periods (e.g. 50-50 or 70-30) will also depend on the hydroclimatological conditions in the study catchment (e.g. the occurrence of dry and wet periods) and the hydrological model used. The systematic study mentioned by Mehmet Cüneyd Demirel provides a nice example of the detemination of the optimal data length, but results obviously depend on the hydroclimatological conditions, the catchment characteristics and the model used.
Good luck,
Martijn
- Badges
- Science topic
- Similar topics
- Geoscience
- Atmospheric Sciences
- Climate Science
- Climate Change