I want to run 5 different models to estimate stream flow. In order to optimize the characteristics of these models I use Taguchi method. So I have to run different models according to the Taguchi orthogonal array. Therefore, I have different models with different inputs and different data lengths. For example the first test is: using rainfall and temperature in ANFIS model with 2 year data length, while the second test is: using rainfall, temperature and discharge for previous day in SVR model with 10 year data length. So, the inputs, Data length and model type is changing in these tests. What is the best performance evaluation criterion for this study? NRMSE can be a good criterion because it normalizes the RMSE and in this way, it removes the effect of data range.
Now, I want to know if there is any better solution for this problem.
I think if you applied a GLM you can classified your models according to their AICc. Model with the lowest AICc is the best one.
good luck
If I am understanding you correctly, I fear you are confusing models with experiments. The Taguchi method is for selecting the number of experiments needed. Models, and model runs, are not experiments. If one runs a model with the same input, one should get the same results (note that some sort of randomized parameter would be different input, it is completely within the modeler's control). The Taguchi method is designed to deal with the fact that one can run the same experiment multiple times and get differing results, due to factors beyond the experimenter's control. Thus, I fail to see how it might be applicable. Perhaps I am missing some detail that is intended to be implicit, but needs to be explicit for me to pick-up on it.
Hello Vahid--the criteria may depend in part on the modeling objectives. For example, is accurate modeling of low flows more important than accurate modeling of high (flood) flows?
- Badges
- Science topic