In Mathematical and computational models, when you estimate and assume the values of some parameters, there is always a strong tendency that your model will not predict an accurate result or such model can even generate some errors.
How do you correct the errors generated by your model especially during a predictive process? and How do you ensure that your computer model generates an accurate result?
Your answers will be highly appreciated
Regards
Try looking at the work of Kennedy & O'Hagan on Bayesian Calibration of Computer Experiments, or a short review I wrote: http://eprints.soton.ac.uk/146605/1/Black-box_calibration_for_complex_systems_simulation.pdf . The idea in these works is to model unknown effects with what is, essentially, a statistical model of the difference between your model and your data.
It would be very worthwhile reading Oberkampf's Verification and Validation in Scientific Computing.
A reliable use of models for a simulation of a process or for identification always need a procedure for calibration and verification. In the case of a model that simulates a process, the end values of the model should fit with actual data. In the case of an identification model, the developer should start with identifying entities (e.g. plant species) that are already known. These are only a few examples of situations that are presumably treated in more detail in the references mentioned in the other answers. Finally, it is very important to rely on a model only in the domain where it is calibrated and tested for. Extrapolation is very risky and need to be tested first.
References:
Calibration of simulation models: http://www.pressesagro.be/base/text/v15ns1/53.pdf ; check figure 2 for calibration of two alternative models with actual datapoints.
Parameters for evaluation of identification models, including domain effects: http://dx.doi.org/10.5772/51362 .
You may use an intervals for variable values or sampling the possible values.
The validation of the model is required the experimental data and the use this data to correct the output of the model during the process.
The only possible way to check if a model is correct through comparison it with real data and verification of errors in relation to this.
.
Numerical methods will the quality of the numerical scheme (important factor), but say nothing about the quality of physical prediction. Use of inappropriate assumptions, variables in the absence of simulations, for example, lead to inconsistencies in the mesh refinement solutions, better not solve mathematical schemes.
.
Your question suggests you did a decent effort to learn a model first, but then observe errors during "a predictive process", i.e. you make a number of predictions and you get feedback releaving that some prediction were not accurate.
First , if indeed you made a decent effort to learn a model, it is not possible to avoid errors during prediction, since in most cases the available training data just does not allow for learning a perfect model (or a model for which the probability of a large error rate is high).
Second, once you learned the model and observe errors, you can learn a new model taking into account the new training instances (those which revealed the errors of the first model), hence getting an improved model also fitting the new data. Repeating this will imporve the model as the size of your training set grows.
If re-training your model is too expensive, there is a large literature on incremental learning, i.e. update your model with a minimum of effort using just the newly arrived data.
If your learning system is not sufficiently strong to learn the model at once (e.g. due to limitations in the hypothesis space or search space), there are methods such as "gradient boosting" which learn the difference between the predictions of a model and the actual values, so that in a next iteration you can use as prediction the prediction of the original model plus the predicted difference between actual value and predicted value. This 'gradient boosting' step can be repeated iteratively.
As I read once on NAFEMS' website: "The results of a simulation must be assumed to be wrong, unless there is evidence that the opposite is the case".
It's a matter of a VV&A process related to your (the model's) intended purpose. I wrote something on this that might be too general for your case, but some of the references are more specific:
http://kau.diva-portal.org/smash/get/diva2:555764/FULLTEXT02
Try looking at the work of Kennedy & O'Hagan on Bayesian Calibration of Computer Experiments, or a short review I wrote: http://eprints.soton.ac.uk/146605/1/Black-box_calibration_for_complex_systems_simulation.pdf . The idea in these works is to model unknown effects with what is, essentially, a statistical model of the difference between your model and your data.
One interesting approach is that of parameter’s estimation in real-time by the recursive least squares algorithm, which convergence after disturbances experienced at the observed system can be improved by application of a 'forgetting factor'. For a possible example of application you can see that of the energetic optimization and adaptive control of aerobic fermenters, reported in the following investigation (in Portuguese):
C.M.G.A. Queirós, "Controlo do Oxigénio Dissolvido em Fermentadores para Minimização de Energia Consumida", Master Thesis, Instituto Superior Técnico, Universidade Técnica de Lisboa, Lisboa, 1997:
https://www.researchgate.net/publication/233799574_Controlo_do_Oxignio_Dissolvido_em_Fermentadores_para_Minimizao_de_Energia_Consumida?ev=prf_pub
Thesis Controlo do Oxigénio Dissolvido em Fermentadores para Minimi...
It would be useful to know the type of model you have or the application you have in mind. In any case, one option is to use the new data available to recalculate the values of the parameters that define your model (assuming for example that you have a model based on first principle equations or at least a model with fixed parameters). This is very typical in adaptive control applications.
- Badges
- Science topic
- Similar topics
- Engineering
- Modeling