I used to compare models based on their DIC by dividing the data into training and test sets. Recently, I am working with spatial statistics (Library INLA) in R, and I think it might be more valid to do not split the data (lots of information may be ignored). Can you suggest any paper that uses DIC in this manner?
Hi Mr. Farkhondehmaal
Please state your topic, field or contex of research.
Ali Reza Hashemi
Hi,
I am using Bayesina regression with INLA for a spatiotemporal dataset. I want to compare the result with the result of GLM (and perceptron).
A. If you later want to know if the model is "valid" or "works," you need to have a hold out sample. Generally, the sample should be random -- not selected because of the danger of even unconscious preconceived notions of bias.
B. If you are simply trying to understand what is going on, or just get a better "fit" on historically data, then you do not have to have a holdout sample -- BUT then do not come to the conclusion you have a good model that "explains." You may end up with better predictions but you really do not understand what is going on.
C. Try this. After you do A, separately run on just hold out sample.
D. After do C, separately run full sample.
Study the different results with a view towards understanding -- NOT for the purposes of teasing out what gives the most statistical "significance."
You may find a reading of
Article Economic models: perspectives for the professional
helpful.
Good thoughts and good luck,
John
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training