I'm using "out of sample maximum likelihood" to assess model fit from several candidate prediction models generated in SAS 9.3 using PROC GLIMMIX or PROC GENMOD (binary distribution). "Out of sample maximum likelihood" is calculated as: ∑(measured * log(predicted) + (1-measured)*log(1-predicted)). Has anyone used this method to compare similar models, or can you recommend other statistics to compare models and assess model fit? I'm seeking method alternatives or references for application of "out of sample maximum likelihood" to assess model fit.
I am not experienced, but I found this:
http://www4.stat.ncsu.edu/~boos/papers/ios5.pdf
to be an interesting presentation.
We proposed (among other approaches) to use cross-validated (out-of-sample) maximum likelihood estimates to decide on whether one should include a further variable in a structural equation model tree. However, we base the decision on a heuristic and have no formal treatment beyond that, so I guess it's not too valuable for your problem but this may something interesting for you anyhow:
Brandmaier, A. M., Oertzen, T. v., McArdle, J. J., & Lindenberger, U. (2013). Structural equation model trees. Psychological Methods, 18, 71-86. doi: 10.1037/a0030001
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers