Many indicators are used to evaluate topic modeling results (Log-Likelihood, AIC, perplexity) but in my experience (using r topicmodels) results don't converge. What can be the explanation? Which indicator is the more performant?
It really depends on the number of samples. From my humble experience, I suggest KICc or HQc instead of AIC.
In my experience, often due to a low size dataset. Can you tell more about your data ?
my data consists in 500 Web of Science records (title, article and keywords). I've tested many indicators, which give very different results for LDA: around 8 topics for AIC or KIC but 50 for Log Likelihood.
Yes this is a small dataset. One way to get another estimate would be to run hdp on it. What's your final objective ? This might help going around the problem.
HDP seems to be one solution, but to my mind there is no implementation under R. My aim is to identify major topics, and their evolution, in a small research field.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling