Looking at a lot of methods, I want to use the degree of confusion to compare the results of model generation and human results. I plan to use the LSTM model to do the training set with human text and the test set with different machine text. Compare the perplexity in the tests. The Internet says this thing is to evaluate the quality of language models, I want to compare the difference between man and machine. I don't know if this will work.
Chen Yijia Perplexity is the multiplicative inverse of the language model's probability assigned to the test set, normalized by the number of words in the test set. A language model is more accurate if it can anticipate unknown words from the test set, i.e., the P(a sentence from a test set) is the highest.
Have a look: https://towardsdatascience.com/how-to-evaluate-text-generation-models-metrics-for-automatic-evaluation-of-nlp-models-e1c251b04ec1
- Badges
- Science topic
- Similar topics
- Social Science
- Media
- Journalism