Hi, everybody.
I collected a list of 700 example sentences from domain specialists. And used this list as a basis for generating new 9 k sentences using a generative language model. Now, I am looking for methods for evaluating the quality of my generated corpus.
I have trained an n-gram language model using the generated corpus and measured the model perplexity in the specialists' sentences. I have good results on it, but I think I can evaluate it using other methods.
If you have any related research, please let me know.
Thank you in advance.
Dear Jayr Pereira ,
There are to ways to evaluat: human-bias or cross-validation and F1 score.In human-bias you need at least 2 experts to evaluate the results of your system manually and then you show their evaluation as evidence.
In second way, you need to compare your results with any other system results using the same dataset and the same gold standards and metrics (e.g., precision, and F1).
Hoping this helps!
Dear Jayr Pereira,
I have a similar question in the evaluation.I have a study in text generation, but the challenage is how to evaluate it objectively.
I am interested in the n-gram，can you give the github address about your n-gram?
Thank you very much.
Hello, Chen Yijia ,
I actually use the NLTK package for language model: https://www.nltk.org/api/nltk.lm.html
- Badges
- Science topic
- Similar topics
- Cognitive Science
- Thinking