I have trained Word embedding using a "clean" corpus in fastText and and I want to compare the quality of the Word embedding obtained against the word embedding from the pre-trained multi-lingual embedding in BERT which I perceive(discovered) to be trained on a "very-noisy" corpus(wiki).
Any Suggestions or Ideas on how to go about evaluating/comparing the performance would be appreciated.
As a first step, take the same model deep enough to approximate your data. and just compare the metrics that are important to you(F1, AUC ...) with both variants of the Word embedding .
Thanks Mark Panenko, to get the metrics I am required to perform classification tasks with the models right??
It is best to test for your task. If you are doing text classification, I would recommend starting with an AUC assessment. If the entity recognition is non-zero F1.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training