I am trying to crowdsource mean opinion scores (MOS) for a speech synthesis model. As mos is a subjective metric I'm trying my best to avoid subjective bias.
So far my plan is to have a GitHub page containing the samples and ask different users (from social media) to give an opinion (1-5) for the samples through a web interface. From that, I will calculate the mean opinion score after doing statistical post-processing (following this paper https://github.com/Netflix/sureal/blob/master/resource/doc/dcc17v3.pdf).
Recently, many papers mention Amazon Turk for this type of task, but for my task it's an overkill. Do you have any suggestions regarding this?
Hi.,
A Mean Opinion Score is a measure of quality of an event or experience. ... Score can be employed anywhere human subjective experience and opinion is useful. ... with Twilio, we have written up a number of client best practices and can route your communications traffic over the internet or through our high ...
Kindly Refer this link:
https://arxiv.org/pdf/1511.02919
Conference Paper CROWDMOS: An approach for crowdsourcing mean opinion score studies
- Badges
- Science topic
- Similar topics
- Social Science
- Media
- Social Media