I have tried some digital tools, but none of them provides a specific level/score. I need that tool for a pre-test and post-test quasi-experimental design where I implement a new program that is expected to improve pronunciation.
The best tool I have seen so far is ELSA Speak. However, ELSA Speak doesn't give the option of taking the test for many subjects.
Note: I am looking for a digital and objective tool, not raters or likert scale.
Pronunciation intelligibility is subjective, and it is hard to evaluate with perfect accuracy.
For such situations, I adapt the idea behind the 6 Plus 1 Traits model.
Attached is an oral proficiency rubric I have used. It is not ONLY intelligibility, but it shows how you can break down one of the English skills into component parts and evaluate each to achieve a total score.
Hi, there are two large distinctions you can make for this: human-rated and AI-rated.
For human rating, you need several raters. Generally if you're trying to rate intelligibility, you have raters try to transcribe what the person said and see what percent they get right. You CAN have them make decisions on pronunciation, but that is really more of a pronunciation score than an intelligibility score (i.e., if they can understand it well enough to transcribe it, it must have been intelligible)
For AI-rating, you can use any of a number of standard ASR (automatic speech recognition) softwares that are automatically built into every computer and smartphone. If you have not collected the data yet, I highly recommend this route as it is cheaper, easier, rather accurate (see paper below), and has 100% reliability in that the computer will always give the same score (humans are falliable in this regard). If you have already collected audio data, the paper below also recommends a way that you can still use free ASR tools to get transcriptions.
Either way, the paper also has some references to otehr papers that describe the human model in detail. Hope this is helpful
Spring, R. (2020). Using Multimedia Tools to Objectively Rate the Pronunciation of L1 Japanese EFL Learners. ATEM Journal: Teaching English through Multimedia, 25, 113-124. Article Using Multimedia Tools to Objectively Rate the Pronunciation...
This is a very interesting research project. The first question that came into my mind was 'Intelligibility for whom?' For example, some speakers of Singaporean English may be perfectly intelligible to other Singaporeans, but much less intelligible to visitors from Europe, North America or Australia.
Regarding automatic speech recognition software, I suspect that many of these may use General American as the yardstick. If so, software of this kind may give a lower intelligibility score to those who have learned, for example, British or Australian English.
- Badges
- Science topic
- Similar topics
- Psychology