Say I have two audio files, one with a specific word spoken by a native speaker, the other with the same word spoken by a learner. When I say word, I probably need to make this phoneme.
Using the native speaker - file, I want to detect the differences to the learner's file. Obviously gender difference, pitch, speed, and the like should be ignored.
To make this clearer: the initial phonemes in Jack, chin, gin, etc. are often pronounced wrongly by non-native speakers. Could an algorithm detect this? Or would I classify Jack pronounced by 10 native speakers pronounce Jack, and then feed the learner's Jack?
Is this possible at all? Would TensorFlow be a tool to try?
Thank you for your ideas.
Dear Leopold Hamminger,
Look the link, maybe helpful.
https://www.tensorflow.org/tutorials/sequences/audio_recognition
https://medium.com/iotforall/sound-classification-with-tensorflow-8209bdb03dfb
Regards, Shafagat
Based on my understanding, when you're thinking about ML, you taught the machine what has happened in the past (or historical data), and then when new data comes it automatically classifies your new data into a given category.
Since your problem is only to compare between to any audio files and to detect the difference. I dont think ML would be the first choice (unless you have existing data that clearly classify and indicate the difference).
I think one easier approach is to read the two audio files as a matrix of numbers and you compare the similarity between these matrices in terms of pitch, speed..
One of my lab's mate is currently working on detecting the similarity between the cover song and its original one. The method I think would be the same.
Vinh T. Nguyen you are absolutely right, converting the audio files into matrices would be another way to go about it, and I thought about this first. However, I thought it would be a daunting task to isolate a particular phoneme.
It doesn't mean that an ML solution would be any easier. Again you are right in pointing out that existing data is needed, and I thought of collecting some from different native speakers. My hope is that existing data could be used to "machine learn" the phoneme j in Jack, for example, likewise the phonemes that follow. If I then present a learner's 'Jack' it would either recognize it as such (if pronounced like the native speakers did), or not.
Which may mean that a learner's 'Jack', that sounds like 'Chack' could possibly also be machine-learned... This needs some more thinking and any additional ideas are very much welcome.
Meanwhile again thanks a lot, Shafagat Mahmudova and Vinh T. Nguyen
I think the approach mentioned in the paper maybe a right solution for your problem. https://arxiv.org/ftp/arxiv/papers/1601/1601.01577.pdf?fbclid=IwAR017Cf6SqvramF1BO0ZjLxXnBmTDqnivkFKbZw-2XEGZDhE5r2qF7HiC7s
They conducted the experiments to identify gender from audio files using a set of features known as MFCC (Mel-frequency cepstrum coeffcients). I think the issue is that you need to gather a proper amount of training data to perform the classification.
Nhat Le excellent literature link, it appears that Support Vector Machines are superior as classifiers. Very useful also the reference to Vapnik's Statistical Learning. Thank you!
- Badges
- Science topic