For the purpose to improve the performance of a baseline HMM-based recognizer, I would like to integrate more external features extracted from speech signals into the recognizer. my question about combining MFCCs features extracted via HTK Toolkit and the new pitch features extracted via Praat script (fused vector will contain both MFCCs and Pitch features frame by frame) ?
Any help or guidance would be appreciated
Though I never dealt with praat pitch extraction, mfcc+pitch are easy to extract in kaldi (see https://github.com/icedwater/kaldi/blob/master/egs/wsj/s5/steps/make_mfcc_pitch.sh).
Kaldi features can also be converted into HTK format if needed.
As for your question, could you precise what is the problem? If your features are synchronized at the frame level, you concatenate mfcc and pitch vectors per each frame.
Hi Mohamed,
If I understand your question well, you need technical assistance. You can create USER type (No. 9) features from concatenated MFCC and pitch.
Hi Mohamed,
In HTK Book at the beginning of section 5.10.1 HTK Format Parameter Files you can find the header format. According to it you can define the structure of your features.
Becareful - pitch features have a strong personal bias - female pitch is substantially higher than male pitch on the average, although the variance by sex is quite large. If you are going to use fused features, you should allow for personal settings in the model.
Dear Mohamed,
I agree with Mr Jordan rian Cohen. Information about pitch is rather removed from MFCCs. Besides of pitch variance by sex, there is also high pitch variance by intonation and emotion. I am not sure if you will obtain any improvement of speech recognition accuracy but if you obtain please let me know.
I have tried mfcc together with pitch feature, and it works well in Chinese ASR, because Chinese language is a strong tonal languange. However, for pitch to be effected, something should be dealt carefully:
1. mfcc frame shift and that of pitch should be the same, so that the total frames are the same
2. for tonal language ASR, the tonal information is rather saved in delta pitch and deltadelta pitch, so the static pitch can be removed;
3. be careful, for speech where pitch is not found (no resonance), do not try to amend it by averaging methods, this is harmful for speech recognition, just set delta pitch and delta delta pitch to zero
That's my experience to make these two features work.
I agree with Mu Su. Pitch feature can work well for tonal languages like Chinese, Vietnamese but also for some Afroasiatic languages like Chadic, Omotic. Pitch distinguishes the meaning of many words in these languages. As I know, Arabic language is not tonal so in my opinion pitch feature will not work well for this language.
Best reagards
Robert Wielgat
- Badges
- Science topic