Hello I work with Convolutional Neural Network and LSTM in speech emotion recognition, in my result I see that CNN has shown better performance than the traditional LSTM in my speech recognition .
Why this?
Normaly LSTM should better in Speech recognition as I use sequential data.
Thanks
Abdoulaye Diallo ,
Could you give more information on the dataset?
Is it continuous speech? How long are the utterances?
are there multiple speakers in a single recording?
In addition, what preprocessing did you use for the speech recognition and the speech emotion recognition?
Hi,
Which kind of performance? Accuracy? Processing time? What is the database and what is the database size? Is it a common benchmark used by other researchers? What are the specifications of both networks?
Regards,
Ali.
Hello for your answers. It’s common databases used for research. One utterance per files (about 1 second).
Datasets are: RAVDESS and Toronto and Berlin database. First test only in ravdess databases cnn perform better than lstm. All combine also cnn perform better.
Yes performance on accuracy.
Abdoulaye Diallo
I took a look at the RAVDESS Audio_speech_actors_01_24.zip and from the samples that I heard I would say that they last from 1-2.5 seconds at 48 kHz. The question that I have for you is: how are you segmenting the audio file to feed it to the network? for example for speech recognition you can do ~30 ms for phoneme recognition, ~130 ms (syllable) and ~15 ms for phonological features. The use (and mixture ) of each can alter the accuracy performance that you get. To put it in another way, depending on the length of the sequence will be the information that you pass to the network (especially for LSTM's).
In addition, I would be careful and check the literature on how to make baseline configuration on which both architectures can be compared on equal footing.
Thanks Arturo Geigel I I've extracted the features using two different librairie (Parselmouth(PRAAT)) and (Librosa) then I concatenate these features in one matrix (I s this a good idea?) to feed it in CNN and LSTM. Windows length is 25ms.
Feature extracted in Parselmouth are: f0, hnr, jitter(s) and shimmer(s).
Librosa: mel, mfcc and chroma.
I'm right.
I have not dealt with emotion recognition but based on my experience with speech recognition, I would suggest
1. Look at the literature for a baseline which you can reproduce and then base the performance comparison based on the particular differences
2. Look at the ranges of each of the features ( you have several) to see if normalization is warranted
3. If you do not know the relationship between features and emotion, try them individually and then combine them
4 some features can have better oucomes from particular encodings (e.g. conversion from continuous to discrete, etc.)
5. I woul try different lengths, not just 25 ms
Another question: what is the sliding window difference?
Abdoulaye Diallo ,
just to clear my last question that was kind of ambiguous. What is the sliding window offset? If the sliding window offset is too high you might loose important information in the sequence
Regards
Sliding windows offset is it the size frame windows? I set it to 25ms. But only for mfcc the others features I leave parameters default.
The offset is the displacement from sample to sample. If you measure your sliding window from the beginning of your sample to your next sample, a sliding window offset of 25ms means there is no overlap between them. Lets take that you slice a sequence of phonemes every 25ms with a sliding window offset of 25ms, what guarantee do you have of obtaining all phonemes within those frames?That is why you do it with some offset x
Thanks @Arthuro, I’v to decrease the sliding windows size. Thanks for this comment.
- Badges
- Science topic