I trained a 3-layer LSTM network to extract d-vector embedding using keras. I extracted MFCC features from TIMIT dataset as input to the model, and defined a custom loss function (i.e. GE2E loss).
After less than 213444 batches I got zero loss (on train and dev sets), however, when I use the model to predict d-vecotrs (even using input form training set) I keep having nearly the same output (i.e. The cosine similarity between any output vectors is 0.99999xxx).
I double checked the code and the loss function implementation, it seems to be correct.
Any idea what might cause such problem?
Seems like you may have over-fitted the model with data. Try using regularization, it may help to reduce the over fitting.
I think that you train the model a lot. So the model is over trained. this causes that train loss is near zero but test loss on a different data will be large. because the model is very fit only for train data not any other new data set.
you can use a validation data set to cut training before over fitting.
also you can use clipping technique.
I used the TRAINING set it self to validate the UNEXPECTED fast convergence of loss value, and I found that the trained model failed to predict correct output even for the DATA IT TRAINED ON, so I do not think it is over-fitting problem.
Md Fahim Sikder, Mahsa Afsharizadeh
Alhasan Alkhaddour This problem may be due to the "batch normalization". When you are evaluating your model, you should disable batch normalization. Use "model.eval()" when you want to evaluate the model (so batch normalization will be disabled) and use "model.train()" again when you want train the model.
- Badges
- Science topic