There are three kinds of model to learn phoneme embedding as follow: PPMI+SDV, word2vec dan RNN Encoder-Decoder. My suggestion, you can use RNN Encoder-Decoder to build a phoneme embedding combine with CNN to recognize the phonemes.

So is the embedding vector the output of " RNN Encoder-Decoder " , or the output of a middle layer?

thank you

Asep Fajar Firmansyah

Yes, you can use RNN Encoder-Decoder to produce the phoneme embeddings, it means RNN maps each the phoneme to embedding space.