1) The idea is to recognize context out of a conversation about a topic. For instance if two people are talking (assuming without overlapping their voices), it should be able to differentiate between the two voices either by only differentiating between two voices or by differentiating by recognizing the users(which would require training the voices therefore I would work on it once the rest of the project is done).
2) After differentiating the contents of the conversation based on who spoke what, I would further analyze the contents.
I think your problem formulation is a little unclear...
1) It's a bit confusing when you say "and convert it into text". If you want to actually do speech-to-text, it is not quite related to speaker recognition...
2) In title you say recognize the user. But then say that you want to find where 2 or more people speak. To find a target speaker in the utterance, you can start with conventional GMM or i-vector based approaches (have a look at https://pypi.python.org/pypi/bob.spear/1.1.2 to start with). However, classifying overlapping speech (from 2 or more users) needs different techniques and in fact a bit more complicated
Feel free to provide more details and reformulate your question to get more relevant feedback
Thank you for pointing those out.
I'll reformat it but meanwhile, I'll make it more clear here...
1) The idea is to recognize context out of a conversation about a topic. For instance if two people are talking (assuming without overlapping their voices), it should be able to differentiate between the two voices either by only differentiating between two voices or by differentiating by recognizing the users(which would require training the voices therefore I would work on it once the rest of the project is done).
2) After differentiating the contents of the conversation based on who spoke what, I would further analyze the contents.
OK, now it is clear. Technically, what you want to do is called speaker diarization followed by speech recognition with further content analysis. It consists of several steps: voice activity detection, finding homogeneous segments, speaker clustering and re-alignment of speech boundaries.
First I have to say that both diarization and recognition are challenging, especially if your data are conversational. Even worse if these are telephone quality recordings. Prepare yourself to quite many errors on various stages. In practice, for further NLP processing you will likely need to work with N-best transcription hypotheses or with word lattices to mitigate the errors. As for tools, I do not think what you want to do can be easily done in pure python scripts. There are tools though that work quite well for your task:
1) Speaker segmentation and clustering (aka diarization). One tool is LIUM diarization written in Java ( http://liumtools.univ-lemans.fr//index.php?option=com_content&task=blogcategory&id=32&Itemid=60 ). Another one is Idiap tool written in C++ (https://www.idiap.ch/scientific-research/resources/speaker-diarization-toolkit). For my tasks the second one worked better and faster, but the first one is already integrated in some speech recognition systems (see p.2)
2) Speech recognition. Here you have CMU Sphinx with already trained acoustic models for several language and quite easy tutorials. Here you can find an example of how to use it together with LIUM diarization ( http://cmusphinx.sourceforge.net/wiki/speakerdiarization ). Another too is Kaldi, which allows you to get better accuracy, but is more heavy and a bit less documented. This guy also integrated it with LIUM diarization ( https://github.com/alumae/kaldi-offline-transcriber/blob/master/scripts/diarization.sh ). There is a good english model for Kaldi, too.
Sorry for quite a lot of text, but you should keep in mind that the task is hard, even if you have just 2 non-overlapping speakers.
I will track this tread if you have more questions. You also have forums for both Sphinx and Kaldi for further support. Have fun
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python