It seems interesting to compare and contrast lip reading with spacial-and-temporal pattern recognition and classification. So the core question to ask here is that, to which extent, these two are considered similar or in common in terms of data pre-processing, feature extraction, and modelling.
Interesting question. On my view are linked.
If we talk about the structure of the recognition network "Lip Reading". That's probably it:
convolutional neural network at the input, which works frame by frame, collapses the frame into a vector. The sequence of vectors is then fed into a recurrent neural network with long-term memory( e.g. on LSTM cells) At the output of a fully connected layer predicting the word.
You can quickly develop with keras and test the relationship by word recognition accuracy.
- Badges
- Science topic
- Similar topics
- Analytical Chemistry
- Extraction