I am trying to make an LSTM model which generates tags for every word in a sentence. As the input, I am using glove vector for each word and want to get a tag as the output. All of the packages I checked, either require fixed time steps sequence or an embedding layer. But I want a model which won't have embedding layer and also time steps is variable. Which is the best package that could possibly help me in building such model?
If I understand correctly your problem, you can use keras LSTM implementation with time distributed dense output layer (https://keras.io/layers/core/#timedistributeddense). You will thus generate prediction per each word in a sequence. See a small example also here http://stackoverflow.com/questions/34966614/sequence-labeling-in-keras
As for fixed time steps (do you actually mean sequence length?), usually you should at least specify some maximum sequence length. Then you can model shorter sequences with masked input/output. This is done, for example in tensorflow sequence-to-sequence machine translation (https://www.tensorflow.org/versions/r0.10/tutorials/seq2seq/index.html#bucketing-and-padding)
Keras does was much easier for me to deal with tagging, extraction for seq2seq length. Example looks at long sequences, and was helpful for me, If I'm comprehending you properly, https://github.com/fchollet/keras/blob/master/examples/stateful_lstm.py. Glove works too, http://nlp.stanford.edu/projects/glove/. Their is a text generation bit in keras too, gives you prediction, your then in the comment above running seq2seq...
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence