I want to use embedded word vectors as feature in existing conditional random field (CRF) with gazetteer features for sequence labeling task in text. One way to do this is make cluster of word vectors and take the cluster id as feature, Is it possible to use word vector itself as feature. Since CRF needs binary features and embedded word vectors are real number. How would one use real number valued feature in CRF ?
There is one paper of Bengio's group. "Word representations: A simple and general method for semi-supervised learning". They have used the same but its not at all clear to me.
There are continuous CRFs, called CCRFs, which could cope with real-valued inputs. But that will make things quite complicated. And the paper you mentioned uses CRFSuite, which doesn't work with continuous values at all, which is strange.
Short answer: you cannot use real-valued embedding vectors in a nominal-feature CRF.
Three ways out:
1. Nominalize the vectors using clustering as you proposed, or produce a fixed list of the highest/lowerst-valued dimensions and use these as ids. Does not work so well
2. Switch over to a neural classifier and provide your gazetteer and other features as additional 'neurons' in the last layer before the classification layer. Works better, but you'll lose the CRF capability of normalizing over the label sequence.
see: https://www.ukp.tu-darmstadt.de/fileadmin/user_upload/Group_UKP/publikationen/2014/2014_GermEval_Nested_Named_Entity_Recognition_with_Neural_Networks.pdf
3. Use representations from distributional semantics that produce nominal features, such as distributional thesauri. See e.g.:
https://www.lt.informatik.tu-darmstadt.de/fileadmin/user_upload/Group_LangTech/publications/GovindEtAlMoo_ICON2014.pdf (Fig 1)
- Badges
- Science topic