I just found an interesting paper myself: http://www.aclweb.org/old_anthology/C/C14/C14-1111.pdf That might be a good starting point to find more papers on that topic. Nevertheless: more hints are welcome.

Hi Christian,

did you try google scholar?

https://scholar.google.es/scholar?hl=es&as_sdt=0,5&q=HMM+for+%22morphological+analysis%22&scisbd=1

Regards,

Alessandro

Google scholar indeed provides ample references, but I would like to make an aside: the most important part of any such system would in fact be POS tagging since when you know for sure which POS you are dealing with, it is relatively easy to infer any relevant info — just strip the prefixes/suffixes one by one and do look-ups. In the context of known POS rule-based models would surely blow HMMs out of the water because they know what they are doing.

Hi Dmittry,

yes you are right. However, since the 80's (two level morphology) we didn't see many new things in rule based morphology. On the other hand side, the availability of rule based systems is not very good. Therefore, most people use the Porter Stemmer, which might be fine for retrieval, but not for linguists, or they use lexicon based approaches, that are fine for English, but not for many other languages. Common software packages like NLTK or GATE do not have rule based morphology components for common languages like German or French. I guess one reason is, that writing a rule based morphological analyzer in practice is not as easy as it seems to be at first sight.

First doing POS-tagging and subsequently stemming is also a very common approach. However POS taggers seem to make a small number of very stupid mistakes. Many of the mistakes could be avoided, if the morphological analysis would have been done before POS tagging, and the tagger would only propose parts of speech that are morphological possible (e.g. a POS tagger should never asssign Gerund in English to something that doesn't end with "-ing").

This is just to say, that I could imagine that someone has tried out HMMs for morphology and has written a niece paper about it.

@alessandro: yes I tried google scholar, but I have the fieling I am missing something.

The problem is that it is hard to directly use HMMs for morphology since they need predefined sequences. If you have already partitioned the word into morphemes, all the work is basically already done: even if some of the morphemes can be present in different POS their combinations must be easily and more or less uniquely parseable. However, if the morphological makeup of words is opaque, we should probably begin with POS tagging and then do the decomposition. HMMs are probabilistic generative models heavily dependent on the relatively rigid word order; they seem to be inherently bad for POS tagging and morphological analysis of inflective/fusional languages. What you propose (“never asssign Gerund in English to something that doesn't end with "-ing"”) can be easily incorporated into a discriminative maximum entropy model, which can actually have a HMM as a component. For instance, the Stanford Parser POS tagging module is a MaxEnt model, even though it is aimed at English: http://nlp.stanford.edu/software/tagger.shtml