eig is used to solve generalized eigenvalue problems A x = \lambda B x, whereas

polyeig is used to solve arbitrary polynomial eigenvalue problems.

In case of rotordynamics you have to solve a quadratic eigenvalue problem.

( \lambda^2 M + \lambda D + K ) x = 0

You might use PERMAS.

I guess you are interested in the Campbell diagram.

Please feel free to visit

http://www.intes.de/kategorie_permas/anwendungen/rotierende_systeme

for further information.