Has anybody tried to solve a nonlinear eigenvalue problem; either using inverse iterations or minimization of the nonlinear Rayleigh quotient? Sharing numerical experience and/or insight into this problem is appreciated.
The typical method is used by our research school is shooting method. You take a boundary conditions, e.g., on the left side, for the DE-system as initial, solve it using Runge-Kutta method at the necessary interval with some arbitrary Ra (and other parameters) and make a correction to satisfy the right side boundary conditions. It is provided by some iteration methods, which are not really stable but effective. I'll try to find any paper in English.
Besides, You can try the spectral methods like Galerkin expansion. It gives approximate results, but it is really effective and simple method.
One possible algorithm is described in "Stability of Parallel Flows" by Betchov and Criminale (Academic Press, 1967), chapter "Machine analysis of viscous flows".
What is the form of the eigenvalue problem? If it is A(s)v=0, where the element of matrix A are the functions of eigenvalue s then several numerical method exists e.g.
Singh K. V. and Ram Y. M., “Transcendental eigenvalue problem and its applications”, AIAA Journal, 40 (7), pp.1402-1407, 2002.
F.W. Williams, and D. Kennedy, “Reliable use of Determinants to Solve Non-Linear Structural Eigenvalue Problems Efficiently,” International Journal for Numerical Methods in Engineering, 26(8), (1988), 1825–1841.