I have a system of more than 10 equations with the maximum order going up to 12. I have applied Newton-Raphson method, but it didn't work out in my computer due to memory issues. So, I need an algorithm that is memory efficient and faster.
Given the strength of NR method, I do not know of any other method which can converge faster. NR has a convergence of 2, i.e., quadratic. Also NR has the inherent problem of diverging! You can try Muller's method which has a convergence of 1.84, besides it can yield real as well as complex solutions! However, coding Muller's method for your system of equations could be tricky.
You can also strengthen NR approach using line Searches and backtracking. Or use Multidimensional secant methods: Broyden’s method. See, Chapter 9 of Numerical recipes by Press, Teukolsky, Vetterling and Flannery.
However I am wondering, why you are not using standard approaches of matrix algebra for solving such problems? There are plethora of algorithms available if you transform your problem in the matrix framework.
Thank you. How can I incorporate the range of values to look for a solution, that is, a variable x can take only solutions in [a,b] which is a finite subset of Real line. How I impose this condition in my formulation?
@ Aravind, Well the preliminary guess and the range to look for should be intuitively obtained from the "Physics" of the problem. This is why I had included the option of Muller's method, which requires three guesses to begin with. When I use it, I try to give far away points as guesses. And then it tries to interpolate a quadratic polynomial through the points and then fine tunes itself in the successive steps to reach to the solution.
In my opinion the Barzilai and Borwein method is very effective for solving a nonlinear system of equations. This method is convergent for very complicate elliptic operators when the initial guesses are faraway from the approximate solution. See for example
T. D. Todorov, Nonlocal problem for a general second-order elliptic operator, Computers & Mathematics with Applications, vol. 69, issue 5, 2015, pp. 411-422.
Most of the Newtonian methods need the initial guesses to be very close to the exact solution. Otherwise they are divergent. On the contrary the Barzilai and Borwein method slightly depends on the initial guesses.
Memory problems? WIth 12x12 matrices? Unbelievable. I think the origin of the problem is connected with high powers, thus causing purely numerical troubles, either underflows or overflows.