See this: http://www.sciencedirect.com/science/article/pii/S0165168402004498

Thanks a lot Laurent. now i got this paper and one i forgot to mention ofcourse the S.O.R technicque( sucessive over relaxation technique).

You could lower the number of iterations by using the symmetric Gauss-Seidel method as a preconditioner to a Krylov method like conjugate gradient method http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf

http://www.netlib.org/linalg/html_templates/Templates.html

Gauss-Seidel as well other stationary iterative method need O(N) number of iterations need for convergence (a problem that is twice as big as the original one, needs 2 times as many iterations).

The problem is that these methods are good in reducing high frequency error components, while the low frequency ones are reduced slowly.

Multigrid methods instead apply a few iteration of e.g. Gauss Seidel on your original mesh and then approximate the smooth residual on a coarser mesh and apply a few iterations, approximate on an even coarser mesh and so on until finally the mesh so coarse that a direct solver is efficient and the problem is solved on this mesh. Afterwards the solution is interpolated to the next finer mesh and a few iteration of Gauss-Seidel are applied, interpolate to next finer mesh ... until the finest mesh where a few iterations of Gauss-Seidel are applied.

Ideally Muliigrid methods need the same number of iterations regardless of the problem size.

http://www.kfunigraz.ac.at/imawww/borzi/mgintro.pdf

If you do not have mesh hierarchy use an algebraic multigrid methods. We habe one in DUNE http://dune-project.org but you may also use the one of hypre or Trilinos.

https://e-reports-ext.llnl.gov/pdf/333205.pdf

Thanks Dr. Blatt. I am now getting out of the bush. I am a novice and am only aware of the conjugate gradient method which is a directed search based on the error vector.

depending on your matrix it may also be possible to use a limited update method for SOR that relates to Gauss-Southwell relaxation. The idea is to restrict the update of the solution vector in each iteration to only those values which are influences by large changes in the step before. This especially makes sense for sparse matrices. For details see http://mia.sourceforge.net/wollny02comput.pdf (Section II.B).

you can try Successive over elaxation technique.(SOR)

Thanks Gert Wollny for the link which i found it to be very informative. Thanks Anil , I have tried the SOR and found out that the SOR for the problem i am solving reduces the number of steps from 20 to 12 giving a set of solution of equivalent accuracy.