As is well known, iterative methods for solving linear systems such as Successive Over Relaxation and the like, are very attractive for solving many problems such as sparse matrices. These methods, in general, are formulated in the context of determined system in which the number of equations is equal to the unknowns. Now and for sake of simplicity, let us assume that we have one additional observation and We need to update the previous solution. In other words, now we have an over determined system with the provision of this additional observation. The question is how to include this observation to the to update the previously computed parameters. Indeed, the theory of parameter estimation provides a lot of guidelines to handle this task to get an optimal solution in the sense of least squares. But let us assume that we need to stick to the iterative approach for parameters. Then with assumption how we could handle the additional observation for parameters update and what kind of errors that we need to minimize.