(as we know, critical load in linear analysis in the presence of lateral load and in absents of lateral load is similar to each other, now whats the effect of nonlinear analysis in calculating the critical load?
If you perform a nonlinear analysis with lateral load before the calculation of the eigenvalue of the buckling problem, the stiffness matrix is that of the deformed structure. In linear analysis the stiffness matrix is that of the undeformed structure. You can also think of the buckling problem with lateral load as a coupled nonlinear quasi-static problem where real load-displacement history is calculated for both loads acting simultaneously (usually called postbuckling); this very likely requires the use of a displacement control algorithm.
but i mean that in linear analysis in order to calculate critical load in a column with lateral load and without lateral load, critical load are the same in linear analysis.now i want to know that in nonlinear analysis how critical load will be calculate and dose nonlinear analysis has any effect in critical load?if you have any example, it can help me
It is quite difficult to give a correct answer to your question unless all details of your specific problem are highlighted. But as far as I understand you deal with the classical problem of a flexible column under compressive dead load.
In the linear buckling analysis, the buckling load does not depend on lateral forces since the bending stresses due to these forces have no effect on the column stiffness (see previous comments by Martin).
If you perform nonlinear analysis, you will obtain two different solutions for the two loading cases. In the absence of lateral forces, the buckling load is expected to be very close to that predicted by linear analysis (provided deformations due to compression are negligible). If lateral forces are applied in addition to compressive loads, you can obtain load-deflection curves which show how the column respond to compression. In this case, however, the buckling load cannot be determined as straightforwardly as in the linear analysis since no bifurcation points occur on the solution curve.
As Stanislav says, you should clarify a little more what do you need to calculate. I think that what you need is to perform a nonlinear analysis with the lateral load, and then calculate the buckling load. This will give you the effect of the lateral load on the critical load and buckling modes.
With the presence of lateral load, the buckling shape will be determined by the lateral load direction, and the buckling process will be smoother. Using nonlinear analysis, we can follow the load deformation path before and after buckling, as in snap through problem, while using linear buckling analysis we can only get critical load and no deformation path.