Please share the system and its time domain (TD) simulation too. The problem may be either incorrect estimation of the eigenvalues, or improper simulation time. The TD solver has a significant impact on the results. In some situations the trapezoidal rule conflicts with the Euler method for solving DAEs.

You can find the states associated with the given eigenvalues using the participation matrix.

Check your linearized model that you have developed. Definitely there must be some errors in your equations while you tried to solve them through ODE.

Dr. El-Shimy is right. You need to revise the simulation or as Syed A. Raza said, you have to be sure that the linearized model properly represent your converter and its control system.

The key point is "What solver you use in MATLAB?"

ode functions (or other variable step solvers) use "dynamic step change" to reduce the numerical instability during the simulation running.

I think that your linear model is continuous.

You have to simulate your model with "fixed step/discrete" solver and use the same step-time in your simulation. If you do so, you may observe the instability in your converter control system; otherwise, the unstable eigenvalues are because of incorrect modeling procedure.

Thank you Hossein Mahdinia Roudsari for your feedback. Generally, your comment is excellent; however, in the specific case of the system under consideration, we (Wael Telmesani , and myself) found in a private discussion another issue related to the way components are modeled in the MATLAB/SimPowerSystem toolbox. In summary, there was a modeling issue that is mostly related to the MATLAB itself due to the consideration of the electromagnetic transients in modeling all components.

Thank you all and special thanks to Prof.El.Shimy and Dr.Raza for answering my question.