Thanks for your response. Yes, I tried all the options available in the odeset. The problem is: in some set of conditions its working but with other parameters it does not work. Better, do you have any idea how to analyse the instability with the Jacobian matrix?
ODE15s is used for solving stiff differential equations. If this scheme is causing instability then try to play with odeset, or use ode23tb instead, as it is more robust. Secondly if you can share the ODE you are solving then you may get more valuable comments for your problem.
What is the nature of the instability. Is it a time step instability (too large) or precision instability (time step too small). Also how can one be assured the problem does not lie in your modelling. Here I support what Ali is saying
see http://matlab.cheme.cmu.edu/2011/09/18/error-tolerance-in-numerical-solutions-to-odes/