It will be better if you elaborate the problem...preferably with some figure....especially define the elastic foundation properly...what it is...if it is a fixing on test setup you can ignore it...My first advise will be don't simplify the geometry.....and do a nonlinear buckling analysis....
There is a rectangular plate of dimension 20x10x1 m^3,E=200e9,nu=0,3 and stiffness of winkler elastic foundation is 100 kN/M^3. I want to know how to perform buckling analysis in ansys either in apdl or workbench
Your objective is still not very clear to me....so i don't know if my reply will make any sense....Type of analysis will vary with objective even if structure is same..
I assume its a very thick steel plate (1m - i have serious doubts on this thickness...please excuse me if i am wrong) just resting on soil (as you are considering winkler foundation) and subjected to transverse load, now if you want to know the reduction in critical load of your plate due to deformation of foundation (comparing it with beam on rigid foundation - your description seems like so) or change in the buckling pattern etc....in my opinion you need to model the foundation also (if interested in the behavior of structure)....If you just want critical load for your case (beam on elastic foundation)...classical way of doing it is sufficient (thats a simple problem so I assume you r not asking it)...
If you want to simulate the change in behavior of structure due to introduction of elastic foundation....ANSYS gives two options for carrying out buckling analysis (eigenvalue and nonlinear)....Try eigenvalue first just for an assessment but in my opinion nonlinear buckling analysis will only give any useful information and it will be very tough to carry out this analysis in a FE software...as on application of load (more flexible) foundation will deform and you may get convergence issues....to avoid it you may try deflection control problem instead of force control...
I guess your problem is that the plate remains straight under longitudinal load. You should turn on the large deflection option in the analysis settings, then the plate will be buckled. I had the same problem when I do the buckling simulation of beam.