In VASP you can initialize the magnetic moments of the atoms with the MAGMOM tag. If you are performing calculations with collinear spins, simply set ISPIN=2 in order to run spin-polarized calculations, and set the MAGMOM tag to the initial values of the magnetic moments for each atom. As an example, for an AF state as the one you ask about, you could set the MAGMOM tag to

MAGMOM = 1.0 -1.0 -1.0 1.0

Be aware that in this tag you should follow the same order as in the POSCAR file.

Also, if you want to calculate non-collinear magnetic systems (or if you include spin-orbit coupling in the calculations with LSORBIT=.TRUE. , by which  the LNONCOLLINEAR tag is automatically set to true also) then you should specify the x y and z components of the magnetic moment for each atom, again in the same order as the atoms appear in the POSCAR file.

Hope it helps

In hematite antiferromagnetism ground state, there are two types of Fe-Fe pair. The long Fe-Fe pair has the opposite spin, while the short one has the same spin. So To descrip the spin state, we need to set + - - + or - + + - spin.

Thank you so much for your answer Hugo Aramberri . As you mentioned, when we include spin orbit coupling in the calculation (by LSORBIT = .TRUE.) we need to specify x, y, z components of the magnetic moment. But as an example, if we already know that , a magnetic system is a collinear antiferromagnetic in nature with one of its element's magnetic moment along y-axis/ x-axis (rather than along z, which is a general case). How do we show this in VASP. Do we need to set SAXIS = 0 1 0 or (1 0 0 ) with MAGMOM = 0 0 tot_magnetic moment in INCAR? Or is there any procedure to do so? I am new to VASP. Can you please suggest regarding this. It would be a great help.

Thanks

Hi Anuroopa Behatha , if you want to set magnetic moment in some different axis (not in z axis), you need to do the spin orbit coupling (SOC) calculation using LSORBIT =.True. tag. In this case also you have to set your Magmom = 0 0 1 direction. VASP will take care of the magnetic moment according to lowest energy configuration.

Thank you so much for your reply Tuhin Kumar Maji . Actually I gave MAGMOM = 0 0 1 direction and SAXIS = 1 0 0 for an example in vaspwiki , NiO ( link1 given below) assuming that OUTCAR would show the magnetization along x-axis. But magnetic moment in OUTCAR appeared in along z-axis only, no matter what SAXIS we take ( energy varies for SAXIS along 3 directions ). Whereas the other example from vaspwiki (link2 given below) shows that in-plane magnetization is given depending on MAGMOM. So I couldn't understand what is the exact way to prove that , the system is having in-plane magnetisation (i.e along x/y-axis) . Can you please explain this in more detail ?

link1 : https://www.vasp.at/wiki/index.php/Determining_the_Magnetic_Anisotropy

link 2 :

https://www.vasp.at/wiki/index.php/Spin-orbit_coupling_in_a_Ni_monolayer