I am using VASP code when I am doing pure LaVO4 unit cell structural optimization then volume verses energy minima is coming at 1.4 kb pressure but when I am doing with Eu doping 5*1*1 supercell minima is coming at -40kb. Is it reasonable ?
Try to relax the structure with e.g. ISIF=3. Due to the doping and the resulting symmtery breaking crystalographic axis of your system might behave differently.
Adding Eu may well change the symmetry of the crystal, did you optimise the lattice before you started your E-V curve calculations? I assume you're using a finite-basis-set correction, but are you sure your plane-wave cut-off is high enough to capture the main stress contributions? The FBSC can only correct energies and stresses which are already reasonable.
It might be worth trying a 5x2x2 supercell as well to see whether the periodic interaction is significant -- if this is too expensive, you could try shifting your k-point set origin to (0.25,0.25,0.0) and see whether it makes any difference (assuming you're using a gamma-point centred grid to start with).
(I'm assuming you've re-converged with respect to plane-wave energy cut-off and k-point sampling when you add Eu, but if not then you should definitely do that first.)
It is of course possible that you're getting the correct answer, but I would check these things first. Apologies if this is not completely relevant, I use CASTEP and am not familiar with VASP specifically, but they are both plane-wave DFT codes so the principles are the same.