These are excellent questions and you have clearly thought about this a lot before asking them. I'll try to address your specific points, but first some background:

In general, an N-particle system *cannot* be described accurately by N 1-particle wavefunctions. The N-particle wavefunction depends on the positions of all particles:

Psi(r1,r2,r3,...,rN)

and describes the complete state of the system, including any particle-particle correlation.

In the special case that the particles are independent, then this wavefunction is separable and we can write

Psi(r1,r2,r3,...,rN) = psi_1(r1) psi_2(r2) psi_3(r3) ... psi_N(rN)

where "psi_j" is the 1-particle wavefunction for particle j. This is quite a good approximation if the particles have widely differing masses, and is usually used to separate the nuclear and electronic wavefunctions.

Clearly, the separable form above does not describe indistinguishable fermions (e.g. electrons) because it does not have the correct antisymmetry upon particle exchange. If we just have 2 fermions, then the above expressions gives:

Psi(r1,r2) = psi_1(r1) psi_2(r2)

If particles 1 and 2 are interchanged we ought to get minus this, but instead we get:

Psi(r2,r1) = psi_1(r2) psi_2(r1)

In many quantum chemistry approaches the correct antisymmetry of the N-particle wavefunction is ensured by using a Slater determinant of 1-particle wavefunctions, i.e. not in a purely separable form, but as a linear combination of all possible separable forms. In the above case this would give:

Psi(r1,r2) = ( psi_1(r1) psi_2(r2) - psi_1(r2) psi_2(r1)  ) /sqrt(2)

where sqrt(2) is just a normalisation constant. Now if we exchange 2 particles we get

Psi(r2,r1) = ( psi_1(r2) psi_2(r1) - psi_1(r1) psi_2(r2)  ) /sqrt(2)

                = -Psi(r1,r2)

as required. I should stress that this does not mean this is "the answer", it just satisfies the required symmetry -- more sophisticated methods exist which, for example, consider many Slater determinants.

So far we've focused on quantum chemistry methods to solve for the N-particle wavefunction. The complication is that density functional theory (DFT) tells us that in fact the ground-state properties of a system don't depend (explicitly) on its wavefunction at all, they depend only on its particle density. Since the same density can be described by different wavefunctions with different Hamiltonians, this raises the possibility that the density computed from the full N-particle wavefunction could be the same as one computed from N 1-particle wavefunctions -- this is the Kohn-Sham approach. In this approach all that is needed is to find the appropriate modification to the single-particle Hamiltonian which leads to it giving the correct ground-state energy and density, and this modification is called the exchange-correlation potential.

Hopefully this has addressed some of your general questions; now for your specific questions regarding exchange and correlation for electronic states:

1. Correlation energy is usually defined as E_exact - E_HF in Chemistry, but not in Physics (there isn't really a precise, accepted definition in Physics). You are completely correct that the term "exact exchange" is a misnomer, because it is only exact if we use the correct wavefunction; I prefer the term "Fock exchange".

2. You are essentially correct and, as we've discussed, there are no "exact" 1-electron wavefunctions (in general). Regarding the Coulombic repulsion itself, in DFT this is approximated by the Hartree interaction:

E_H = int dr int dr' [ n(r)n(r') ] / |r-r'|

in atomic units. This *does* contain a Coulomb interaction between the density of a particle and itself, and is the major component of the "self-interaction error" in common DFT approaches. The trouble is that since the 1-particle states we use in Kohn-Sham DFT are fictitious we could take a different linear combination of them for each fictitious particle and get the same total density -- however, the self-interaction would be different! This is why you cannot address the self-interaction problem straightforwardly in a purely 1-particle density scheme. The main ways to treat this are either to subtract the individual 1-particle density's energy contributions (the SIC method of Perdew-Zunger, 1981); project onto specific, precomputed states and remove their self-interaction (e.g. DFT+U); or move to 2-particle density matrices, as you mention.

3. Yes, we are essentially saying that in general there are no 1-electron wavefunctions in an N-electron system; however it is not necessarily true that there are no 1-*particle* wavefunctions. If you had the N-electron wavefunction you could ask questions such as "what are the best N 1-particle wavefunctions to represent this state?". These 1-particle wavefunctions do not represent individual electrons, but something more like the normal modes of the quantum system; these particles are called quasi-particles of the system, and they can be very useful in describing fundamental properties and excitations of an N-body system. They can even be used as a framework for a quantum N-body approach, e.g. the many-body pertubation theory called "GW" in Physics and "dynamically-screened Hartree-Fock" in Chemistry.

If the concept of quasi-particles seems odd, perhaps it helps to think of motion in a bath of water. The motion of the water surface cannot be represented as a product of the individual water molecules, but you can easily compute the normal modes of the motion. The complicated collective response of the water would be extraordinarily difficult to compute, even classically, but it has simple normal modes, and if we consider the motion in terms of these normal modes we can basically ignore the existence of water molecules altogether! The normal modes look just like single states, and even wave packets on the surface of the water are easily expressed in terms of these normal modes. In just the same way, all the complexity of an N-body quantum system can lead to very simple, elegant 1-body descriptions -- if only we can find them!

Sorry I haven't had chance to go into more detail, but I hope that's helped a bit at least.

All the best,

Phil Hasnip

Dear Phil,

Thanks for the was a brilliant answer. The answer to point 1 is pretty clear. For the answer to point 2, I read up on the hartree potential term. From what I've understood, the hartree potential is calculated by the poisson equation and then applied as an effective potential and that clearly contains the self interaction error. 

1. When you say " The trouble is that since the 1-particle states we use in Kohn-Sham DFT are fictitious we could take a different linear combination of them for each fictitious particle and get the same total density -- however, the self-interaction would be different!"  - Here, do you mean that because the linear combinations for the N 1-particle states can give us a new set of N 1-particle states (which, lets say,are somewhat less fictitious than the original ones) without changing the total electron density, it doesn't change anything with regards to physical accuracy because self interaction error is apart of the formalism to calculate the hartree potential, which includes all electrons when being calculated?

What if we use the following method -  we exclude the density of a given particle from the total density (this would be the 'remaining density') and calculate the hartree potential corresponding to this given particle by solving poisson equation for the "remaining density". This would be of course computationally very expensive because now we have a seperate hartree potential for each electron. However, would this process not be more accurate than doing nothing at all because now we have gotten rid of the self-interaction error, although the 1-particle densities are still fictitious.

2. The N 1-particle wavefunctions are not necessarily fictitious in an N body system as I found out in these threads from a cursory look ( http://physics.stackexchange.com/questions/27971/examples-of-exact-many-body-ground-state-wavefunction?rq=1 ) . For electrons this becomes otherwise because they are interacting - as was replied to me here ( http://physics.stackexchange.com/questions/229399/many-body-wavefunction-and-exchange-correlation ).

3. I know about the DFT+U and GW methods. In fact the very reason why I came upon the original question was because I have been trying to use these  methods like a blackbox without knowing the mathematical basics. I thought better to start from the very low level basics. Never had this overview, thanks a lot for that.

Best,

Abhishek

Dear Abhishek,

Sorry, I was not clear enough in my initial reply.

1. It is possible for some N-body wavefunctions to be expressed as a product of N 1-body wavefunctions, but it cannot be done in general (only if the particles are independent).

Yes, it is possible to fix the self-interaction error by modifying the Hamiltonian. The usual method is not quite as you propose, but instead the usual Hartree-XC functional is applied and then the contribution from the 1-particle density is subtracted [1] (don't forget that XC has self-interaction as well). In either case, the self-interaction error is negated but the Hamiltonian becomes eigenstate-dependent, which is rather unsatisfactory and makes it difficult to compute total energies, forces, stresses etc.

2. In Kohn-Sham DFT the 1-particle wavefunctions are fictitious, but this is not to say that such 1-particle wavefunctions are always fictitious.

All the best,

Phil

[1] J. P. Perdew and Alex Zunger, Phys. Rev. B 23, 5048 (see link)

http://journals.aps.org/prb/abstract/10.1103/PhysRevB.23.5048