The thermodynamic stability of a crystal structure depends on the Gibbs free energy. This is the minimum free energy of a crystal structure where the unit cell is allowed to change in response to temperature. But calculating the Gibbs free energy is tricky, so people use the Helmholtz free energy instead. This is the minimum free energy for a fixed unit cell, that means that you assume the crystal does not change with respect to temperature, you assume that there is no thermal expansion and the vibrational frequencies do not change with temperature. This is actually a very good approximation, the thermal expansion has a small effect on the phonon frequencies.

Free energies are negative values and the more negative, the more stable is the structure.

I don't know what method you intend you use, but regardless, it is important to try to account for phonon dispersion. A common method is to perform the vibrational calculation on a supercell. A 3x3x3 supercell covers 27 k-points in the Brillouin zone and is good enough for the high frequency modes. Very long wavelength phonons cannot be calculated with the supercell method as you would need very large supercells.  Using the elastic constants of the crystal, you can use Debye's method to extrapolate the dispersion. This improves the accuracy of the calculated free energy, but most people don't do this.

Note that a phonon calculation assumes that the structure is an energy minimum. So you have to optimize the structure with the same energy method you use to calculate the vibrations/force constants.

Dear Dr. Nyman:

    Thank you for your kind suggestions.

    Here I calculate the phononic properties using the supercell approach as you recommended with the PHONOPY code. Difference in the Helmholtz free energy of phonons between two structures is observed (please find our result in the attachment). However, positive value is found for both structures at 300 K, while the phonon dispersion shows that the black structure is thermodynamically stable (no negative frequencies are observed), while the red structure is unstable.

    Is there any evidence from the figure attached below to explain the relation between the free energy and stability?

I think you have plotted the vibrational contribution to the free energy Fvib.. To get Helmholtz energy you need to add the static energy. The curves start at a positive value at T=0, which should be the zero point energy. The curves then go to increasingly negative values with the increase the temperature. This is the entropic contribution -TS.  The derivative of the curve is -S. This is NOT the phonon dispersion.

Phonon dispersion means that the vibrational frequencies change for different k-vectors. Waves that go in different directions have different frequencies. By making supercells, you calculate phonons going in a larger number of different directions.

From the graph you have shown you can see that the red structure has a larger entropy. The black has a larger zero point energy. You need to add the static electronic energy to see the free energy and the relative stability between them.

And "thermodnamically stable" does NOT mean that there are no imaginary frequencies! It just means that the structure is a local energy minimum. Thermodynamically stable means it is the global minimum.

I am sorry that I forgot to attach the phonon dispersion (please find our result in the attachment, a for the black structure while b for the red).

Here I still have two question:

1. How to extract the static electronic energy? (e.g. Does it mean the energy without entropy in VASP?)

2. Since no imaginary frequencies in the phonon dispersion just means a local energy minimum, how to investigate the global minimum? Do you mean Gibbs free energy or something like that?

Yes, the total energy without vibrational contributions.

I don't know what kind of system this is, you haven't described any details so it's hard to help you...

There might be other crystal structures that you haven't considered, that you don't know about, that are even more stable. There are often several different crystal structures of any one compound. One of them is the most stable, and that is the thermodynamically stable one. All the others are still stable in the sense that they can exist, they are mechanically stable and correspond to energy minima with real phonon frequencies. But they are not thermodynamically stable, which means that they will eventually transform to a more stable structure. The transformation can however take a very very long time, so they appear to be stable. We say they are kinetically stable or metastable.

The system belongs to a new two dimensional material, 2D boron sheet...

Do you mean only one structure is thermodynamically stable, while others with real phonon frequencies are mechanically stable?

Yes.

Dear Jonas Nyman,

How does analyze phonon DOS and phonon bandstructure ? What informations we get from phonon DOS and phonon bandstructure plot ?

Can we say system stability by phonon DOS or phonon bandstructure ?

Thanks

Shilendra

The phonon DOS can be used to calculate the vibrational entropy, vibrational energy and heat capacity. Please see this paper:

http://pubs.rsc.org/-/content/articlehtml/2016/cp/c6cp02261h

The supplementary information to this paper contains formula for calculating the properties. 

updates:

https://www.tandfonline.com/doi/full/10.1080/21663831.2017.1298539