I want to know how one can conclude that there is a hybridization of different orbitals of the atoms in a compound, by seeking the density of states (DOS) versus energy plot?
Or in other words, What are the conditions of a hybridization in the DOS?
Are you using HF or DFT methods? Because in the first case what you get from the calculation are orbitals, but in the second case in general you get orbitals in the Khon-Sham representation.
Then in my modest opinion it could hard be to find a sort of hybridization, maybe you could do it by a comparison between orbitals in similar systems, where in the first you assume no hybridization and the second is what you're studying. The main problem is that orbitals are in the Khon-Sham representation, so you can make comparisons between orbitals only in that representation.
If in the DOS, there is an overlapping of some orbitals like p and d, is it the sufficient condition for concluding that there is a p-d hybridization? Or some other conditions are necessary?
Generally a gap/pseudogap shows up in the DOS around the Fermi level when a hybridization between p-d orbitals occurs, representing covalent bonding between the atoms. DOS peaks above (antibonding states) and below (bonding states) the gap/pseudogap appear at the same energy levels for different atomic species involved if you look at the local DOSs. The width and depth represent the strength of the hybridization. Similar discussions may be found in PHYSICAL REVIEW B 90, 054109 (2014) and reference therein. Hope this answer helps.
As like inteerference patterns. If the density of states of two orbitals are lied in the same energy range ( x-axis, energy axis) then they make hybridization, as for example, if p and d orbitals make hybridization then the value of the density of states (DOS) plot (i.e.k the value of y-axis will increase) will increse as like interference patterns.