Well - you might use a clustering algorythm to see whether the clouds of related solutions obtained fall into the same clusters, especially the top-scoring solutions.

Docking scores from different docking algorithms may show strong, weak, or no correlation at all, as the case may be.

It might be that the best-scored docking poses will have some unrealistic torsional angles: If you have access to the Cambridge Structural Database, check if the torsional angles in the docking poses are common or not, and filter out the docking poses accordingly, regardless of the docking score. (This torsional quality check is implemented in SeeSAR, a commercial GUI & command-line software.)

But true validation can only be done experimentally: structure-activity relationships and atomic resolution structures of the protein-ligand complexes.

https://www.ccdc.cam.ac.uk/solutions/csd-core/components/csd/

https://www.biosolveit.de/SeeSAR

Dear Noah Schuster,

I am not sure about suitable method for comparing as above mentioned., as different Docking methods use different docking algorithms.

However, I am of the opinion, for checking the RMSD.

In bioinformatics, the root-mean-square deviation of atomic positions, or simply root-mean-square deviation (RMSD), is the measure of the average distance between the atoms (usually the backbone atoms) of superimposed proteins. the lower RMSD value we get from re-docking, the better the docking pose corresponds to the binding mode of the ligand & is usually a threshold of 2-2.5 angstrom is used.

What you can do : Compare your docked full-length structure with the co-crystallized structure (as procured from Auto-Dock Vina & HADDOCK). The catalytic domain should not be affected too much by the rest of protein. Compare in terms of RMSD, docking pose, and accuracy and contacts etc. The values should be close to the co-crystallized structure. What you are taking is the co-crystallized conformation of small molecule (Ligand) and again you will be trying to reproduce the same conformation in the given binding site. In docking RMSD value is used to compare the docked conformation with the reference conformation or with other docked conformation.

In addition to RMSD values, Types of key Interactions in-terms of Hydrogen bonding, Hydrophobic , Ionic do play an important role in protein-ligand binding (evaluation of Binding energy of poses will determine the fate/ affinity of docked pose).

Based on above evaluations, you will be able to select your better ligand-docking experiment of corresponding complexes.

Hope this helps.

Docking protocol can be validated by docking of crystallographic ligand, provided that (1) there is such a ligand and (2) the ligand shares the same/similar core structure (scaffold) with the compounds of interest.

If your target protein structure has a co-crystallized ligand you can try to extract it and re-dock it using both algorithms and inspect the results to see which one of them predicted the "native" position of the ligand more accurately, you can do it by calculating the RMSD between the predicted and the native pose of the ligand. The smaller RMSD value means that the predicted pose is very close to the native one. And then just choose the algorithm that results in small RMSD values and use it for your case.