Dear Sergii,

Software validation seems quite anonymous to me. I wonder how one can validate software with a particular set of proteins and ligands. Regarding optimization of protein and a ligand for docking studies, a fruitful discussion has already been made on RG earlier under the heading "Docking of crystal structures without minimization?" Please go through the same.

About the optimisation of the protein, the point raised by the Sergil seems to be authentic.

I would say that in some cases we need to optimise the geometry and in some cases we dont need to even bother about it. Moreover, if we have a protein with good resolution deposited in any repository than we can simply use it for docking without optimising it for the geometry. See, the main goal of the docking is for the pose prediction and it can be predicted with the accuracy when you will give the good input for it. If we have in hand the good structure of the protein than why not we go for docking with that. But let me tell you the other phase of it, that this is not the case all the time. Some proteins are very dynamic in nature so one need to take the average structure by geometry optimisation or some other time non-resoluted structure is there. so we need to tune our methadology accordingly.

Futher, as suggested by Mohsin, you can look on the discussion page..https://www.researchgate.net/post/Docking_of_crystal_structures_without_minimization

Moreover, regarding software validation, I am not in the favor of the software validation rather it should be method or algorithm validation.

Upto now, I am not able to understand, Why do you want it? I dont think any need of it.

Bioinformatics is all about the predictions, how good how bad it is based on the hands of one who is handling it.

Simply Garbage in Garbage out---

I hope it will help you.

I'm really surprised for such a negative feedback.

Obviously, we can repeat "garbage in - garbage out" and discourage people, but if so - lets use rational arguments.

"I wonder how one can validate software with a particular set of proteins and ligands" - that seems pretty easy. Sergii said he has a "training set". He didn't say he wants to give any ultimate answer, but he chooses a set of structures for some kind of benchmark. Tell me, what wrong do you see in this approach? I'm sure Sergii knows the challenges that stand in front of him.

"I am not in the favor of the software validation rather it should be method or algorithm validation" - in fact, most of approaches are similar and the main, biggest difference comes from scoring functions. Therefore, it's hardly possible to use a chosen software and test all the solutions. That gives pretty much sense in testing different software, as Sergii wants.

Sergii, "then how can I be sure that the software will be available to predict binding poses for separate ligands and proteins, not isolated from X-ray complexes?" - that's the point, we can NEVER be sure. It's just a prediction, what makes comparison you want to do even more interesting. Couple months ago I made something related to this problem and I used reproducing of most important interactions as a scoring function (maybe one day finally I'll submit this paper ;)). According to the minimization problem - that could be thing you might want to compare, as these calculations are pretty cheap.

Good luck! I don't know which software you want to use, but I believe many researchers may be interested. Thank you in advance - that's something what I always wanted to do, but time flies... And feel free to contact me - I still have some ideas or even ready results which could support you work.

FIrstly, i request Rafal to make this discussion congenial and logical. Now, back to the answer, Itiratively, I will say there is nothing like validation of the software rather its a method validation. Ok you tell me, How one can validate a software??

Let me remind you that scoring function is also inherent to the software and by scoring function you cant get anything that relate to the experiment; it is just a number used for the ranking purpose.

If anyone by taking a small data can validate a particular software than I am 100% sure that no software will give the right prediction. Its the nature of algorithm that really treats the protein or ligand in some or different way which thereby decide its accuracy.

I will ask one more question, Why we have so many methods and software? If one software can give benchmark results than according to you other should have close down their shops. But its not so? the reason I gave you already

Now, in conclusion, it depends on the expert how he/she utilise the information in order to get good results/prediction. Hence, its a garbage in and garbage out.

I hope you will convince by my points.

Basically, it would be great to have an opportunity to rank pure algorithms. Unfortunately, it's impossible as most codes offer at least slightly different approaches and doing this would require writing everything to work in one, integrated environment (and it's not Sergii's case). Moreover, some of them have closed codes what makes any extension impossible.

Sergii wants to make some kind of docking software validation. In this case, it means he wants to determine how a chosen software behaves on examples of his choice. That's obvious that a software is a set of routines that implement more or less established algorithms, a kind of pipeline (or protocol) which aims to give a set of docking poses. And, indeed, that is a software validation - of course on a limited dataset. If it doesn't have sense - why is it pretty widely done by people working in this field? Even authors of docking software provide their test datasets (for instance, ZDock developers - https://zlab.umassmed.edu/zdock/).

Some comments are unfortunately misguided as nobody claims that this kind of experiment may give any kind of an ultimate truth. Why are there so many codes? Because scientific community constantly tries to make things better and they have different ideas. It is also well known that different docking approaches have their pros and cons and none of them is really good. But with knowledge of drawbacks one can expect - it's still, at least, a bit easier and more accurate to predict anything. And still, the lack of an universal solution makes it necessary to carefully choose one which fits best to our needs.

Finally, lets better make more Science than argue on things which are rather minor problems :)

Cheers!

Very well said "lets better make more Science than argue on things which are minor problems" this is what I have already pointed out. You might be aware of the fact that most of the people do same what Sergii wants to do but, the statement made was "routinely used methodology looks slightly strange for me". The statements made, were not discouraging rather rational.

Ya, that make sense...

Thanks Rafal, Mohsin and of course sergil for the discussion

Good luck!!

I hope this pubblication could help!

https://www.ncbi.nlm.nih.gov/pubmed/26035098

Good Luck ;-)

This was a trendy area of research about a decade ago (see refs.), which basically resulted in a few useful (sort of) outcomes:

1) Accuracy of a docking tool depends on the protein target and it is impossible to predict which tool would work best for a given target.

2) A few docking tools, such as Glide, Dock, Autodock, etc., have shown an ok accuracy with any kind of targets.

3) The problem is not in docking (that is, correct placement of a ligand), but in scoring (high-throughput assessment of the ligand-protein affinity). There are no easy fixes to this problem because an accurate affinity assessment taking into account all solvation and entropic effects takes weeks of computing time, not seconds.

Personally, I would not recomment to start a massive docking benchmark these days. This would be a loss of your time with a little benefit to the community.

Kellenberger, E., Rodrigo, J., Muller, P., and Rognan, D. (2004) Comparative evaluation of eight docking tools for docking and virtual screening accuracy. Proteins: Structure, Function, and Bioinformatics 57, 225-242

H, C., PD, L., F, G., T, L., and J, L. (2005) On evaluating molecular-docking methods for pose prediction and enrichment factors. J. Chem. Inf. Model. 46, 401

JC, C., CW, M., JWM, N., RD, T., and R, T. (2005) Comparing protein-ligand docking programs is difficult. Protein Struct. Funct. Bioinform. 60, 325

Warren, G., Andrews, C., Capelli, A., Clarke, B., and LaLonde, J. (2005) A critical assessment of docking programs and scoring functions. J. Med. Chem. 49, 5912-5931

Huang, N., Shoichet, B. K., and Irwin, J. J. (2006) Benchmarking sets for molecular docking. J Med Chem 49, 6789-6801

McGaughey, G. B., Sheridan, R. P., Bayly, C. I., Culberson, J. C., Kreatsoulas, C., Lindsley, S., Maiorov, V., Truchon, J. F., and Cornell, W. D. (2007) Comparison of topological, shape, and docking methods in virtual screening. J Chem Inf Model 47, 1504-1519