Way back when for my Masters I was the first in my lab to perform Long term potentiation in Chonically implanted animals. Figuring out the surgery almost killedmy career befroe it even started. Using coordinates from a variety of papers I could not get the appropriate pp/dg Ep to show itself. Eventually I got it, but during my first publication made a very big deal about reporting the number of animals implanted versus the number of animals used that had good EP placements. I think it was something like 11/22. (Hargreaves et al., J Neurosci 1990). During the rest of my grad careeer after going to sufficient conferences and talking to other Ph.D. candidates, and any PIs that owould deign to talk to me, I did learn a number of tricks that are never reported. i then went and did something similar for my first Postdoc and somewhere around my 2nd year there, I began to look back and wonder, why i thought this surgery was so difficult, since my hit rate was up over 90%. It got to the point, where i claimed I could do the surgery blind folded. This was also bolstered, by quickly learning EPs in CA3 to CA1 circuit. different EP different electrodes, but similar principles.

Then did another Postdoc and learned single unit recording, same rats, same hippocampal structure, but different electrdoes and devices and recordings, and again for a year and a half I was thrown for a loop. Although confidence was shaken, I knew then that it could be worked through and had to refind people to talk to, and rethink what variables of the surgery and electrode construction I could alter. In the end there it was not the surgery, but learning to plate the tips of the fine wire electrodes (12micron diameter wire), and plate in such a way that the tips would hold the plating, and to test out the now lowered impedance for cross shorts, and stable lowered impedance.

Somewhere along this process, my favourite quote has become...

"Research is what I am doing, when i do not know what I am doing". (from some German mathematician). Science as a system of thought, can be applied to the development of methodology as well.

At first everything is hard, especially if you are developing expertise in isolation, alone without anyone, but books and papers to guide you.

You truthfully write: "... research findings may often be simply accurate measures of the prevailing bias". Looking widely, a major problem is people following an 'accepted' method which is actually invalid. In my field, Reilly's Law of Retail gravitation is widely used, but I showed in my PhD thesis that it is fallacious! We cannot believe any such journal articles. Caveat emptor.

With something as finicky as High Permformance liquid Chromatogagraphy not all details may make it into the methods sections. In every field there are tricks that are assumed to be known, if you are just starting out these may not be apparent and you may need more guidance on the techniques in general. If you are experienced at running HPLC in a more than one lab, for more than one reason, then there still may be some details left out, and one can either visit the lab in question, or contact them directly. if the work was performed some 10 yers ago such remembrances may be difficult.

As for all findings being Fallacious... some false findings are accidental and non-reproducible, some are out right falsified, but most are replicable when appropriate techniques and skill levels are applied. To catch false or flawed findings is the job of peer review, which is not perfect, but best we have at the moment.

Eric, you really gave an interesting perspective. I have now realized that some of the work which is being labelled as false, might not be because of unethical practices of the researcher in question but purely ineptitude of the person labeling. Instead of engaging the author wholesomely, it is easy to get into the blame game. I like that take!

Way back when for my Masters I was the first in my lab to perform Long term potentiation in Chonically implanted animals. Figuring out the surgery almost killedmy career befroe it even started. Using coordinates from a variety of papers I could not get the appropriate pp/dg Ep to show itself. Eventually I got it, but during my first publication made a very big deal about reporting the number of animals implanted versus the number of animals used that had good EP placements. I think it was something like 11/22. (Hargreaves et al., J Neurosci 1990). During the rest of my grad careeer after going to sufficient conferences and talking to other Ph.D. candidates, and any PIs that owould deign to talk to me, I did learn a number of tricks that are never reported. i then went and did something similar for my first Postdoc and somewhere around my 2nd year there, I began to look back and wonder, why i thought this surgery was so difficult, since my hit rate was up over 90%. It got to the point, where i claimed I could do the surgery blind folded. This was also bolstered, by quickly learning EPs in CA3 to CA1 circuit. different EP different electrodes, but similar principles.

Then did another Postdoc and learned single unit recording, same rats, same hippocampal structure, but different electrdoes and devices and recordings, and again for a year and a half I was thrown for a loop. Although confidence was shaken, I knew then that it could be worked through and had to refind people to talk to, and rethink what variables of the surgery and electrode construction I could alter. In the end there it was not the surgery, but learning to plate the tips of the fine wire electrodes (12micron diameter wire), and plate in such a way that the tips would hold the plating, and to test out the now lowered impedance for cross shorts, and stable lowered impedance.

Somewhere along this process, my favourite quote has become...

"Research is what I am doing, when i do not know what I am doing". (from some German mathematician). Science as a system of thought, can be applied to the development of methodology as well.

At first everything is hard, especially if you are developing expertise in isolation, alone without anyone, but books and papers to guide you.

Your last statement indeed sums up my current experience. A month ago I just dumped a certain study after gathering enough information from current literature that it cannot be done. I guess, I should now start to ask right questions to anyone who will bother listen. Thanks, a thousand times!

Some disciplines have dark hidden secrets. These are not always deliberate attempts to keep the outsider out. Many times it is naively assumed that everybody has come through the exactly the same educational system and syllabi, and therefore knows all the research methods, statistical techniques, and the full history of the field.

A few thoughts.

1) I note that the title uses the word false not wrong. So if I do an experiment that shows some effect, and someone later shows that my experimental technique was flawed, does that mean that the observed effect was not true? Alternatively, 95% of the research is false, but health care outcomes improve and we grow more food on less land than we did 400 years ago. Thus there may be cases where the hypothesis was correct and the conclusion was correct, but the experimental design was not sufficient to prove the connection that is being claimed. Right for all the wrong reasons.

2) What do you mean by trust? I would argue that at the time the research was done the people doing the research did the best job that they were capable of doing. However, consider the nature of statistical tests. Given "this outcome" I calculate that the probability of observing "that outcome" is less than 1 in 20 if I assume that the null hypothesis is true. I therefore conclude that the alternative hypothesis is true. Yet there is always the possibility that I am only seeing that 1 in 20 chance. There is no absolute truth in this game even if you only conclude that the alternate hypothesis is true once in 200 million tries.

3) If I look in the library and every article that I look at reports three statistical tests, and I look at 20 different articles, then it is quite likely that I will find at least one significant outcome that is a result of chance. Look at the arguments for multiple comparison procedures, especially the problems with the LSD test. Take that same logic and apply it to entire manuscripts and to all the papers in a library of your choice. In this problem, the only way to weed out the bad apples is to have independent labs repeat experiments, and do this several times if possible.