Perhaps more video methodology journals are needed. With a few more options for researchers to cite, it will allow for a higher replication rate.

Certain limitations could be researchers being reluctant to display their proprietary tools in experiments to be recorded. Perhaps companies could provide more video instructions for the kits they provide and methods even?

Karen, very timely question and potential discussion, especially in light of the new PNAS paper on new statistical standards. That's a whole other discussion, so it's good you focused the question on one issue that could improve reproducibility. These are just some random and rambling thoughts:

One thing that I think everyone responding here should consider is this: Scientific inquiry is done in many settings, not just the laboratory where data collection and lab work is more amenable to video monitoring. In ecology, observer may be observing animals at far distances through binoculars or trudging through harsh habitats trying to keep an eye on a bird foraging in the high canopy of a tropical rainforest. Or, an observer may be surveying animals and may detect vocalizations of a distant animal that cannot be detected by either microphone or video recording.

For phenomena that occur faster than we can observe them, there’s no doubt that videos can improve data accuracy. I think the main issues are how people use the videos once they’re taken. Though data storage constraints are becoming less of a problem, many practitioners may be disinclined to keep the terabytes (plus) of video data. Video backups may be the only way of proving that the later coded data is accurate and precise. Inaccurate video coding was one factor that ultimately led to resignation of Professor Marc Hauser from Harvard:

http://tinyurl.com/2w82t9k

So, even in the presence of videos and video backups, the published work is only as good as the transcription of the phenomena of those videos. Without videos, raw data is equivalent to those data taken by a worker sent to the field with nothing more than a pencil and a piece of paper. That is to say, the accuracy and transparency of methods is only as good as the subjective observer allows.

My final take: Videos may help improve reproducibility of some results, if these are may available to the public in unedited form. Publishing the video from which published analyses were made encourages precision of measurements. However, the kind of data that will allow estimation of error (for accuracy assessment, etc.)--such as observer ID, weather, other ambient conditions that may lead to measurement bias--may not be captured by video.

Rather than going the video route for most studies, I think other options may be better in terms of improving accuracy and reproducibility. One of these is registration, in which scientists (including evolutionary biologists, ecologists, social scientists, medical researchers, etc.) register their methods and lab notebooks in real time. These wouldn’t need to be published immediately, but they do need to be logged with time stamps, so that future readers could reproduce the methods exactly. The same certainly could be done for video recordings and subsequent transcriptions.

Karen i guess this is true for many biological experiments and video option can be an excellent supplementary material. In the field of my interest radioecology and climate changes issues am not having this specific problem and sample extracts are always there, internal standards and triplicate samples often provide the confidence in analyses.

For biochem/physiology experiments, the problem with reproducibility can generally be attributed to:

1. Lack of exact/precise methods. Most papers contain enough details for replicating RT-qPCR, but not exactly how the cells were treated. Treatments are generally stated as concentration, leading to large variances in volume (for example, 1.5-3mL/well in 6-well plate). Not as much of problem for animal experiments, where absolute mass/volume drug per body weight are always listed.

2. Cherrypicking data. This problem is especially severe with data from microscopy or any subjective scoring/counting/thresholding. Rarely do methods clearly state both precisely how something was done AND why it was done that way.

3. Paper reviewers fail to catch poorly performed study. Too often reviewers ask for more experiments, yet they do not notice if an existing experiment was improperly ran or the data analyzed incorrectly. Also, with there being countless journals, even a poorly performed study can be published in in some low impact factor journals.

It's a nice idea, but IMHO won't solve much. If one will want to keep something secret/inaccurate, they can simply cut the video etc.

Making a clear video of all aspects of every study is virtually impossible and would be a waste of time and money.

Asking a competing scientist to send a video of the aspect of the study you are interested in could be a useful alternative.

If he or she is not willing to do that you may draw your conclusions.....

Sometimes I also got problems to reproduce some published datas. The biggest problems are in the filed of microscopic analysis. The most analytical programms (ImageJ,..) are able to run with macros. Why isn't it possible to publish those macros and also microscope settings? I know that it would not be possible to use this settings 1:1, but finally you got a direction.

Most methodic parts in papers are often incomplete or not explicit. So, why isn't it a duty to attach detailed protocols, with buffers,... in a supplementary data part?

I think the general problem is that detailed protocols make results challenging. But a solution could be for example to establish a good open source online portal for protocols with a discussion part. Everyone can upload protocols for disscusion anonymously and can disscuss them with other scientists. Protocols with the most potential and with subparts for modification in specific questions are than available in a standard protocol chapter. If anyone want to publishing with such a standard protocol they can link it in the publication.

Obviously this isn't a brand new idea

http://www.jove.com/

No, video is not a new idea. However, the last link I provided in the initial question was to a recent article in The Scientist (November 11, 2013) in which the co-founder and CEO of JoVE suggested that video would improve reproducibility of studies conducted in the biological sciences. It is the latter idea that is relatively new (and untested, to my knowledge).

Is video subject to the same unethical practices as with text (leaving out key information needed for replication)? I suppose so, but perhaps not likely. A scientist who would deliberately leave out critical information to impede competitors is unlikely to go to the trouble of making a video in the first place (and perhaps inadvertently reveal whatever "secret" they are intent on keeping to themselves). They simply won't bother with video and will just publish an inadequate/inaccurate text description.

In my experience, those scientists who make videos (or other detailed visualizations) to augment their text descriptions have nothing to hide. They want to ensure that others can repeat their study. Or their motivation may be to encourage others to use a new method. The colleagues I know who make videos of their methods are primarily interested in helping others (peers, students, resource managers, farmers, health-care workers) conduct and apply a procedure properly.

Can a video help others visualize a method better than a text description? I think so, particularly something new or not widely known. Seeing someone else carry out a procedure in a video can alleviate doubts and encourage adoption of a method. Also, you can see things in a video that are not usually described in a paper: How was the instrument or sampling device positioned? How much force was applied during sampling? Exactly where was the sample taken? How was the sample placed in the fixative? What does a viable sample look like? A short video can save someone weeks or months of trial and error. No, you can't show everything in a video, but you can show the critical steps that should not be left to the imagination or guess-work.

A final point to ponder: The process of creating a video to demonstrate a technique is quite different from writing a description in a manuscript. Video forces you to visualize everything and to think explicitly about whether your depiction will be clear to other scientists who want to repeat your work. I've found that shooting a video makes me aware of details that I might not think to include when writing a text description. It is that attention to detail that can help improve reproducibility.

One other: I've not read the PNAS paper on new statistical standards yet, but it sounds interesting—and likely to cause some widespread heartburn.

I guess Karen you are very right thouse who dont want to make the experiment transparent wont make the video in first place, unless they want to decieve.

Peter makes a good point about videos needing to be "open access". A few journals that publish videos as supplemental material either allow their secondary posting on YouTube or use YouTube as the primary distribution avenue. If you want your video to be seen by a lot of people, especially non-scientist end-users (resource managers, health-care workers), it must be freely accessible.

If you don't have a profit motive in producing videos about your work, free-access will be beneficial to you and your visibility in the scientific community. Google typically suggests YouTube videos on its search pages. If you have a video for one of your research topics, it will likely put you on the first search page (leading more people to know about you and your work). For example, if you google "subsidence" and "Mississippi River Delta", you will find that a video I published through USGS is the first (and only) video suggested. In contrast, links to my text documents (with no video) may be buried on page 30 of a Google search. More info here: http://thescientistvideographer.com/wordpress/how-to-increase-citations-of-your-research-articles-with-videos/

As for who should take the lead in making videos, my vote is for the scientist. More journals will probably begin offering a video-creation service but for a fee, which may be out of the reach of many scientists and students. Making a video is not as difficult as most imagine. Smartphones can shoot 720 or 1080 p video and are even being used to shoot full-length movies. You can purchase powerful movie editing apps for mobile devices that cost only $5.00 US and that are no more difficult to learn than PowerPoint. In fact, you can shoot, edit, and publish a professional-looking video entirely with a Smartphone. Here's a 15 minute tutorial: http://thescientistvideographer.com/wordpress/how-to-create-a-science-video-with-your-iphone/

Shooting and editing a video does take time, but in my experience, not much more so than for a PowerPoint presentation. Like many of you, I initially balked at the idea of video because I thought it would take a lot of time to learn and would interfere with research and writing goals. I've since done a complete about-face and now see it as an essential skill for the scientist of the 21st century, along with writing and oral communications.

There was a time when PowerPoint was viewed with skepticism as something "too difficult" or "too fancy" for the average scientist to learn or use on a regular basis. We know how that turned out.

I like the idea of video documenting certain technical aspects of your research. It strikes me as a way of not only making experiments more open and reproducible to fellow scientists, but also making science more accessible to non-scientists. It seems to fit in very well with the part apparent global move towards more open access science. I think if we approach our research with communication in mind, video creation won't take up that much additional time. I already take photos of my research with the sole aim of using it in future oral presentations. Why not add a few video clips with the same goals in mind?