If your students are new to this I suggest by starting with a bit of structure for them. Give them a reading rubric or set of questions to answer as they analyze the paper. For example: What were the goals of the study? Why are the research questions important? Do the methods collect data that are useful for answering these questions? Do the analyses answer the questions? Ultimately, what were the answers to the research questions? What have we learned by obtaining these answers? If you were to do the study over again, is there anything you would change?

In my experience, students can quickly wrap their heads around these types of questions and learn to read papers to a deeper level very quickly. Often the biggest sticking point lies within interpreting statistical analyses--so I often have to give stats lessons and refreshers as we read different papers together.

One trick that I have learned to make sure students are better prepared for discussion is to use an online discussion forum (my institution uses Blackboard) and make the students post at least one meaningful interpretation of the paper the day before class. I make these posts part of their overall class grade.

Hope this helps.

When I think back on my academic years, one of the most meaningful and useful exercises I experienced was when our professor assigned one seminal article to each student in the class. We were challenged to read it, digest its every detail, and then present it to the class-- emphasizing the results of the study, of course. It may sound counter-logical, but by expecting us to focus on the particulars of that one published article in order to see the conclusions that were reached, it better enabled us to see through the fine points of other studies in order to absorb the importance of their findings. A secondary benefit was that, by of the student presentations, we ultimately all benefited from each others' immersions into their assignments (as if we had read each of the studies ourselves).

If your students are new to this I suggest by starting with a bit of structure for them. Give them a reading rubric or set of questions to answer as they analyze the paper. For example: What were the goals of the study? Why are the research questions important? Do the methods collect data that are useful for answering these questions? Do the analyses answer the questions? Ultimately, what were the answers to the research questions? What have we learned by obtaining these answers? If you were to do the study over again, is there anything you would change?

In my experience, students can quickly wrap their heads around these types of questions and learn to read papers to a deeper level very quickly. Often the biggest sticking point lies within interpreting statistical analyses--so I often have to give stats lessons and refreshers as we read different papers together.

One trick that I have learned to make sure students are better prepared for discussion is to use an online discussion forum (my institution uses Blackboard) and make the students post at least one meaningful interpretation of the paper the day before class. I make these posts part of their overall class grade.

Hope this helps.

Very interesting idea John Swaddle to have them post on the discussion forum in advance, as that might help enforce pre-lecture readings. Unfortunately, with 80 students enrolled, it might be challenging for them all to come up with insightful but unique contributions...

John Gramieri, I would love to do a deep reading course like that, but again, I think the size of the class prevents each student giving a presentation.

I really like the rubric idea. I've put together the following for them.

"When you read any journal article you should ask yourself the following questions

What is the goal of the paper? Objectives? Question they strove to answer?

What approach or method did they adopt?

What are the assumptions involved in that method and are they possibly violated in this case?

What are the major findings from the study?

Do those findings agree or disagree with other studies? If they disagree, what might be the reason?

What are the implications of the findings?

Obviously, the greater the implications, the more important the validity of the methods. If a study seems to contradict the other available data, double check their methods.

Data collection: Is the sample size big enough? Was it random and representative of the population or could it be a biased sample? Are replicates truly independent? Is there evidence this was an abnormal case?

Interpretation: Did they use the right statistics? Do the conclusions seem logically drawn from the results?"

Any suggestions for improvement?

Rebecca, I regularly teach a class of 75-80 and have them read primary literature at various stages in the semester. For this size of class I create separate online discussion threads for each element of the reading rubric, e.g., a separate online discussion thread for "what is the goal of the paper?" and another for "what approach or method did they adopt?", etc.... When I've done this I've often got 75%+ participation in the online discussion before students get to class. I think that chunking the assignment like this helps each student see a particular element they can contribute to.

Reading students' comments ahead of class also helps me call on students by name in a less-threatening way as I can praise a student for their post and get them to explain it more. The comments also give me documentation that I can use to judge how students' engagement with the literature is changing over the semester.

From my perspective, the most important information to glean from a publication is the set of concepts presented. Second most are the details that are critical to the phenomenon reported. Only for the laboratory practitioner is it reasonable to place strong emphasis upon methods, though an overview of method concepts is certainly proper for even the lay view.

Hi again John Swaddle. That is a really good idea to break down the components of the rubric. Structurally, I'm sure it helps them approach the paper with less intimidation, and by creating separate threads it probably provides more opportunity for students to contribute unique ideas. Thanks for the suggestion.

This approach that I am about to suggest may seem a little off the wall but, I am going to post it anyway. Breaking it down in a rubric form is a great idea. I did a little research in the past and studied the different forms of emotional intelligence. Included in this study was the different ways and or methods that people use to comprehend things. I made the connection with; how Jesus spoke in parables to the masses, which at that time was basically speaking to them in their own language. By associating the curriculum with everyday common knowledge and language as well as current events, I found that I was able to comprehend and retain the knowledge, better, faster and longer. Believe me, I can be pretty thick at times. You could also, if your curriculum includes field trips, Have your students use their 5 senses to comprehend better, (See, Hear, Taste, Smell, Touch.) As a child I spent many hours walking and playing in the woods and learned things that people take for granted. It is like scuba diving for the first time. You descend into the ocean 40ft. down and once you lose your fear and adjust the buoyancy, you begin to discover that even the smallest particle of sand on the ocean floor is alive. I do have a point I promise. I learned to use all of my 5 senses when it comes to observing and studying things. Both spiritual and waking. Hope this helps. Good Luck!

I agree with William Buckley, if you are reading a lot of different publications, you will not learn if you try to remember all the methods and data. Focus on the main concepts. Read the abstracts, and introduction sections before all the detailed data and conclusions to understand the main concepts. If the methods and details are relevant to your assignment or research, then go back after and take notes on those. doing it all in one shot makes it difficult. The main concepts are the most important to begin with....at least that is how I did it in my classes. Seemed to work well. Good luck.

Hi there! Interesting discussion overall. I teach Population and Community Ecology to undergrads in their 4th semester of Biology in southern Brazil. Apart from the English language problem, their main trouble was adopting a critical stance, not necessarily doubting the findings of the paper, but not automatically believing everything was right because it was published. So in a way I agree with the list of questions suggested by Rebecca ealier.

The way I do it takes three different approaches: first, I start the course with basic sampling theory, then organise groups and get each group to criticise a given paper (usually I choose papers with obvious problems to easy things out) but mainly considering the methods section. Then the groups present their findings in short seminars to their classmates - depending on how bad or limited the paper is, we can even get some laughing on the poor authors, which I don't support, but there you go. I guess this can get them to be more confident and less stuck about details of field/lab methodology, data handling and even statistics.

I then evaluate them with two exams also based on papers, which they can read in advance, and I propose questions which try and connect the theory they've seen in classes and what the paper presents.

But what I think works best is, again in smallish groups, doing fieldwork (usually on a single day, with sampling theory in mind and leading to real data), and then getting them to WRITE a paper on that. I do explain all the way what kind of information needs to go on each part of the paper - usually one class for each part (Intro, Mat&Meth, Results, Discussion), or even two classes for Mat&Meth if there are problems with stats. So it takes a long time, but being on the "other side" of the process of reading a paper really opens their eyes...

I'm not a teacher or anything, but the key sections of any article usually are abstract (in some cases partly - introduction) and conclusion. One could guess that they are designed especially to keep in mind the punchline of the article itself, and to make the writing more logical and understandable for others, while everything else gives just some more detalization.

You could probably make that point for them: first read abstract/intro and conclusion -> get overall look into what's to be discussed and what is stated; then probably search for some background material elsewhere, if there is a lack of understanding (e.g. some models used or unfamiliar scientific terms - if their explanation is not present in the article body); and in the end (if it's needed) get to the article body to sort out particularities.

Slide notes from my lecture on this - just a personal opinion though

My personal suggestion:

Title

Abstract

Both of these can be found from databases….

You should now know if it would be useful to read the rest of the paper

First Paragraph of discussion (normally summarises the results in plain English)

Introduction (especially to find other appropriate literature through references)

Discussion

Often you don’t need to read the methods and results

But reading these sections will help you write papers, and evaluate the science in the paper

I am inclined to agree with you Richard. A little extra research goes a long way. It gives a little more insight into the topic of discussion for the writer and adds more depth and interest for both the writer and reader of the paper. Does this make sense?

Crickets, Hmmmmmmm???

how about the classic "How to read a book"

http://www.amazon.com/How-Read-Book-Intelligent-Touchstone/dp/0671212095

I think that is a helpful suggestion to tell people to focus on opening paragraphs, as in well written paper these do summarize the section to follow. Definitely I want them to read the methods section, though. I worry that if we teach people to skip the whole methods and results, they are not learning to critically assess the papers, but rather just to accept the published conclusions. Is there no danger in that?

In truth, critical assessment of methods comes only upon attempted application of same; reading about them is likely to provide an incomplete basis to judge value, utility, and suitability for any particular purpose. Indeed, this is why we have laboratories.

Hi William, perhaps in many disciplines, but as a field ecologist, there is very little I do in a controlled laboratory setting. For someone to independently repeat my results, they'd need to sample the same ecosystems. For an exact repetition, they'd need to do it at the same time, as inter annual variability is huge. I think for my students to critically read conservation biology literature, reading the methods and results sections with an eye to sampling adequacy, appropriate testing of assumptions, and methodological logic is enough. At least, we can't really expect more. It might be a great lesson for them to try to repeat a modeling study on their own, however.

In spite of such limitations, appreciation of methods is still best achieved by attempted application of same, and especially where statistical methods are strongly applied, it is not so difficult to gain a broad appreciation even if specific observations make difficult attempts at experimental duplication.

Let's make a radical suggestion, that you will best accomplish the teaching goal if you get the ego of the student involved. Wrap your goals around a motivator, and turn them loose. Something I should think that is akin to giving sugar to toddlers.

I agree with William. I've never taught in a formal setting, but I have some strong memories from having been taught by great teachers. I had the good fortune to be taking out of the classroom several times and, as class projects, be expected to re-enact the methods and materials of those who had long-before published their findings. Mimicry can be a powerful tool in the learning process and, seeing that broad, scientifically-sound conclusions can be determined by relatively mundane actions is memorable. They don't have to reach exactly the same findings-- they need only see that they can "prove" something via the process. It worked for me and it has allowed for me to understand the scientific process-- even as I read someone else's work (which gets back to your original inquiry).

The point is that without hands-on time, reference to hands-on activity are less meaningful than they otherwise might be. Further, though experience truly comes only through the extension of time, even the little bit of time afforded in a semester or quarter is better than no time at all; the fumbling student in the lab at least fumbles in the right place. A gain in appreciation by time in service has always been recognised (by some institutions, like CalPoly) as an important part of total education. If you want your students to get the most out of methods, you need to give them appreciation for same and that can little come from a reading of papers by neophytes.

I think that hands-on experience is where labs and other assignments come in. In this class, for example, I have a lab where they calculate biodiversity indices (alpha and beta) from different wetland plant communities, some of which are from a National Park, some have been restored, and some are heavily grazed. This assignment makes them think about what all those indicators and estimators actually measure and what those values imply about the success of restoration efforts or the impacts of grazing on plant community richness and evenness. I think they learn a lot from the exercise and hopefully when they read a paper that reports a "significant increase in diversity" they will question exactly what that means and how it was measured. Definitely hands-on experience is essential, but my question was about critically reading the literature. I guess within my teaching philosophy I see students needing the ability to evaluate and interpret a paper even when they cannot attempt to repeat its methods.

This thread of discussion began with the concern of how to read a paper. A big point that I forwarded is the difficulty of inexperienced persons to understand the implications of subtle details of experimental method, and that it would therefore be that the most suitable expectation from such students is that they should understand the forest (concepts) even if they should by this approach miss a few trees (methods).

The question then turned to justification for not expecting students to concentrate upon methods, and thus the last post by Rebecca. In concluding my contribution to this discussion, I review the points of this last post.

Neophytes will not generally catch the implications of method without guidance, having no *feel* for the materials used, the subtle and touchy character of some sampling and measurement techniques, why it is that a particular molarity of solution is critical to observing some biological phenomena .... this list is endless.

Further, students have only so much time to devote. The more you fractionate your instruction with important detail, the more you will burden your students, especially having to determine which information is truly more important and which can be minimally attended.

The goal of teaching is no different than is the goal of cooking - digestibility and appeal. Too much gastronomic variety spoils the meal, and while perhaps a bit disgusting as a metaphor it is never the less apt that (as with birds) predigested meals go down quicker.

If you feel that certain points of method are important, make sure you emphasize these points in your lecture; don't expect your students to get such points out of the paper, solely by a reading of the paper.

Dear Rebecca

I use a methodology that encourages the student to participate in the reading.

The article is distributed a week before discussion and in the discussion's day the students will be aleatory chosen to opine on the article

This is a critical reading where some details are discussed,by the group:.

Who wrote this article? The author belongs to a research institution? What?

It is a relevant article? The methodology was good? The results were discussed as well? The conclusion is related to the objectives? What is the level of evidence of the article?

This makes the student not only read the article, but also a critical analysis of the impact of that Article