I am doing a study to identify the students' reading comprehension ability about Economics & Business Administration books in English. The problem is which I use both survey questionaire and test to have more objective result because students are sometimes over confident about their ability shown in survey, and the real ability is opposite. Hence, the time limit for survey and also for reading comprehension test should be well-handeled and well-set to avoid respondents' tiredness.
Not knowing who the students in question are, I find it difficult to offer a definitive answer. Among other things, the answer depends on (1) the students' level of English comprehension, (2) the students' command of the subject matter on economics and business, (3) the number of questions administered and (4) the pressure placed on them to perform.
My best suggestion is to run pilot tests to see how much time is required for completion and then add a margin of safety of, say, 10-15 minutes. If you are concerned with overconfidence, note how long it takes the "top-scoring" test takers to respond and adjust accordingly. You may even find that the time taken to complete the test may not be so related to performance.
I hope this helps!
Dear Emmanuel Yujuico.
Thanks for your sharing. Can you support me with any materials or papers as an example? Actually, I have to do it urgently with short time limit, and I need some more info to add to my theoretical framework and make decision whether to or not to put a short reading comprehension test during the survey.
Keeping in mind the limitations about different tests and different audiences, have a look at the following papers assessing the relationship between test-taking time and performance.
http://www.ets.org/Media/Research/pdf/RR-05-20.pdf
https://kb.osu.edu/dspace/bitstream/handle/1811/54580/HonorsThesis_LeahMcDannellFinal.pdf?sequence=1
Hello,
The time you will determine in the collection phase must be analyzed in pre-tests. I recommend verfificar the complexity of text and issues to take into account the response time. Do not forget that the understanding of the questionnaire should be the respondent and not yours. It is precisely why the pre-test will help you.
best regards
Thanks for your sharing, Emmanuel Yujuico, Marxia Oli. Sigo and Luciano Ferreira da Silva.
I will have a pilot test first and determine the an appropriate testing time besides making the questionaire as understandable to respondent as possible.
First of all, pilot the test and questionnaire. The main decision, however, depends upon what kind of reading you want to test. If you wish to see whether they can alternate between dfferent ways of reading at need, e.g.skimming, scanning and reading for detailed comprehension, and mot to mention making inferences, you need a time limit, set to ensure that they cannot read all of the test for detailed comprehension, like many students tend to do. If so, those who cannot vary how they read will not be able to finish. An example of this is the IELTS reading modules. Well designed reading tests will have items requiring different ways of reading. If you just want to see what they have understood the time is not crucial, but again, how valid will this test be? If it is just a short text with a few questions, this will not be a valid test of reading.
I agree. First try to have a pre-test on a good sample and include some additional questions after the survey, to have feedback on the tiredness of the respondents. Otherwise, you can do it online and measure the respond time for each question (for every set of questions). That might help you to understand some patterns.
- Badges
- Science topic
- Similar topics
- Books