I think you will need to get in touch with Cihi to get the answer to your question.

No body can deny the vocabulary size effect on passing the exam like TOEFL, but the second effective is the test takers test taking skill. With skillful learners, I had observed that 4000 vocabulary size had been successful too. But I did not considered how many were word family. Good Luck I am following to get more scientific response for this question.

Shhram, If you look at Cihi"s paper (http://www.cartoonwordpower.com/research/pdf/conversion_methodology2013.pdf), you will see that people who had a vocabulary size of 5,902 got TOEFL pbt scores ranging from 380 to 595 with a mean of 502. If you look at the graph in the paper, it also suggests that people with vocabulary sizes of between roughly 4000 and 8000also got TOEFL scores of 502. Clearly the figures presented show that there is quite a large margin of error in these estimates, in spite of a correlation coefficient of 0.81, or 66% shared variance.

It is difficult to find information covering the PBT TOEFL word list specifically. Chugo and Oshigian (2009) studied word frequency on the TOEFL PBT and the iBT regarding the usage of lemmas in the Standard Vocabulary List (SVL) and found that 6,242 lemmas or 5,000 word families were needed to account for 95% of word frequency of PBT practice tests. Kaneko (2015) studied vocabulary for the iBT listening section and found frequent usage covering 1,000 word families yielded an average of 85.09% in total in the iBT. This study suggests that a 6,000 lemma-frequent level is adequate for the iBT reading and listening sections. Although these studies do not pinpoint the actual number of words lemmas needed for an actual PBT score, they do examine the words necessary for a specific percentage of words appearing in the tests. These percentages could be shifted into an appropriate corresponding score range on the test to determine the necessary level of word lemmas. For example, if 6,242 (number of lemmas for a 95% word frequency) multiplied by .11 (lets call it the fluctuation factor) yields a score of 688 (or roughly 95% total PBT score), then reaching the word lemma necessary for a 550 score would require dividing the desired score level of 550 by .11 yielding a lemma of 5,000. This is an example, but information can be transferred from one scale to another using a mathematical formula.

Dear Martin, Shahram, and Jeff,

Thanks so much for your responses. I have read Chujo and Oghigian (2009) recently as well, so that reference was very fresh in my mind. If a 550 TOEFL ITP or PBT can be achieved with 6,000 + lemmas or 7,000 tokens, that makes the job much more doable in the EFL university context with tools like Word Engine--thanks Guy Cihi. I have been frustrated with some measures like the VLT in regards to vocabulary size estimations as we assume that Japanese students are very good at process of elimination in multiple choice formats which Gyllstad, Vilkaite and Schmitt (2015) recently showed to result in VLT overestimates of 11-26% in vocabulary size, so if you estimate 4000 words with VLT they might only have 3,000 words... From my research, I see that Japanese college students with a 415 TOEFL ITP might have recently added 500 + words from Word Engine and have 3,500 words total, but sadly, our first year students often carry a 37% deficit with regard to the most frequent 2,000 words on the GSL, so in spite of posting impressive VLT scores, are victims of high school overreach and the continuation of that overreach at university. So, anyways, my thinking is that we cannot assume that if we read a report that says the student knows 4,000 words, we can assume these are the 4,000 most frequent words... Every system that uses estimations results in gaps at the individual level... Maybe I am being overly cautious? But what if we were to look at the most frequent word knowledge of those students who get a 500? I am thinking that individual high frequency gaps might account for those who have high scores on vocabulary size but are on the low end of the scatterplot results---assuming other skills are equal... The other problem of course is that students maybe get lucky if their vocabulary happens to mesh with a particular test version--also an important variable. Anyways these are just ramblings from me trying to catch up with everyone. Sorry for the delay in posting and thanks again for your interest in my question.

http://www5d.biglobe.ne.jp/~chujo/data/6-2-06_Kiyomi_Chujo.pdf

http://www.norbertschmitt.co.uk/uploads/pdf-(463-kb).pdf

Tnx Paul it was very useful.

Paul,

I think we need to be careful in the way we interpret studies like that by Gyllstad, Vilkaite and Schmitt (2015). Basically, they are claiming that their test is an accurate measure of vocabulary knowledge and that, because the VST does not give exactly the same results, it has a problem. The problem with this is that the two tests are actually measuring different abilities. The VST requires testees to match a word and its meaning (or translation equivalent in the case of bilingual version), while Gyllstad et al’s study requires them to produce an “L1 translation equivalent or definition, L2 synonym or definition, picture, gesture)”. My feeling is that the matching task is more sensitive to ‘shallow’ vocabulary knowledge than the ‘L1 translation, etc.’ production task, which is why it is easier and gives higher scores (especially with a bilingual version of the VST). The much less sensitive productive vocabulary size test gives vocabulary size estimates that are probably significantly lower than those found by Gyllstad et al. That does not mean it is an inaccurate measure of vocabulary knowledge but simply a different measure of vocabulary knowledge.

Of course, testees guess and reach answers by elimination. If you feel this is a problem, there is a formula for correcting for this: adjusted score = raw .score – (no. incorrect answers x 0.25).

To pick up on another point you make, I’m afraid we cannot assume that if a student knows 4000 words those words will be the most frequent 4000. The VST is organised by band, so it is possible to do a kind of profiling to see at which frequency band vocabulary knowledge dips or starts to drop off. This suggests you could use it as a diagnostic test to help you decide what vocabulary to introduce, although, with only 10 words tested per level, it could only provide you with very approximate results.

Finally to muddy the waters a bit, Bhatia Laufer (at Eurosla a couple of years ago) suggested that the Japanese version of the VST produced an overestimate of vocabulary size because cognates were over-represented in the test.

Dear Jeffrey, In any testing situation, practicality is a very important consideration and compromises have to be made. I think it would be rather difficult to measure the vocabulary knowledge of large numbers of students in the way that Gyllstad et al did.

If you don't like the receptive vocabulary size test, there is a productive version and it should not be too difficult to make a parallel depth of knowledge (VKS) test (although it would probably take a long time for the test-takers to complete it).