I hate international testing! but only because it as been politicised, not because of the testing itself. if the testing was simply used by educatonalists to indicate which countries were doing something right in their education processes, and then to research exacly what that MAY mean for one's own country, it would be alright. unfortunately, PISA testing seems to be used to make systemic decisions base on merely superficial analysis of PISA data.

Another issue is the multifactorial nature of quality in education. What constitutes a successful system? One in which there is a widening of the educational gap, where a moderate proportion of students do very well, and a largish proportion do quite badly, as in Australia, or a system where there is a narrowing of the gap and a general upward mobility of attainment. Or is it possible that systems exist where PISA data is not outstanding but still a reasonable proportion of students continue to be involved in both formal and informal education and society is in general successful and happy. Another issue is the disparate nature of PISA data. Both Hong Kong and Finland do quite well but probably for an entirely different set of factors. Which system should your country follow?

What I would like to see from PISA data is a concerted effort by researchers to describe a more or less common set of attributes that a system should have to maximise the full range of outcomes and eliminate issues of inequity as currently exists in the Australian systems.

In other words PISA is just data. to make use of it requires research, common sense and deep thoughtfulness, which unfortunaately are often missing in political decision making.

Miranda, I was hoping others would respond to this as I think it is an important issue. Since no-one has yet, I will continue:

What is the time scale of the change in your country's change from high achieving to low achieving? Is it really likely that teaching has suddenly become so much worse that students en masse are less well educated? I hope that there isn't a knee jerk reaction to these changes because from my observations this is rarely a positive outcome. I hope your government uses the data to establish think tanks to examine the range of possibilities and options for change over a longish period with intelligent terms of reference to prevent such a knee jerk reaction.

Thanks Mark. We wait for others to respond. But sometimes I see that RG members do not respond to matters that are important; that are also within their expertise. So we just wait and see.

Back to the question, I have always noticed that our respondents are very important to our research and so our choice of respondents is critical. At a convention that I attended last week, Dr Z from our exam board presented a paper on school-based assessment. She gave us info on our teenage students' recent performance in TIMMS and PISA. She also mentioned that this performance may influence foreign investment in our country.

During the Q&A, I put forward my view that our previous performance was much better. I mentioned that if we are looking for disciplined students, we can easily find them in the many Convent schools throughout our country. I asked her, from which schools the recent respondents were selected. But she said she wasn't able to answer the last question. Thanks again Mark, hope to hear from you and the others!

No serious answer can be given if one doesn't take into account "what are pupils taught in a certain educational system?", i.e. there is no value in asking, for example, PISA-style questions to pupils that are taught to reside definitions and theorems...

Even so, any set of any style of questions that are used to "test an educational system's efficiency" should first be adequately labeled as to "what these specific questions check" among the possible "knowledge factors"...

On the other hand, yes, in Greece governments use PISA outcomes at will, whenever they're about to dispute teachers and professors... but they remember the "Mathematical Olympiads" whenever - usually before elections - they want to speak of a "successfully reformed educational system"... You see, greek pupils often do well in Olympiads (along the form and style of Math they are being taught), but badly in PISA!

P.S.: it is, therefore, amusing to add that when I checked my 7th - 9th school-year pupils of the Mathematics-club upon their efficiency on PISA 2003 subjects they - surprisingly? - did quite well, I must say! ;-)

Thanks Athanassios. We wait and see what other RG friends have to say. I believe many countries are concerned about TIMMS and PISA :)

As for Germany, where I taught in during the 1980s and again about 4-5 years ago. I think that the benefits of PISA findings included the fact that many Germans and their educational institutions had done well through the 1970s and 1980s well in the great variety of educational models in practice in that country. (There were at least 5 major different models of high schools functioning at the same time across the land in the public school secondaries.) However, by the 1980s and 1990s, a great percentage of Germany's were not coming out of the public schools and training with a greater variety of skills as is required in fast changing European labor and educational markets today. There was also a clear class and ethnic bias in the system--i.e. minorities and foreign born peoples were being hurt more and more by the status quo.

Then along came PISA.

After poor results on PISA circa 2000 in Germany, that country began to try to provide access to a more important variety of models for serving the greater majority of students than ever before. This led over the recent decades to greater variation in teaching, learning and educational models at all levels--not just at the secondary but at the primary and tertiary levels, too.

Sadly, like in India and elsewhere, the governments of Germany have often relied in that land too much on the private sector to weed through the humongous PISA data and to show the way for the country. This has led to a great injustice upon leaving high school for many German students --whose parents never new the debt of higher education that this generation is now paying for across the land. Education is getting more costly--practicums are often still hardly paid for--training and access to good training and helpful advice and directions for students is still lacking.

In short, data is data and what one does with it is one thing. How to collect more helpful data is even more important for the future. However, never rely on one instrument for your policy or for making major decisions--PISA by itself was never enough to assist in educational reform.

First, I apologize to Kevin, for late appreciation. I really wasn't aware. I agree entirely:

'How to collect more helpful data is even more important for the future. However, never rely on one instrument for your policy or for making major decisions--PISA by itself was never enough...'

Thanks to Jose, for sharing experiences of implementing PISA. Please post anytime! The info is going to benefit me, and my community.

PISA data (also TIMSS data) provide an extremely rich source of information that can be used for many different purposes depending (for example) on your interests and also depending on the level of statistical and analytic sophistication you can bring to bear.

Currently I am using a very simple kind of data - percent correct data on a selection of released PISA items - to investigate the kind of mathematical thinking and knowledge that the items demand, then comparing the extent to which students in different countries are able to correctly answer those questions. It might be relevant to classroom practice.