The PISA test meets criteria for quality, reproducablity and rigour, and does indicate the general educational standard of those students tested in that country (although there is some question of western bias) However, does its limited value outweigh the risks. All high stakes testing runs the risk of negative impact on  student psychology. There are other ways to test the general educational standard of a country that do not impact negatively on students. One of the major problems is the ego driven education policies designed to raise a country's position on the international table, not to improve holistic outcomes for children. 

Hi,

The Program for International Student Assessment (PISA) is an international assessment of the national education.  It measures 15-year-old students' reading, mathematics, and science literacy. It also includes measures of general or cross-curricular competencies, such as problem solving.  It emphasizes functional skills that students have acquired as they near the end of compulsory schooling. It is coordinated by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries, and is conducted in the United States by NCES.  This assessment was first administered in the year 2000 and is conducted every three years. The most recent assessment was done in the year 2012.

The PISA 2012 assessed students’ mathematics, reading, and science literacy. It was also included a computer-based general problem solving assessment and optional computer-based assessments in mathematics literacy, reading literacy, and financial literacy. In U.S. 15-year-old students participated in these optional computer-based assessments.

Around 510 000 students between the ages of 15 years 3 months and 16 years 2 months completed the assessment in 2012, representing about 28 million 15-year-olds in the schools of the 65 participating countries and economies.

This method of asscessment is good.

for further information kindly see the attachment link:

http://www.acer.edu.au/ozpisa

Thanks.

Dear Mark Gould,

thanks a lot, your vision on student impact is very interest, and I did not think about it.

thanks...

I speak as a specialist in literacy and one who recognises literacy as a social practice. From that point of view, the PISA test is simply a conformist instrument which allows policy makers to neatly categorise and classify people. Think of it. Is there only one form of literacy? Can those who claim to be 'highly educated' compete with the specialist literacy knowledge of 'less educated' artisans? The ISA test is constructed to fit only one mode of literacy and as such can not be an objective measure of actual literacies. Easier, more streamlined, but subjective is my view

It is an interesting opinion the negative impact on students' phycology, but i believe that students (at least in Greece) do not care about Pisa's results. A similar question could be the negative impact on peoples. Citizens of a country that always be on the bottom of pisa's scale could have negative phycology about their education policy or their personal skills.

There are some other points to analyze.

The first point is the statistical ensemble that is taken in each country. In some countries a preselection of schools and their pupils is taken in a way that the outcomes of PISA tests are certainly significantly different from those that had come out if a representative cross section had been taken. Such results may not be seriously compared to results in a country where a representative cross section is taken.

The second point concerns the criteria on what a pupil in a certain class should be able to know and to do. Having participated as an expert for the engineering strand in the AHELO-project (which is seen as PISA for students at universities), I am not sure whether these criteria are really optimum there. Therefore, I doubt whether the PISA criteria are really optimum.

The third point concerns comparability of learning success at a given age. Curricula and learning progress on a time-line may differ significantly from country to country, which may be due to cultural characteristics. Even if in the end, i.e. after successful finishing secondary school, learning objectives of certain subjects may be comparable for different countries, over-all comparability may not be given at the age of fifteen. (In my country, some types of secondary school are usually finished at an age of 16 years, while other types are finished at an age of 18 years. Depth and broadness of learning objectives may differ considerably, therefore, at an age of fifteen).

Consequently, I doubt whether PISA results are scientifically valid in all concerns. However, they are a first approach towards comparability. From that point of view, they are immensly important, and OECD should further develop the processes.

Concerning the original question of Mariano Pierantozzi: At the moment, PISA is of limited value for comparing educational systems. However, it might turn into a good tool, if a serious effort is done to improve it.

I think it is also important to remember that no one tool can measure the total impact and success of an educational system.  Countries have different values and purposes for public education.  Those values drive how they design their programming.  Reducing a country's success to a few subjects and one exam can be very damaging.  However, as one data point, I think the scores can provide some interesting information.  I do not believe it can give us the total picture. I fear that too much emphasis on the quantitative comparisons of one country to another using the PISA scores can end up narrowing curriculum and scopes of purpose for schools.  We are certainly seeing a narrowing of focus in this country based around our current accountability systems that preference math and reading.

Gordon Ade-Ojo's point is a very important one. 

PISA measures analytic capabilities which are strongly influenced (due to its origin) by the mainstream Western "scientific" paradigm. PISA measures skills that are relevant particularly in urban and industrial (including all the more recent encarnations) settings. It disregards some of the gravest issues that threatens the world, particularly the gulf between the rich and poor, which can and is being exploited by radical groups. 

Now, even when such international implications are not considered, rural settings require additional skill sets that are not dealt with by PISA tests such as hands-on artisan skills as pointed out by Gordon.

If you have ventured in the Amazon jungle (not as a tourist) through its constantly changing landscape of  large and small rivers, you know that without the guidance of experienced natives you would not survive more than a couple of days there. Such valuable skills and wisdom their inhabitants possess are not captured by PISA tests. 

A very insightful book by Andreas Fuglesang and Dale Chandler, Participation as Process (1986), argues that the poorest of the poor also have important skills such as the ability to listening to others or memorizing things and facts. Survival skill is also a must, and efficient use of resources is second nature for them.

PISA, in my view, is an excellent measure of which countries will dominate (or are already dominating) advances in industrial productivity, particularly in manufacturing and services related to industry.

I do not want to be ironic, but It might be a good measure of the extent of standardization and globalization in the world. The larger the number of countries that participate in PISA and strive to perform well, by adapting school curricula so as to ensure students perform well on PISA, the more standardized will the world become. Who benefits from such a trend? Certainly not the poor, not the marginalized ethnic minorities, not the aboriginal communities.

I am not arguing that PISA does not bring contributions. It does bring benefits for those who can enter the "mainstream" world economy. But certainly it is not an exam that should influence the educational systems of many countries, particularly developing ones. These countries, along with other countries which value diversity, should be wary of being overly influenced by PISA standards.    

Thank you very much, Dr. Kondo, for your insightful contribution to this discussion from a multi-cultural perspective. I wonder what Hofstede would have to say about the tacit assumptions behind the PISA programme ...

http://www.amazon.com/Cultures-Organizations-Software-Third-Edition/dp/0071664181/

First of all, the educational system in certain countries are so different, therefore it is hard to compare them. Also, the choice of tasting group can be seen as a problem.

But, what is the most interesting to me, is that PISA tests results do not corresponded with the scores from our national tests. My country – Poland is located on the quite high place. The national reports say that we achieved success. After information about the PISA results, many people in Poland announced success. Aa an example might be a quote from one of the most important Polish newspapers - Gazeta Wyborcza: “We have reason to be proud and satisfied. Our youths are the most talented in the world. About that my generation could only dream of - said Prime Minister Donald Tusk” (Gazeta Wyborcza from 03.12.2013). The 2012 scores are better than ones from last years and give us a place in top of the UE.

But, there so many voices that young people in Poland have big problem with the mathematics skills and with the understanding of written texts. Last year, about 30% students did not pass the high school graduation exams. To pass these exams is necessary to go to university or college. It seems that good score on PISA test of Polish students and the reality not always go together.

In addition, the practice shows that the situation with Polish students is not very good. Many people working in school say that the level of knowledge and level of different skills is going down.

Now, we can ask the question about where is the problem. I mean – are the groups for PISA test are chosen inappropriately. Or maybe, the PISA tests are too easy? Or maybe, our national exams are too demanding or too difficult .

I do not know the answer to these questions – but we should thing of them and try to find the answer

Artur, another possibility is that the PISA tests, because of their style, only effectviely test decontextualised knowledge, so your students can be good at learning the basics, but are not being taught well to apply the knowledge in new situations. this would show up as lay people complaining that they do not seem too know as much as they used to. it is similar to the complaint that university professors know a lot but can be deficient in common sense. An outcome of too much emphasis on 'the basics' can have this result.

Point well made Mark. It isd the essence of the social construct of understanding literacy. Literacies must be contextualised and als, PISA is uni-contextualised

All international (and indeed national) comparisons run the serious risk of being totally flawed.  This is because variable control cannot be achieved.  Every country, (indeed every school and university) has its own conventions.  When it comes to assessment, the types of questions asked, the way they are asked, the subtle emphases in curriculum and way teachers function ..... and so on....all these vary in ways that cannot be quantified.  International tests essential measure recall - it is a sad fact of life.  Indeed, the countries with the most rigorous procedures that encourage efficient recall will always do best.  Some are better ways for education, some are very clearly not.

It is interesting that the way students gain correct answers can sometimes be utterly baffling and outwit the best intuitions of test-setters.  Just try interviewing a sample just after they have been tested.  Or, even better, let them 'talk through' their answers as they answer.  We can be amazed.

One year, I analysed the questions set in an international test.  Most were structurally flawed.  I then took these test items and mixed them up with test items devised by practising school teachers.  I gave them to conference delegates in a workshop and asks dfthem to 'shred' the questions.  They did not know the sources.  With unerring accuracy, they applauded the questions set by teachers and found the flaws in the ones from international comparisons.  This is not because the setter of the latter are foolish.  It is because, in their attempts cover every possibility, the questions became so contorted and flawed.  They were trying the impossible.

Therefore, my advice is to drop all comparisons into the nearest waste bin.  They make for good fodder for the media.  They can destroy excellent educational systems and great curricula quite apart from misleading politicians.

@ Norman Reid : Your experiment should be run all over the world. May be an eye-opener for some, encouragement for others, not to remain silent in the face of evident absurdities of/in the system.

I have found the various contributions interesting and generally revealing all kinds of weaknesses in international testing.  Other than the assertion that PISA tests are good - which is simply an assertion - all the evidence suggests that we should stop doing these kind of comparisons.

An interesting question to ask is WHY carry out such testing.  Politicians in countries that do well applaud themselves.  Politicians in countries that do badly blame the teachers (who do NOT decide curricula, assessments, resources or whom they teach).  

Education is a CO-OPERATIVE exercise not competitive.   Let us aim to help everyone and not impose spurious companions that can seriously damage education provision.  I can find no evidence that test data actually improve anything.  They do not point to clear ways forward.  Properly conducted research does that!

Norman, I agree with most what you have said. Nevertheless, I find it interesting to collect data for purposes of comparison - provided this is done on a scientific basis (and this is one of the weak points of PISA).  In my country, education is regulated by the states of the federal republic. Therefore, PISA has helped to reveal that some of our states are doing it better than others. Though nothing has changed yet, the public has now discovered that there is something to change!

Your are completely right: until now, consequences of PISA are poor, but changing the course of a heavy-weight ship has never been a simple task. Let us be patient, but insisting to change to the better...

First of all, we should bear in mind that the main purpose of PISA is to identify the features of the educational system  that contribute positively to student performance. Therefore collects information about education systems, society, economy and other variables. Comparing countries'  performance  we can locate these positive characteristics, which is good because policy makers have the necessary information for future improvements of their education systems. If something is problematic are probably 1) the definition of literacy (personally I think is the best and most complete definitions of those I've read) and 2) how the definition of literacy assessed through  questions of PISA. Now, how the national policy makers use PISA's data is another issue. As far as it concerns Greece, policy makers don't take into account the results of PISA and politicians justify the failure of Greek students to the fact that PISA concerns other educational systems. The newspapers although blame teachers.

 Dear Ingunn,

Thanks a lot for your suggestion.

I'll read it very soon,

mariano