Our PISA results made us depression in 2013. Each of strategic organizations of education, including ministry and its different background offices published their ideas on what we should do to rise our students achievement. We have National Core-curricila since decades; teaching books have been changed for being in line with NC;  teachers were attended to different courses relating to competence-based education but nothing changed significantly in results since 2006.

Some researchers pointed out, while teachers' awareness towards their students' keycompetences will not be changed spectacular improvement cannot be expected. The problem is that, teachers in their lessons  mainly deal with theoretical part of the curriculum and have not enough time for improvement of their students' competences.

Why teachers do so???? Only one answer among many else. They feel stress due to the final exams of secondary schools because the results of these exams generally mean enrolling points for students to be attended the universities. Final exams at secondary schools, have two levels in my country. To pass it at higher level means good chance to choose high- quality universities but this type of exam is based particularly on theoretical knowlede.

(The basic level of final exams is addressed to students who would like to continue their study at colleges or not too high-quality universities even want join in vocational trainings. )

Our PISA results made us depression in 2013. Each of strategic organizations of education, including ministry and its different background offices published their ideas on what we should do to rise our students achievement. We have National Core-curricila since decades; teaching books have been changed for being in line with NC;  teachers were attended to different courses relating to competence-based education but nothing changed significantly in results since 2006.

Some researchers pointed out, while teachers' awareness towards their students' keycompetences will not be changed spectacular improvement cannot be expected. The problem is that, teachers in their lessons  mainly deal with theoretical part of the curriculum and have not enough time for improvement of their students' competences.

Why teachers do so???? Only one answer among many else. They feel stress due to the final exams of secondary schools because the results of these exams generally mean enrolling points for students to be attended the universities. Final exams at secondary schools, have two levels in my country. To pass it at higher level means good chance to choose high- quality universities but this type of exam is based particularly on theoretical knowlede.

(The basic level of final exams is addressed to students who would like to continue their study at colleges or not too high-quality universities even want join in vocational trainings. )

In the Kingdom of Bahrain, the PISA results as well as TIMMS have led to a national educational reform initiative in the country. Because the results were so low in comparison to other countries in the world and other countries in the Arab region, the government decided to do something about this and the result was the introduction of educational reforms in K-12 schools that focus on everything from school improvement projects to teacher training and educational leadership training, etc. 

Dear NINA, do you know how the results of this arrangement can be seen???? Please explain it @

I my thesis I am examining how the achievement gap between low, medium and high SES groups changed during the time in PISA, and what economic, political, and sociological factors have been determining these changes in each PISA attendant country. The final product of the statistical analysis of my research will be suggesting some country specific strategies educational policy makers needs to consider in each country. I have selected this topic for my thesis because I have been observing a continually increasing attention of policy makers and academics on PISA from many countries since I have started my PhD. From literature we know the pressure of effect PISA results has created in many European countries such as England and Germany, and it has spread to other countries like the US. It is not hard to oversee that OECD will continue shaping countries educational policies through the PISA at least for another decade not only because of the its success (even though this is still debatable in some countries), but also because of its intelligence in creating new measurement tools for countries educational success such as adult assessment program, and teaching and learning survey. Therefore, my presumption is that OECD and PISA will go on being the next thing in educational assessment, and probably  even in educational policies for countries one from another.

Standardization, output-orientation, formalization, a boost for pedagogical diagnostics and classroom-management, substitution of content by methods.

I would say that the educational system has abandoned a huge part of its autonomy in this process of an economization of education. PISA et al were the key instruments of this change in Europe since the 1990ies. 

I have described this development in an article available here (start from p 7)

Article Semioses and Social Change. The Relevance of Semiosis on the...

Dear coleagues, look at this site

http://blogs.edweek.org/edweek/marketplacek12/2014/09/us_education_efficiency_ranks_in_bottom_50_percent_study_says.html?cmp=ENL-EU-NEWS2

its impact can be strong for us.

álvaro

I think seeing another study referring PISA results support the existence of our discussion. However, at the same time, it is very important to read studies well. I have to admit that I am suspicious about the findings in the report. One of the reason why I am suspicious is that the time dimension of the econometric analysis is too small (it is only 5), when we consider the number of variables they have listed in the Annex D section, my concern gets even more serious. Another question I have in my mind about the study is the econometric model they have used. I assume (since it was not specified otherwise), the report uses an aggregated variables and ignores the individual differences with in each country, which may easily mislead the results. While those are possible problems with the econometric analysis, I also have some question about the result of the analysis in terms of policy suggestions. I understand why efficiency is an important issue for production, including educational production. However, I don't understand how they have demeaned the efficiency of educational institutes to only two factors of education inputs (teacher salary & class size). As an end to my critics, I should say I am having problem to see how educational policy makers can decrease teachers' salary and while they increase the class size to create more efficient systems as they they have suggested to the US.

Mustafa,

Such instruments as PISA are single pictures of a complex social/cultural/educational context...often very different in the respective countries - and NOTHING ON WHICH TO BASE EDUCATIONAL POLICY CHANGE! PISA, etc. have no predictive validity for a nation's GDP and, in fact, such international exercises have NO SIGNIFICANT RELATIONSHIP TO RESPECTIVE NATION'S ECONOMIC COMPETITIVENESS ... meaning that countries scoring high on PISA do not have correspondingly high economic or innovation productivity!!!!!

M. L. Johnson, ED.D., Ph.D.

Dear Dr. Johnson,

Thank you for your response to my comment. In fact, I share similar concern with you about the effectiveness and random selection of samples of PISA in some countries, which would directly create apprehensions in the validity of test results. However, I am also aware that these concern of mine is true for only limited numbers of countries. For the rest of the others it is really hard to say (even statistically) that PISA is given only single picture. As you may already know OECD does not only collect the test score of students. The PISA data includes tones of information about the characteristics of students as an individual and a member of the classroom, their schools and even their families. Moreover these variables are not collected as self reported survey data. They are collected from all three sources; students, their families, and their teachers and principles. I don't say  that PISA data is the only or even the best source for the analysis of social, cultural, and educational context, but I do admit that it is so complex, powerful, and also promising for the future advancement as a tool for policy changes. About the predictive validity of PISA scores on nation's GDP and such, I would rather to wait like 10 more years because the first PISA has been done 14 years ago. So the student tested in that survey has just reached the age of 29-30 this year. If we consider them getting a tertiary education. We could say that they have nearly just finished their 5th year as a workforce. So it is too early to say how their result in PISA can be associated with their productivity at work. Moreover, when we consider the fact that PISA has been done in every 3 years, it can be obviously seen again that it is too early for any time analysis for a better comparison of countries for the predictive validity of test scores in regard to GDP or such. Lastly, I do not know much about how the test score of countries are associated with their economic and innovative productivity since my thesis is about how the achievement gap between low and high SES students within the countries has changed during the time depending on their social, economic, and institutional conditions. However, I could say based on a report by Bloomberg and published by Business Week ( The Link is: http://images.businessweek.com/bloomberg/pdfs/most_innovative_countries_2014_011714.pdf ) the capability of PISA test results are undeniable even for economic and innovative productivity concerns. The report shows that some of the high ranked countries in PISA such as South Korea, Japan, and Taiwan are also do really good job in terms of innovation, not only in overall scores as the US, but also in individual categories like Manufacturing capability rank, Tertiary efficiency rank, and Researcher concentration rank.

Mustafa,

Two things to consider in education public policy. First: ask the question of "What kind of graduate - skills and knowledge - is desired?" Second: "What are the experiences needed to achieve the goals? Few countries try to educate ALL students through secondary school in the PISA and TIMSS testing protocol; making comparable demographics near impossible. And, in regard to economic competitiveness and PISA/TIMSS scores, Tienken (2008, International Journal of Education Policy and Leadership) examined historic data and found NO significant relationship. Further, there is a difference between Disruptive innovation and Incremental Innovation. NO Nobel Prizes go to theses countries you mentioned  for their Incremental innovation. The Nobel Prizes go to countries where Disruptive innovations are generated AND those countries are seldom at the top of the PISA/TIMSS scores. BEWARE of the intervention chosen; as that program may produce graduates with attributes that do not help your country.

Cheers,

ML

Dr. Johnson,

I admit the importance those two questions in policy, but (if you excuse me) I also find contradiction for the US when I look at the possible answers and the statistics about the national education. So first of all what kind of graduates the US need?

IF we do not need most of our work force over educated, and have them with less educational attainment level,  why do we have US ranked 24th in Bloomberg's ranking for manufacturing capability rank even though the it is ranked 1st in high tech density rank. Does that mean the educational quality (not the attainment level) of workforce in manufacturing is lower than some other countries? If the answer is "Yes", is does not it contradicts with the policy that focuses more on quality of education than the attainment level?

OR IF we, indeed, need our workforce to be well educated [which probably requires more people in the tertiary education based on my conclusion from Tienken's (2008) paper since USA is a developed country already] then why the percent of students who entered a tertiary education institution and received at least a first degree is only little bit over the 50% even when we consider the fact that USA is the first country in institutional spending for the tertiary education, and relatively high ranked country for similar measurement for primary and secondary education? (Link: http://www.ncee.org/2013/07/statistic-of-the-month-highlights-from-education-at-a-glance/) Even more, why it is ranked 37th in the tertiary efficiency rank in Bloomberg's report? So are we failing in our aim to have our workforce get well educated by failing them in higher education institutes?

About Tienken's (2008) study, I have a lot of concerns. First of all, we cannot generalize his findings for PISA because in his paper he has analyzed only PISA 2000, and 2003 (not even 2006). Secondly, I have concern about using Spearman Rho test to define such a complex relation between educational outcome and economic conditions. It can be only a preliminary analysis for a further discussion and a deeper research. That analysis does not even consider the individuals. On of the very important reminder I have been hearing in my school since I have started my graduate study is that with out considering the clustering effect any study in education could be misleading. So aggregating thousands of students from each country for each international test into a one ranking score of a country is should be called "unfair" in the simplest term. There are some more question in my mind about the research, but the last thing I want to say is that to have such a strong result like "no significant relationship between two variables", we as a researcher at least consider the fact that there is a time differences between two variables. So if a country have well educated students growing (especially for developing countries) it is very natural to expect that the effect of the quality of education will be seen in a future time (to me not earlier than 10-15 years).

Lastly, I want to say couple words about the Nobel Prize. First of all I am no one to judge the fairness of Nobel Prizes, but I know there are people doing that. If we ignore these noises, I have another question about the winners. I don't have any statistics about their personal life, but I am curious how many of them has gotten their early education through a US institution. I have checked Wikipedia and found that little more than 25% of them were born in another country.

I want to repeated again, I am not defending a statistically significant relation between PISA ranking and countries economic conditions because I have never done any research on it. What I defend is (to me) the fact that OECD's work on education starting with the PISA is gaining importance among educational authorities. And I think it will go on this direction for some more years because OECD is continuously increases the quality of its existing tools and develops new tools in this concern such as PIAAC after PISA.

Regards,

Mustafa

Mustafa,

First, The Swiss-based World Economic Forum made a recommendation that is worth mentioning here; that "Developing countries need to create literacy and basic education programs and developed countries (U.S. etc) should crate education programs focused on innovation development." Thus, determine which category your context fits into and work on that type of programming. Second, do an internet search on Nobel Prize criteria ...the criteria are objective and rigorous  - virtually all of the medicine, chemistry, physics, biology and economics DO NOT come from highest scoring PISA/TIMSS countries. Third, read Tienken's research study on international tests and economic competitiveness...and it does not matter who thinks otherwise; the data speaks without opinion. 

Cheers,

ml

Dear Dr. Johnson,

(I have typed a longer version of my comment, but then my internet browser caused me loose it. Here is I wanted say with that comment in summary)

First, I appreciate your valuable comment and recommendation, but respectfully I do not want to keep the discussion turning around the association between the failure/success of the US in international assessments and its economic achievements because I believe time will be a better judge in that concern. The only thing I want to add is that I DID READ Tienken's paper, and as you may have seen in my previous posting, I have a lot of question about the methodology specifically when I consider the strong opinion the author has at the end of the analysis even though I see the point he wants to make. I wanted to mention what I think about the research because I believe what PEOPLE THINK about a hypothesis or a study is the ONLY thing that MATTERS to establish and advance of way of thinking and making correct decisions about phenomenons. As I mentioned in my previous posting I understand the value of the Tienken's study and I accept it as a preliminary analysis for future studies, but the analysis does not take me to the point the paper intended to reach basically because it has too shallow statistical examination of the extremely complex relation between a country's educational outcome and its economic position. I have a lot more to discuss, but I do not want to be disrespectful to the main topic of the discussion question. That is why it is all I want to add now.

Regards,

Mustafa

In Portugal the Pisa’s results and the place in the ranking has a huge impact. On the other hand, the Pisa report has a strong influence in curricular policies and in the schools work