OECD should be ashamed; PISA scores announced; doing more damage

Last week I spoke at Online Education Berlin. I thought that I was the opening speaker, but the people who arranged this meeting had set up the OECD representative of PISA, Andreas Schleicher, to be first, so that I could respond to him. I was going to talk about AI and pragmatic learning in online courses, but altered my talk to be an anti-test rant in response to his remarks. He never spoke to me at any time. Here is one the slides I used:

Now, not everyone knows what PISA is, but you should know. PISA is the single worst idea being forced down people’s throats in the world of education. It is sponsored by the OECD, which I had previously thought of as an organization that was started to do good. 

I first became aware of the effects of PISA when I was invited to speak in Bogota, Colombia because they had come in 62nd on the PISA rankings. What that means is that they were in an international competition, the World Cup of testing, and Colombia felt they needed help because they came in 62nd. At this meeting it was clear that they needed help all right, not in getting their PISA scores better but in getting some perspective on education.

Here is a current announcement from PISA: 

Coming Soon: PISA Results

On 6th December 2016 at 11.00 am (Central European Time) the results from PISA's 2015 round of testing of 15-year-olds in science, reading and mathematics in 72 countries and economies will be released. 

http://www.oecd.org/pisa/

How sad. 72 countries are waiting to hear how they are doing in the international math competition. Why would anybody care? Here is the analysis from last year:

"Asian countries outperform the rest of the world", according to the OECD, with Shanghai, Singapore, Hong Kong, Taiwan, South Korea, Macau and Japan amongst the top performing countries and economies. Students in Shanghai performed so well in maths that the OECD report compares their scoring to the equivalent of nearly three years of schooling above most OECD countries.

Of the 64 countries with comparable data up to 2012, 32 improved their reading performance while 22 showed no change and 10 deteriorated. If you look at performance at maths, 25 show an average annual improvement, 25 show no change, and 14 show a deterioration in performance.

Qatar, Kazakhstan and Malaysia recorded an average improvement in maths performance of more than eight points per year. The OECD report also praises Brazil, Chile, Germany, Israel, Italy, Mexico, Poland, Portugal, Tunisia and Turkey who it claims have "shown a consistent improvement" over time in maths performance.

I will keep you in suspense about who won this years competition. But let's think about what these scores might mean in each country. I will start with Greece, a country I know well because I was a consultant to a Greek ship owner (to help him invent software for the shipping industry) for many years.

Greece's main industries are tourism, shipping, industrial products, food and tobacco processing, textiles, chemicals, metal products, mining and petroleum. (This is from wikipedia)

I can tell you that there is not a single course in any Greek school that teaches shipping or tourism. They do teach Ancient Greek and Greek history and, of course, algebra and science. Greece came in #42 in PISA in 2012 and I am sure this made them very anxious. What makes me anxious is that Greek schools didn't care about teaching their kids about the jobs they might actually be able to get in Greece. Instead they are engaged in a competition to answer questions like this one:

DRIP RATE

Infusions (or intravenous drips) are used to deliver fluids and drugs to patients.

  Nurses need to calculate the drip rate, D, in drops per minute for infusions. They use the formula D = dv where

 60n

d is the drop factor measured in drops per millilitre (mL)

v is the volume in mL of the infusion

n is the number of hours the infusion is required to run.

1. A nurse wants to double the time an infusion runs for.

Describe precisely how D changes if n is doubled but d and v do not change.

2. Nurses also need to calculate the volume of the infusion, v, from the drip rate, D. An infusion with a drip rate of 50 drops per minute has to be given to a patient for 3 hours. For this infusion the drop factor is 25 drops per millilitre. What is the volume in mL of the infusion?

Now, I was a math major in college and I figured out math on my own when I was about 5. I can’t answer this question. My mind glazes over. I could see teaching nurses how to do these calculations in nursing school, but everyone in the whole world?  Why? I assume the chief engineer on a ship knows how to do math. I am also sure he went to engineering school and rather sure that he actually never does much algebra.  But PISA ensures that every 15 year old student should be able to answer questions like this. 

Wonder how France did? They came in 25th. This is important because science and math matter so much in France. The Major industries in France include automobile manufacturing, aircraft production, chemicals, electronics, machinery manufacturing, metallurgy and tourism.

Surely these require math you say? Well, if they do, you can be sure that the math needed is covered in the schools that teach metallurgy or chemistry. PISA is for 15 year olds. I would be happy to see France offer metallurgy in high school, or tourism, or aircraft production, but they don’t. They teach the basics which includes Moliere, Robespierre, Louis the 16th, and lots of algebra. My daughter attended high school in Paris when she was 13. She got a B in English.  I asked her if anyone in her class spoke English and she said “no.” “Then why did you get a B,”  I asked. She said she never heard of the subjunctive case. So I am pretty sure they also teach the subjunctive case in France which matters so much to a grammarian and so little to an actual speaker who has non-conscious knowledge of their language and can use it but doesn't know what it is called (nor do they care.)

I was wondering if they tested for knowledge of the subjunctive case in PISA and found this:

In the PISA 2012 Student Questionnaire an OCT was operationalised by asking students to indicate their familiarity – on a 5-point scale from “never heard of it” to “know it well; understand the concept” – with actual mathematics concepts (e.g. “polynomial function”) and foils (e.g. “proper number”). Foils were created by combining a term from grammar (i.e. proper, as in proper noun; subjunctive, as in subjunctive mood; declarative as in declarative sentence) with a mathematical term (i.e. number; scaling; fraction, respectively).  

So they expect it to be taught so that they can use it as a foil in math exams. Pretty clever PISA.

How about Finland? Finland is considered very progressive in education. They are said to be eliminating traditional subjects which would be a wonderful thing. And they win PISA quite often which would imply that maybe they aren't eliminating algebra. But surely this is because “math teaches you to think” which everyone believes and I believe as much as I believe that the Great Pumpkin arises on Halloween eve. 

Metals and engineering now constitute the largest sector of Finnish industry, with motor vehicles and machinery driving much of the growth of the late 1990s.

So, Finland teaches machinery in high school? I hope they do but I doubt it.

As I said, I was worrying about Colombia when I was asked to visit there after they “failed” PISA.

“Colombia is the largest export partner of Aruba (39.4%). The petroleum and natural gas coal mining, chemical, and manufacturing industries attract the greatest U.S. investment interest.”

Do they teach mining in schools in Colombia? No. I was advising in Chile too recently. Mining is major there too. Do they teach mining in Chile? No. I fought with the education ministry there who wanted to teach more algebra in order to do better on PISA tests. 

Today the PISA scores were announced. Singapore won. I was recently speaking in Singapore. The cab drivers told me what an undemocratic totalitarian country it was and I hadn't even asked. I met with the Chairman of Singapore Democratic Party who told me he can never win an election because the party in charge owns all the media and he never gets mentioned. Singapore thinks it has a great education system. I spoke to the teachers there maybe 10 years ago and suggested that maybe learning to think for yourself was more important  than rigidly practicing for math tests all the time. My message was received coldly by the teachers.   And then I went outside and many people on the street were cheering for me. The talk had been televised and average people, the doorman, the taxi driver, the hotel manager were all aware that they had hated school and had been taught nothing of value for their lives. But Singapore won PISA . Hip hip hooray.

After my talk in Singapore last year I got a call from people who wanted my help on a project to teach people to get better scores on TOEFL tests. China and India do well on PISA too. How does this relate to TOEFL tests? Many people in these countries really want to go to university in the US or the UK. They need to show they can speak English well and this is shown by?  Multiple choice tests of course. Prospective students need to have great TOEFL scores and great math scores to get admitted. So congratulations PISA, and to all winning countries, on helping your people get our of you country and study in the US the UK. That will help you get rid of your future scientists, who rarely want to leave the US after they get there PhDs here. (I rememberer trying to persuade a brilliant Indian AI student of mine to go back to his country and help it. He looked at me like I was crazy. Now he is the head of AI at Amazon.)

Does the US and UK have it right? Hardly.

In Santa Fe, New Mexico I asked the people  who ran the Indian school there what curriculum I could build for them and they said “Casino Management.” One can well understand why. Could I do it. No. The Governor vetoed it and said “more algebra.”

In Kansas I asked what they needed and was told aerospace engineering. “Really?” I asked. “Yes” they said, we build Lear Jets here and we don’t have a single aerospace engineering course anywhere in Kansas.” Could I do it? Of course not. We have Common Core instead of PISA in the US. But it is all the same. The 19th century curriculum reinforced by testing companies. 

Getting people to think for themselves, becoming capable of employment, able to live happy lives, would seem to be better goals to me, but instead we have PISA. OECD should be ashamed.

Here is part of what the OECD had to say today:

Around 1 in 10 students across OECD countries, and 1 in 4 in Singapore, perform at the highest level in science. Across the OECD, more than one in five students falls short of baseline proficiency: only in Canada, Estonia, Finland, Hong Kong (China), Japan, Macao (China), Singapore and Viet Nam do at least nine out of ten 15-year-old students master the basics that every student should know before leaving school.

This underlines the challenge that all countries, including some of the wealthiest ones, face in meeting Sustainable Development Goal 4 by 2030 to achieve “inclusive and equitable quality education and promote lifelong learning opportunities for all”.

The report reveals the policies in place that successful countries share: high and universal expectations for all students; a strong focus on great teaching; resources targeted at struggling students and schools; and a commitment to coherent, long-term strategies.

Canada, Denmark, Estonia, Hong Kong (China) and Macao (China) achieve both high standards of excellence overall and equity in education outcomes.  A number of countries have improved equity, especially the United States. But in Australia, Czech Republic, Finland, Greece, Hungary, New Zealand and the Slovak Republic, the share of students performing at the highest levels fell at the same time as the share of low performers rose.

“Achieving greater equity in education is not only a social justice imperative, it also fuels economic growth and promotes social cohesion,” added Mr Gurría.

The OECD PISA 2015 Survey underlines that, in the context of massive information flows and rapid change, everyone now needs to be able to “think like a scientist”: to be able to weigh evidence and come to a conclusion; to understand that scientific “truth” may change over time, as new discoveries are made, and as humans develop a greater understanding of natural forces and of technology’s capacities and limitations.

Everyone needs to know how to think like a scientist. Really? Why? And how would these tests assess that? Let's look at a typical PISA science question:

Bird migration

Most migratory birds gather in one area and then migrate in large groups rather than individually. This behaviour is a result of evolution. Which of the following is the best scientific explanation for the evolution of this behaviour in most migratory birds?

1 Flying in large groups allowed each bird to have a better chance of finding a nesting site.

Flying in large groups allowed other bird species to join the migration.

Birds that migrated individually or in small groups were more likely to find adequate food.

Birds that migrated individually or in small groups were less likely to survive and have offspring.

I found this question in an Australian newspaper today. Here is the headline that accompanied it:

Australian school students two years behind world's best performing systems

Or, in others words, we are all losers. We need to be very afraid. Inciting fear seems to be the major outcome of PISA. I wonder why. Mr Schleiker was proud that all countries will soon be the same and his test will make them be the same in their schools. This is exactly the opposite of what needs to be the case. School should fun. School should be relevant to life after school. And every country and every state is different. Their differences should be cherished. And no one needs algebra or science in high school. What we need is people who can think clearly, understand  scientific thinking, be creative, defend one’s ideas, diagnose problems and come up with solutions, That is what PISA should test. Actually PISA should test nothing. OECD needs too get rid of this idea completely. Perhaps they could put their money into helping countries teach kids things that will useful in their own countries.