Testing Nations: Olympics and PISA

This month the British are feeling pleased with themselves, having hosted the 2012 London Olympics successfully and, in terms of medals won, having come third, after the USA and China. But the duration of the games is about two weeks in a cycle of four years which is about 1% of the time. Another and more perennial form of international competition is in academic standards as assessed by PISA, the Programme for International Student Assessment. This post attempts to bring recent Olympic and PISA performances together, and to provide some insight into national priorities.

At which point it is worth borrowing a useful concept from physics: that of properties that are intensive or extensive. As a simple example, the volume of a liquid is extensive and increases with the amount present but the liquid’s density stays the same however large or small the amount – the first is an extensive property, the second intensive. By analogy, the number of medals a country wins tends to be ‘extensive’ – a bigger population in which to find outstanding talent – but there is nothing to stop a small country having very high academic standards, which are, in these terms, ‘intensive’.

The Chart below shows the top 25 2012 Olympics countries in rank order (red) and also their 2009 PISA rank (blue). But alongside the Olympic rank is a percentage figure. This is the extent to which the number of medals won by a country differs by more than 10% from the number that would have been predicted taking account of factors such as population size, GDP per capita, past performance and “host-nation advantage” (more about this in the Notes below).

So, the UK, while it ranks 3 in the Olympics, is only ranked 21 in PISA terms. Its total of medals (29 in fact) was 15% higher than expected. By contrast, the Netherlands, Olympics ranking 13, 17% more medals than expected, but educationally is ranked 11th. At the top of the table the USA (medal score predicted to within 2%) lies at 26 in PISA terms. Russia and Kazakhstan also show signs of Olympics overachievement and educational underachievement.

South Korea demonstrates a balanced achievement, fifth in both PISA and Olympics with a small underperformance (3%) in the latter. New Zealand and Australia make an interesting contrast. They rank at eighth and tenth in PISA (perhaps proximity to Asia focusses national minds) but NZ was also able to over perform at the Olympics by 64% to achieve a ranking of 15th, while Australia, again at tenth in London, underperformed by 14%. Canada, an impressive seventh in PISA, lies outside the chart as it came 35th in Olympic rank because, although gaining 18 medals (5% more than expected), 2/3rds were bronze.

The Table below supports the idea of PISA score being ‘intensive’. Those PISA nations in the top 30 but which didn’t make the Olympics top 25 above are listed together with their Olympics ranking a deviation from expected performance. Some nations were not represented in London. The UK and USA are shown in the Table next to their PISA peers, underlining their markedly different Olympics performance.

In terms of national priorities, the Russian and Kazakhstan show the biggest disparities between Olympic and PISA performance. Not too far behind are the US and UK. However, Olympics success depends on identifying and then focussing resources on a small number of suitably talented candidates, whereas PISA reflects a nation’s ability to raise the educational standards of the majority of its population. The UK has excellent schools, particularly in the private sector attended by less than 10% of pupils, but whose alumni were proportionately over-represented in the UK’s Olympics team. The existence of a similar (and possibly self-perpetuating) elite in the US has been discussed here previously.

NOTES ON THE DATA

1. The 2012 Olympics rankings are taken from the standard medal table, ranking countries first in order of gold medals, then silver medals, then bronze medals.

2. The Financial Times (£) combined results from four different econometric models addressing Population, GDP per capita, past performance and “home advantage” and some other factors to predict the total number of medals each nation might be expected to win, and compared this with the actual achievement. The size of any deviation has been expressed here as a percentage of the expected number, but bear in mind that small numbers produce large percentages (eg Singapore and Iran).

3. The 2009 PISA scores for Mathematics, Science and Reading were added and the nations ranked accordingly. Although China appears here as ‘First’ the data has to be regarded sceptically. PISA cites data for Shanghai (First), Hong Kong (2nd) and Macau (17th) – a more representative score for the whole of China might have come out rather lower. The 2012 PISA rankings will be reported in late 2013 and may shed more light.

4.  UK refers to the United Kingdom of Great Britain and Northern Ireland, known as GB.

Means and Ends

Ross Clark’s article, Our China Syndrome, in the current Spectator (summary here) was sub-headed The claims against swimmer Ye Shiwen reflect irrational suspicion of her country, a theme which he expanded on:

Rarely can a sporting performance have been met with such resentment as that of 16-year-old Ye Shiwen in the 400 metres women’s medley. She had hardly wiped the drops from her goggles before the US swimming coach John Leonard was making coded accusations of drug use, with words such as ‘unbelievable’ and ‘disturbing’. Her offence, it seems, was to have swum a length of the pool faster than an all-American winner of the men’s event, Ryan Lochte.  

… Ye Shiwen had passed the routine drugs test to which all medal winners at the Olympics have been subjected. She wasn’t a drugs cheat after all, …  

Deprived of the opportunity to accuse her of doping, the nation’s Sinophobes changed tack. Maybe she wasn’t on drugs, but she was the product of a ‘brutal training regime’ … Then came the extraordinary accusation that she might have been genetically modified. …  

BBC’s Newsnight devoted its main story to the fantasy that athletes might be having their genes modified to improve their performance — interspersed with clips of Ye Shiwen winning her medals. After 15 minutes of this, the scientists invited on the programme to discuss the issue admitted that there was no evidence it was actually possible to manipulate an athlete’s genes in this way.

There is certainly no question of Ye Shiwen having failed any tests. However, the scientific position may not be as categorical as Clark was assuming. The Times monthly science supplement, Eureka, gave its August issue over to Science and the [Olympic] Games, including an article by Kaya Burgess, Testing times: getting ahead of the cheats, in which she tries to answer the question:

The doping authorities at London 2012 are implementing the most stringent drug testing regime of any Games. But can cheats still slip through?  

… There is little doubt that there are more ways to cheat in sport than ever. Since the British Olympic Association first started working on the Olympic bid, in 1997, performance enhancement has evolved beyond the abuse of steroids and stimulants to include ever-subtler ways of enhancing the body from within. The challenge for UK Anti-Doping (UKAD) is to try, at last, to get one step ahead. This year, every medallist will be tested, and so will about half of all other athletes competing. More than 6,000 samples are expected to be analysed in the course of the Games. But the testers at the Olympic testing laboratory in Harlow have to know what they are looking for.  

… Existing drugs approved for medical use will already be on the radar of the World Anti-Doping Agency (Wada), which will have analysed the potential for cheats and prepared watertight arguments against those who claim innocent use. What Wada has not known about until now, however, is the vast range of drugs still in secret development by pharmaceutical companies. For the first time, one of those companies, GlaxoSmithKline, has opened its drug-research pipeline to Wada to help officials flag up new or unknown substances that may have found their way into athletes’ hands through other sources. …  

Steve Clarke, GSK’s head of drug metabolism and pharmacokinetics, … cites the example of Balco — the Bay Area Laboratory Co-Operative, near San Francisco — which was found to have sold steroids, growth hormones, testosterone cream and erythropoietin to Major League baseball players as well as Chambers. “They had looked through the literature, come up with a steroid drug that wasn’t marketed anywhere but that had been investigated as a potential at one stage, and they had chemists capable of making that drug,” Clarke says. The result was a drug that Wada didn’t know about and didn’t have a test for. “That’s why athletes got away with it for so long and why THG was known as ‘The Clear’.”

As Burgess concludes, it wouldn’t be sensible to bet against competitors breaking the rules, perhaps under pressure. None of this reflects in any way upon Ye Shiwen, but she does come from a country which plays with a hard ball and where ends seem to justify means. I have posted here before about Chinese hacking and cyber theft, but some more insight into what has been happening was provided by Bloomberg just as the Olympics opened in an article by Michael Riley and Dune Lawrence, Hackers Linked to China’s Army Seen From EU to D.C., which anyone with the slightest interest in the subject should read in full. Apart from the EU and various organisations in Washington DC, the article reports attacks on computers in Canada and India, and:

The networks of major oil companies have been harvested for seismic maps charting oil reserves; patent law firms for their clients’ trade secrets; and investment banks for market analysis that might impact the global ventures of state-owned companies, according to computer security experts who asked not to be named and declined to give more details.  

… Stolen information is flowing out of the networks of law firms, investment banks, oil companies, drug makers, and high technology manufacturers in such significant quantities that intelligence officials now say it could cause long-term harm to U.S. and European economies.

The article revealed that:

Adding a potentially important piece to the puzzle, researcher Joe Stewart, who works for Dell SecureWorks, an Atlanta-based security firm and division of Dell Inc. (DELL), the computer technology company, last year uncovered a flaw in software used by Comment group hackers. Designed to disguise the pilfered data’s ultimate destination, the mistake instead revealed that in hundreds of instances, data was sent to Internet Protocol (IP) addresses in Shanghai. The location matched intelligence contained in the 2008 State Department cable published by WikiLeaks that placed the group in Shanghai and linked it to China’s military.

Perhaps China’s pharma technology base is nothing like as advanced as their cyber capability, but if it is they could probably run Olympic-sized rings around UKAD and Wada, if they set out to do so. The scope in the long term for innovations of all sorts, overt and covert, in a country in which even a fraction of its population reaches the PISA standards recorded for Shanghai’s 15-year olds in 2009 will be immense.