Testing Nations: Olympics and PISA

This month the British are feeling pleased with themselves, having hosted the 2012 London Olympics successfully and, in terms of medals won, having come third, after the USA and China. But the duration of the games is about two weeks in a cycle of four years which is about 1% of the time. Another and more perennial form of international competition is in academic standards as assessed by PISA, the Programme for International Student Assessment. This post attempts to bring recent Olympic and PISA performances together, and to provide some insight into national priorities.

At which point it is worth borrowing a useful concept from physics: that of properties that are intensive or extensive. As a simple example, the volume of a liquid is extensive and increases with the amount present but the liquid’s density stays the same however large or small the amount – the first is an extensive property, the second intensive. By analogy, the number of medals a country wins tends to be ‘extensive’ – a bigger population in which to find outstanding talent – but there is nothing to stop a small country having very high academic standards, which are, in these terms, ‘intensive’.

The Chart below shows the top 25 2012 Olympics countries in rank order (red) and also their 2009 PISA rank (blue). But alongside the Olympic rank is a percentage figure. This is the extent to which the number of medals won by a country differs by more than 10% from the number that would have been predicted taking account of factors such as population size, GDP per capita, past performance and “host-nation advantage” (more about this in the Notes below).

So, the UK, while it ranks 3 in the Olympics, is only ranked 21 in PISA terms. Its total of medals (29 in fact) was 15% higher than expected. By contrast, the Netherlands, Olympics ranking 13, 17% more medals than expected, but educationally is ranked 11th. At the top of the table the USA (medal score predicted to within 2%) lies at 26 in PISA terms. Russia and Kazakhstan also show signs of Olympics overachievement and educational underachievement.

South Korea demonstrates a balanced achievement, fifth in both PISA and Olympics with a small underperformance (3%) in the latter. New Zealand and Australia make an interesting contrast. They rank at eighth and tenth in PISA (perhaps proximity to Asia focusses national minds) but NZ was also able to over perform at the Olympics by 64% to achieve a ranking of 15th, while Australia, again at tenth in London, underperformed by 14%. Canada, an impressive seventh in PISA, lies outside the chart as it came 35th in Olympic rank because, although gaining 18 medals (5% more than expected), 2/3rds were bronze.

The Table below supports the idea of PISA score being ‘intensive’. Those PISA nations in the top 30 but which didn’t make the Olympics top 25 above are listed together with their Olympics ranking a deviation from expected performance. Some nations were not represented in London. The UK and USA are shown in the Table next to their PISA peers, underlining their markedly different Olympics performance.

In terms of national priorities, the Russian and Kazakhstan show the biggest disparities between Olympic and PISA performance. Not too far behind are the US and UK. However, Olympics success depends on identifying and then focussing resources on a small number of suitably talented candidates, whereas PISA reflects a nation’s ability to raise the educational standards of the majority of its population. The UK has excellent schools, particularly in the private sector attended by less than 10% of pupils, but whose alumni were proportionately over-represented in the UK’s Olympics team. The existence of a similar (and possibly self-perpetuating) elite in the US has been discussed here previously.

NOTES ON THE DATA

1. The 2012 Olympics rankings are taken from the standard medal table, ranking countries first in order of gold medals, then silver medals, then bronze medals.

2. The Financial Times (£) combined results from four different econometric models addressing Population, GDP per capita, past performance and “home advantage” and some other factors to predict the total number of medals each nation might be expected to win, and compared this with the actual achievement. The size of any deviation has been expressed here as a percentage of the expected number, but bear in mind that small numbers produce large percentages (eg Singapore and Iran).

3. The 2009 PISA scores for Mathematics, Science and Reading were added and the nations ranked accordingly. Although China appears here as ‘First’ the data has to be regarded sceptically. PISA cites data for Shanghai (First), Hong Kong (2nd) and Macau (17th) – a more representative score for the whole of China might have come out rather lower. The 2012 PISA rankings will be reported in late 2013 and may shed more light.

4.  UK refers to the United Kingdom of Great Britain and Northern Ireland, known as GB.