In 2015 over half a million 15-yearolds, representing 28 million students in 72 countries and economies, sat down to take a twohour test - a test that has become a touchstone for politicians globally looking for evidence of ‘success’ or ‘failure’ of education systems and the future prospects for economies. The Programme for International Student Assessment (PISA) has been run by the OECD every three years since 2000, testing the skills and knowledge of students in science, mathematics, reading, collaborative problem solving and financial literacy. Each round of results is seen by policymakers and the range of stakeholders involved with school education as both a reason for apprehension and an opportunity for either heralding successes or making a case for change.

In the latest PISA league tables, the UK is ranked 27th for maths, 22nd in reading and 15th in science, placing the nation well behind the highest performing 15-year-olds in countries such as Singapore and Finland, and also behind Vietnam, Poland and Estonia. The results prompted stiff warnings from ministers on the dangers of “stagnating” performance, and a general sense that while much of the rest of the world was moving on, finding new energies and motivations, the UK was stuck in a rut.

What makes the PISA research particularly interesting - and useful for deeper study - is that the test results themselves are just the surface level of the data. Underneath is a rich array of information about individual pupils and their family backgrounds: the education of their parents, household wealth, whether they have books and IT at home as well as mental health and motivation to study. There’s also indepth material on the schools they attend, class sizes, numbers of disadvantaged pupils and pupils with special needs, the resources available, school governance and headteacher’s own perspectives on resources and infrastructure in their school.

This makes it possible to explore the underlying issues and trends. Does it really matter if there are books on the shelves at home? Or if pupils feel anxious about their school work? What impact on performance is there from large class sizes, or if schools have large numbers of students from disadvantaged backgrounds?

PISA Patterns

Our study analysed the PISA data for nine roughly comparable countries in terms of being developed countries with well-established school systems: the UK alongside Australia, Canada, France, Germany, Italy, Japan, Spain and the USA. We wanted to understand more about the factors in a student’s life that are most related to student achievement, and also, what characteristics in a school environment and the differences between schools have an impact. And we did this in a different way. Typically, statistical methods would be used to examine and assess the data in detail. In this case, given the huge mass of data involved across countries, we used ‘machine learning’ - a form of Artificial Intelligence - which can trawl large volumes of data and signpost the most obvious patterns through a visual representation. In this way, attention is drawn explicitly to the factors that matter most.

What catches the eye in the UK is the importance of a student’s background. A dominating factor in determining how well a 15-year-old will do in a PISA test is a collection of issues around economic and social environment: the level of their parents’ education and income, the availability of books at home, having space at home to sit in a quiet space with a desk to study. The same was found in four of the other countries, (Australia, France, Spain and the USA). Related to this was the finding around the influence of the school environment. In the UK - and almost all the other countries - the biggest effect on results came from the proportion of disadvantaged pupils in the school, as wells the percentage of pupils with special needs. On average, schools with a high proportion of disadvantaged students suffer a negative impact on performances, of anything up to 40%.

Social Advantages

The role of social background isn’t of such paramount importance in every country - each has its own challenges for policy-makers to address. In Canada, Germany and Italy, the most significant factor among students is anxiety, where a lack of confidence leads to lower average test scores. In Japan, students’ self-motivation has the greatest effect. Scores of Italian students are lower if there’s evidence of a lack of involvement by parents, such as attending parents’ evenings. The size of the school is critical in France (if there are fewer than 900 pupils, the test performance collapses). In Australia, levels of truancy in a school are an important indicator, as well as the extent to which a school is reliant on funding from the state.

With each of these examples there are opportunities to use education policy, to provide more support for pupils dealing with anxiety, incentivise or penalise parents for their level of involvement, to merge groups of smaller schools, change funding models. The predominance of the social background factor in the UK is not so straightforward. Naturally, calls for action and policy changes have centred around what can be achieved more easily and in more visible ways. So, for example, improving performance in the UK has largely focused on ensuring there is the availability of IT resources for all, there are smaller class sizes and ‘better’ teachers.

By contrast, our findings show that pupil to teacher ratios and the availability of IT per pupil are not a hugely important factor in PISA test score success in any country. Essentially, the study suggests that the fundamental platform for improving educational achievement in a country like the UK would be through policies to address social disadvantage.

There are some regional challenges that illustrate the need for this different perspective on policy. In terms of PISA scores, of the UK countries, Wales has the lowest results in every subject tested, putting its system only a few places above Argentina and Colombia. Despite the similarities in the education provision, the gap between results in Wales and England is also growing. In the wake of the Welsh government’s Furlong report, there has been a tightening of requirements for newly qualified teachers, and reform of initial teacher education; continuing professional development has been enhanced through the New Deal for the Education Workforce. The Donaldson report highlighted the need for curriculum reform in Wales, to better meet employer needs and increase the emphasis on learning skills, creativity, ethical behaviour and a commitment to society. There has also been a commitment to creating more Academy schools, with improved funding via central Government, and a National Academy of Leadership. There is good evidence for the impact of improved standards of teaching and school leadership. But to attempt to understand the gaps in attainment only in terms of the education system is to overlook the major role of the social environment. Addressing deprivation in local communities, the stark inequalities in income and ability to provide children with homes that are more conducive to, and supportive of, learning, is more important.

Another current issue surrounds the development of a ‘Northern Powerhouse’. In June 2014, Finance Minister George Osborne declared that while northern cities were individually strong, they were collectively not strong enough. While London and its region has a population of around 14 million, the closest British cities in scale have only three million (Birmingham and Manchester). More than doubling the scale of a city like Manchester is predicted to have the potential to lead to a 7.5% increase in its per capita output, with a Northern Powerhouse having the potential permanently to add as much as 1% to the national Gross Domestic Product. In policy terms, the emphasis to date has been on a substantial investment of public funds into improving the transport infrastructure, as well as attention to support housing and industrial development.

Again, however, this approach is overlaying new schemes and infrastructure onto a landscape of inequalities and social disadvantage. Crucial for the growth prospects of a Northern Powerhouse is the ability of employers in the north to attract and retain a supply of highly skilled workers. Our research into educational progress in northern regions shows how pupil achievement is comparable with the rest of the UK - but only up to when pupils sit for their GCSEs. After this stage, fewer pupils go on to A-levels and through into higher study. Those students who are more successful are drawn to London and the south for work, creating a regional brain drain. Better transport links and housing aren’t going to create a Powerhouse in themselves. A successful system of education is needed too – and that entails successfully addressing social as well as educational issues.