How should we interpret the Scottish Survey of Literacy and Numeracy?

SSIN

The Scottish Survey of Literacy and Numeracy 2015 is now out. It is a great example for students of agenda setting and framing because an apparently dry account of recent trends may come to dominate the highest level debates in Scottish politics.

The stakes are high because the figures – showing some drops in attainment at key stages, and major gaps in inequalities of attainment at all stages – are being interpreted in this context:

Allegations of government shenanigans

The SNP Government’s opponents reckon that it delayed the results because they were so bad as to undermine the SNP vote in the Holyrood elections in 2016.

https://twitter.com/camusson/status/737585832664530946

Allegations of misplaced priorities

The SNP Government’s critics reckon that it has funded free tuition for the middle classes at the expense of funding to reduce attainment gaps at school (which make it less likely for people in deprived areas to go to benefit from free University education).

https://twitter.com/trewloy/status/737576623231270916

SNP Government priorities

Education will be the big focal point from 2016-21 because the SNP has signalled it as the top priority (symbolised by the fact that John Swinney is now in charge). Indeed, Nicola Sturgeon promised at one point to ‘close the attainment gap completely’ (report on original speech).

The SNP manifesto in 2016 presents more of a mix of aims (using more equivocal language). Its promise to ‘close the attainment gap between young people from the most and those from the least affluent backgrounds’ really means reduce (‘our mission is to make significant progress in closing the gap within the next parliament and to substantially eliminate it within a decade’), while its promise to deliver ‘significant progress in closing the attainment gap within the lifetime of the next parliament and substantially eliminating it within a decade’ betrays the sense that it does not really know how to tell how far it can reduce the gap and declare almost-complete success.

This background gives us a lens through which to view most analysis of the figures

As you’d expect, so far the immediate reaction is rather critical:

The SNP Government response will be trickier-than-usual for the following reasons:

  1. Free University tuition is non-negotiable.

It won’t give up on free University tuition (for Scottish students and, in effect, EU students outside the UK), and it doesn’t accept the argument that its spending on University tuition comes at the expense of its spending on pre-school and school spending. So, it has to find the money, to reduce the attainment gap, from other areas (producing the possibility of limited success in education and worsening results in other high profile areas such as health).

  1. Its ability to link the issue to Scottish independence is now more limited.

The usual (and often plausible) response to the inequalities in attainment gap is to argue that: (a) it is caused largely by socio-economic conditions such as poverty, rather than teaching; and, (b) the Scottish Government does not have the tax/ spending power to reduce the problem at source. Yet, Sturgeon has staked her reputation on making a difference despite these constraints.

So, what happens next?

At some point, we might have a less emotive, less partisan, and more considered look at these figures. If so, I’d like to be clearer on three different kinds of question. The first two are about setting milestones so that we don’t wait until the figures come out before we pronounce success/ failure, and the third is about the figures themselves:

  1. What level of inequality of attainment is OK?

Whatever language the SNP uses, it won’t close this gap completely. Instead, we could benefit from an honest and pragmatic discussion about the level of inequality in attainment that we think is acceptable (at least at each stage of policy development). In Scotland, we tell a good story about consensus politics, but it won’t mean much if the government pretends to hold on to aims it doesn’t think are achievable, while the opposition criticises any gap regardless of progress.

  1. What trends are acceptable?

In agenda setting studies we note the profound effects of trends in data, which can be more important (in gaining attention) than the baseline figures. If we want to try to avoid getting sucked into these lurches of attention to often-minimal change, we could benefit from a sense of perspective on trends. Is any positive effect a cause for celebration or negative effect a cause for prophecies of doom? No. Yet, the stakes are so high that people are ready to pounce at any minute, or at least every year. Governments play that game too, with performance management systems that really don’t help (and, for example, Robert Geyer has a different way to consider long term success and failure).

  1. What do these figures really mean?

I don’t think the survey was designed with these high political stakes in mind. If they are to remain so crucial to political debate, I’d like to see more explanation – to the public and political commentators – of the method and results, to give a clearer sense of how to interpret the baseline figures as well as the trends.

Otherwise, for example, people will point to the wacky drop in attainment progress in S2 (compared to P7) and wonder what the hell happened (particularly since it exposes the major gap in attainment linked to deprivation). The report itself mentions the well-discussed problem of transition from primary to secondary school, but could discuss more the likelihood that this statistical dip is caused partly by measurement (possible explanations include: the expectation for S2 is disproportionately high, given the problems of transition; or, secondary and primary teachers have different view on how well someone is doing when they enter S1).

For further reading, see Lucy Hunter Blackburn’s site Adventures in Evidence, which provides more frequent and in-depth (and critical!) coverage of Scottish education policy than mine.

See also A memo to John Swinney

For some of the coverage, see for example:

Numeracy rate falls among pupils in Scotland, latest figures show

 

 

3 Comments

Filed under agenda setting, Scottish politics

3 responses to “How should we interpret the Scottish Survey of Literacy and Numeracy?

  1. Pingback: The background of new MSPs | Paul Cairney: Politics & Public Policy

  2. Pingback: The future of Scotland in the UK: does the remarkable popularity of the SNP make independence inevitable (version 2)? | Paul Cairney: Politics & Public Policy

  3. Pingback: The Scottish Government’s holistic education policy: a story of profound success or failure? | Paul Cairney: Politics & Public Policy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s