In Policy Concepts in 1000 Words: Success and Failure, I argue that evaluation is party political. Parties compete to describe policies as successes or failures based on their beliefs and their selective use of evidence. There is often a lot of room for debate because the aims of policymakers are not always clear. In this post, I argue that this room still exists even if a policymaker’s aims appear to be clear. The complication is that a policy aim consists of an explicit statement of intent plus an often-implicit set of assumptions about what that statement of intent means in practice. This complication is exploited by parties in the same way as they exploit ambiguities and their selective use of evidence.
Let’s take the example of class sizes in Scottish schools, partly because it is often highlighted by opposition parties as a clear example of policy failure. The SNP manifesto 2007 (p52) seems crystal clear:
We will reduce class sizes in Primary 1, 2 and 3 to eighteen pupils or less (sic)
Further, the SNP Scottish Government did not appear to fulfil the spirit of its commitment. There is some wiggle room because it does not say all classes or set a deadline, but it is reasonable to assume that the pledge refers to extensive progress by 2011 (the end of the parliamentary session). Indeed, the lack of progress was seized upon by opposition parties, who seemed to be partly responsible for the removal of the Education Secretary from her post in 2009. The issue arose again at the end of 2013 when average class sizes appeared to be higher than when the pledge was made.
My magic trick will be to persuade you that, in an important way, the reduction of class sizes was not the SNP’s aim. What I mean is this:
- Each policy aim is part of a wider set of aims which may undermine rather than reinforce each other. In general, for example, spending on one aim comes at the expense of another. In this specific case, another SNP aim was to promote a new relationship with local authorities. It sought to set an overall national strategy and fund programmes via local authorities, but not impose policy outputs or outcomes on implementing bodies. Those two aims could be compatible: the Scottish Government could persuade local authorities to share its aims and spend money on achieving them. Or, they could be contradictory, forcing the Scottish Government to pursue one aim at the expense of another: either imposing policy on local authorities, or accepting the partial loss of one aim to secure a particular relationship with local authorities.
- Class sizes are not aims in themselves. Instead, they are means to an end, or headline-grabbing proxy measures for performance. The broader aim is to improve learning and/ or education attainment (and to address learning-based inequalities). Further, local authorities may have their own ideas about how to make this happen, perhaps by spending their ‘class size’ money on a different project with the same broader aim (I have not made up this point – a lot of teaching professionals are not keen on these targets). Again, the Scottish Government has a choice: impose their own aim or trust some local authorities to do things their own way – which might produce a lack of implementation of a specific aim but the pursuit of a broader one.
- The assumption is always that nothing will go wrong between the promise and the action. Yet, things almost-always go wrong because policy outcomes are often out of the control of policymakers. We like to pretend that governments are infallible so that we can hold them responsible and blame them for being fallible.
Consequently, a key question about policy success is this: how far would you go to achieve it in each case? Would you sacrifice one aim for another? How do you prioritise a large set of aims which may not be compatible with each other? Would you accept the unintended consequences of a too-rigid attachment to a policy aim? Or, would you set a broad strategy and accept that implementing authorities should have considerable say in how to carry it out?
In this sense, it is possible to succeed and fail simultaneously – either by successfully achieving a narrow policy aim but with unintended consequences, or by accepting a level of defeat for the greater good.*
*Or, I suppose, if you are not of the bright-side persuasion, you can fail and fail.
Further Reading: class sizes