How can you tell the difference between policy-based-evidence and evidence-based-policymaking?

‘Policy-based evidence’ (PBE) is a great phrase because you instantly know what it means: a policymaker decided what they wanted to do, then sought any old evidence to back up their policy.

‘Evidence-based policymaking’ (EBPM) is a rotten phrase largely because no one knows or agrees what it means. So, you can never be sure if a policy was ‘evidence-based’ even when policymakers support apparently well-evidence programmes.

The binary distinction only works politically, to call something you don’t like PBE when it doesn’t meet a standard of EBPM we can’t properly describe.

To give you a sense of what happens when you try to move away from the binary distinction, consider this large number of examples. Which relate to PBE and which to EBPM?

  1. The evidence on problems and solutions come first, and policymakers select the best interventions according to narrow scientific criteria (e.g. on the effectiveness of an intervention).
  2. The evidence comes first, but policymakers do not select the best interventions according to narrow scientific criteria. For example, their decision may be based primarily on economic factors such as value for money (VFM).
  3. The evidence comes first, but policymakers do not select the best interventions according to narrow scientific criteria or VFM. They have ideological or party political/ electoral reasons for rejecting evidence-based interventions.
  4. The evidence comes first, then policymakers inflate the likely success of interventions. They get ahead of the limited evidence when pushing a programme.
  5. Policymakers first decide what they want to do, then seek evidence to back up their decisions.
  6. Policymakers recognise a problem, the evidence base is not well developed, but policymakers act quickly anyway.
  7. Policymakers recognise a problem, the evidence is highly contested, and policymakers select the recommendations of one group of experts and reject those of another.
  8. Policymakers recognise a problem after it is demonstrated by scientific evidence, then select a solution built on evidence (on its effectiveness) from randomised control trials.
  9. Policymakers recognise a problem after it is demonstrated by scientific evidence, then select a solution built on evidence (on its effectiveness) from qualitative data (feedback from service users and professional experience).
  10. Policymakers recognise a problem after it is demonstrated by scientific evidence, then select a solution built on their personal experience and assessment of what is politically feasible.
  11. Policymakers recognise a problem without the use of scientific evidence, then select a solution built on evidence (on its effectiveness) from randomised control trials.
  12. Policymakers recognise a problem without the use of scientific evidence, then select a solution built on evidence (on its effectiveness) from qualitative data (feedback from service users and professional experience).
  13. Policymakers recognise a problem without the use of scientific evidence, then select a solution built on their personal experience and assessment of what is politically feasible.

I could go on, but you get the idea. Things get more confusing when you find combinations of these descriptions in fluid policies, such as when a programme is built partially on promising evidence (e.g. from pilots), only to be rejected or supported completely when events prompt policymakers to act more quickly than expected. Or, an evidence-based programme may be imported without enough evaluation to check if it works as intended in a new arena.

In a nutshell, the problem is that neither description captures these processes well. So, people use the phrase ‘evidence informed’ policymaking – but does this phrase help us much more? Have a look at the 13 examples and see which ones you’d describe as evidence informed.

Is it 12?

If so, when you use the phrase ‘evidence-informed’ policy or policymaking, which example do you mean?

For more reading on EBPM see: https://paulcairney.wordpress.com/ebpm/

7 Comments

Filed under Evidence Based Policymaking (EBPM), public policy

7 responses to “How can you tell the difference between policy-based-evidence and evidence-based-policymaking?

  1. Pingback: ‘Evidence-based Policymaking’ and the Study of Public Policy | Paul Cairney: Politics & Public Policy

  2. Pingback: Coming to grips with with evidence-based policymaking: what do we need to know? | The Power To Persuade

  3. Pingback: We are in danger of repeating the same mistakes if we bemoan low attention to ‘facts’ | Paul Cairney: Politics & Public Policy

  4. Pingback: We are in danger of repeating the same mistakes if we bemoan low attention to ‘facts’ | The Knowledge Exchange Blog

  5. Pingback: Governments think it’s OK to use bad evidence to make good policy: the case of the UK Government’s ‘troubled families’ | Paul Cairney: Politics & Public Policy

  6. Pingback: Evidence Based Policy Making: 5 things you need to know and do | Paul Cairney: Politics & Public Policy

  7. Pingback: How can political actors take into account the limitations of evidence-based policy-making? 5 key points | Paul Cairney: Politics & Public Policy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s