How can you tell the difference between policy-based-evidence and evidence-based-policymaking?

‘Policy-based evidence’ (PBE) is a great phrase because you instantly know what it means: a policymaker decided what they wanted to do, then sought any old evidence to back up their policy.

‘Evidence-based policymaking’ (EBPM) is a rotten phrase largely because no one knows or agrees what it means. So, you can never be sure if a policy was ‘evidence-based’ even when policymakers support apparently well-evidence programmes.

The binary distinction only works politically, to call something you don’t like PBE when it doesn’t meet a standard of EBPM we can’t properly describe.

To give you a sense of what happens when you try to move away from the binary distinction, consider this large number of examples. Which relate to PBE and which to EBPM?

1. The evidence on problems and solutions come first, and policymakers select the best interventions according to narrow scientific criteria (e.g. on the effectiveness of an intervention).
2. The evidence comes first, but policymakers do not select the best interventions according to narrow scientific criteria. For example, their decision may be based primarily on economic factors such as value for money (VFM).
3. The evidence comes first, but policymakers do not select the best interventions according to narrow scientific criteria or VFM. They have ideological or party political/ electoral reasons for rejecting evidence-based interventions.
4. The evidence comes first, then policymakers inflate the likely success of interventions. They get ahead of the limited evidence when pushing a programme.
5. Policymakers first decide what they want to do, then seek evidence to back up their decisions.
6. Policymakers recognise a problem, the evidence base is not well developed, but policymakers act quickly anyway.
7. Policymakers recognise a problem, the evidence is highly contested, and policymakers select the recommendations of one group of experts and reject those of another.
8. Policymakers recognise a problem after it is demonstrated by scientific evidence, then select a solution built on evidence (on its effectiveness) from randomised control trials.
9. Policymakers recognise a problem after it is demonstrated by scientific evidence, then select a solution built on evidence (on its effectiveness) from qualitative data (feedback from service users and professional experience).
10. Policymakers recognise a problem after it is demonstrated by scientific evidence, then select a solution built on their personal experience and assessment of what is politically feasible.
11. Policymakers recognise a problem without the use of scientific evidence, then select a solution built on evidence (on its effectiveness) from randomised control trials.
12. Policymakers recognise a problem without the use of scientific evidence, then select a solution built on evidence (on its effectiveness) from qualitative data (feedback from service users and professional experience).
13. Policymakers recognise a problem without the use of scientific evidence, then select a solution built on their personal experience and assessment of what is politically feasible.

I could go on, but you get the idea. Things get more confusing when you find combinations of these descriptions in fluid policies, such as when a programme is built partially on promising evidence (e.g. from pilots), only to be rejected or supported completely when events prompt policymakers to act more quickly than expected. Or, an evidence-based programme may be imported without enough evaluation to check if it works as intended in a new arena.

In a nutshell, the problem is that neither description captures these processes well. So, people use the phrase ‘evidence informed’ policymaking – but does this phrase help us much more? Have a look at the 13 examples and see which ones you’d describe as evidence informed.

Is it 12?

If so, when you use the phrase ‘evidence-informed’ policy or policymaking, which example do you mean?

For more reading on EBPM see: https://paulcairney.wordpress.com/ebpm/